There really aren't that many modem/router combos out there you can actually flash with open source firmware.
Router? Sure, no problem.
Modem? Outside of some very niche Fritzbox devices and others listed on OpenWRT Table of Hardware
There really just aren't that many.
So, fine. I will use a proprietary modem.
I will put it into bridge mode, let it decode the DSL analog signals into ethernet and feed that into a router of my choice.
Question is: Which modem?
I don't want two devices heating up my shelf. I just want a tiny modem that converts analog to digital into an RJ45 and then let's the router do its thing.
Does such a magical device exist, because my search on this front has proved largely fruitless
Not quite what you're seeking, but maybe relevant.
Have you considered an SFP modem like the Proscend 180T? Might be hard to source these days, but if you have a router with an SFP cage this would be a way to accomplish the single box goal.
Woah... I had never heard of SFP slots/devices before. It looks like they're great at converting between mediums. If I could get an SFP device that had RJ45 output, that would be the dream
Maybe you could use a Mikrotik RouterBOARD, some feature a single SFP port. They're great and inexpensive.
If by RJ45 you mean erhernet over standard 8P8C (i.e. RJ45) your looking for SFP-T modules. Can't connect that to your DSL line though since that's not Ethernet.
There do seem to exist G.fast sfp modules for dsl, but do note that sadly sfp module and router brand incompatibility is sadly a thing.
Related note: At our company we also had quite some issues getting some of these more special sfp modules working (e.g. XGS-PON), so ymmv with a dsl one.
(If you're looking for a device to plug sfp modules into, I can second MikroTik here. They're also great about accepting 3rd party SFPs)
Not sure if they're still in production. There was also an Italian company selling one (maybe based on the same hardware).
Like u/floquant I would look at Mikrotik. But you'll need to pay attention to port speed (they still sell 100Mbps gear) and their tested routing throughput numbers to be sure they can hit the numbers you need.