Over on Windows, Nvidia has some hacks in the driver to work around this. With a $70 current-gen card, I can drive two 4k monitors at 60Hz. One monitor connects via two HDMI cables; the other over Displayport MST. (I don't actually have two 4k monitors, but I did convince my computer that my one monitor was two to test this.)
You're right about getting 4 monitors working. Probably possible with an SLI setup, but the current generation of video hardware makes 1-3 monitors easy and 4+ monitors hard. 3 monitors is better than a few years ago, though.