Also it's not just about speed, but power consumption. Once you are fast enough to hit the monitor frame rate then further performance improvements won't improve responsiveness, but you may notice your battery lasting longer. So there's no such thing as "fast enough" when it comes to rendering. You can always benefit from going faster.
This is not true, if your rendering is faster then you can delay the start of rendering and processing of input to be closer to the frame display time thus reducing input latency.
Of course for e.g. games that breaks if the font size changes, letters rotate and/or become skewed etc.
Most GPUs dispatch pixel shaders in groups of 4. If all your triangles are only 1 pixel big then you end up with 3 of those shader threads not contributing to the output visually. It's called 'quad overdraw'. You also spend a lot of time processing vertices for no real reason too.
https://sluglibrary.com/ implements Dynamic GPU Font Rendering and Advanced Text Layout
On the other hand, if you are rendering to an atlas anyway then you don't really need to bother with a GPU implementation for that an can just use an existing software font rasterizer like FreeType to generate that atlas for you.