Also, geography: ping time from Singapore to the US can never be negligibly short, which is pretty noticeable during TLS handshakes.
But yes, likely these considerations do not matter for 90% of websites (not 99.9% though).
The lowest latency solution is to put the content near the user and a CDN is probably the easiest way of doing that if someone needs to server a geographically dispersed audience
CloudFlare is free at my tier and gives me the ability to have the lowest latency.
What exactly leads you to believe you can tell what 99.9% of all websites need?
Unless you believe 99.9% of websites are only accessed by your upstairs neighbors, CDNs provide a couple of important business and operational advantages.
Just to illustrate how wrong and misguided your personal assumption is, CDNs are primarily used to cut down latency, which in accesses from other regions can easily go beyond 300ms. Unless you somehow think that it's ok for your users to be subjected to a bad user experience, a basic CDN service is all you need to employ to lower those latencies by an order of magnitude.
I accept that my images are being served a bit slower but with the compromise that I can server a lot more images.
I also think the person your are replying to is correct. The CDN makes the user experience worse but saves me money, that’s an unfortunate sacrifice.