But then the website might not look like the designer's glistening perfect vision, that only ever worked properly in their browser on their desktop in the first place.
* Some number of people are sensitive to latency
* Some people are unable to read my website
You're gambling that there are more people who would walk away due to latency, than there are who would walk away due to being unable to interact with your site at all.
Personally I suspect that to be a poor tradeoff.