Latency can only go down so much because as parent comment said: laws of physics dictate the minimum response time between two points given the distance. And it will be higher with each hop /media change
No, the majority never even noticed. The irreducible latency some are talking about is imperceptible, not a factor for the vast majority of the populus. Ease of use, lack of install times, flexibility, and ability to spread out cost, all will outweigh that.
Lots of people do discuss this and do care, not to mention to even get to the point of it only being “the lowest possible physics allows” you need great home networking equipment plus a fast/well connected isp and a cloud service provider that isn’t adding any processing delay. That excludes a ton of people.
Cloud gaming isn’t going to be “the default” anytime soon, and for many people it will never be an option.
Heck physical copies are still super popular with a large group of console gamers to this day despite digital delivery being so well tested and ubiquitous