Gmail HTML Basic view is a breath of fresh air

So it's just the usual Gmail link but with /h on the end?

I went looking for something in the settings page for this, but that's actually the official way to get to it, according to https://support.google.com/mail/answer/15049?hl=en

Yeah, it's not in the settings. I got to it first by clicking the "HTML Basic view" link at the bottom while Gmail is still loading (you have to be quick). Then I noticed that the only difference in the URL is the extra /h in the end.

In addition to being a lot simpler, this is also a godsend when accessing gmail from countries with strict firewalls over eg ssh tunnels...fewer resources, less overhead, more reliable

Why did things have to go so wrong?

In a way, the standard view has a lot more functionality. But the UX is horrible IMO. I'd rather have the old simple UX with the additional functionality.

Money, data harvesting, and advertising.

Usually the answer to your question can just be distilled down to: Money.