So I believe the complaint about "short-termism" has become a bit too popular. It's great to prop up your bona-fides as a cynic. It's less good at explaining actual behaviour.
Instead, I believe research has simply become less profitable over time. The decades from about 1920 into the 1980 were a time of extraordinary rapid successes in the hard sciences: the scientific method had been established, and industry, finance, law, transport, and communications all made sudden jumps into modernity, allowing scientific institutions as we know them today to exist.
If you read about the history of physics, for example, you'll see photos of the entire class of 192x at some German university, and every single person on the photo later won a Nobel or had at least some minor unit named after them.
We have continued to improve institutions, infrastructure, and our ability to broaden the chance to get into the sciences. But, unfortunately, it's almost a rule of nature that progress slows over time. That's sort-of soothing, actually, because it means we aren't entirely incapable of prioritising easier thing.
Government-sponsored research can continue at higher prices. But private, for-profit endeavours have a very specific point where expected returns turn negative.