tl;dr Google may have money and YouTube may be one of their main sites, but there's always more to development than "but c'mon, they can afford it" or "surely they have enough engineers".
I understand what you're saying but based on what we've heard from ex-Googlers about user interface/experience design within the company, that people who've not got the first idea about good design are thrust into the role of designing user interfaces and user-facing features, I think it's a minor miracle we have what we have already — and what we have isn't great.
Plus, look at the variety of users and devices on YouTube. From those who watch 5K video on 27" high-resolution monitors backed by grunty graphics and powerful processors, to cheap $29 Android phones that barely keep the battery from draining to zero within a couple of minutes just showing a 360p video, and all the devices in between.
Between Google's issues with user interface/experience design on a company level and the fact that YouTube needs to cater to so many different kinds of people and devices, there's clearly more than meets the eye when it comes to the hidden scale of YouTube.
Since Google basically prints money, I don't think money could be the issue. Since Google is overflowing with engineers, I don't think throwing more engineers at the problem would help. Rather, I think that removing annotations is a good sign that Google is trying to do something smarter: manage the scale of YouTube.
The removal of just one feature, like annotations, lifts a heavy burden from the main site team, the mobile site team, the mobile apps teams, and everything else. It brings platform consistency, meaning some actually new and innovative features can finally get some of that precious engineering time, rather than the inelegance of supporting what was always a janky feature that could only become jankier as devices became more diversified.