I suspect they could theoretically allow background play for other content but that would be confusing to users.
Even when I've used YouTube for music, the ads are always pre-roll. If I watch the pre-roll ad, why not let me background at that point?
Google could allow backgrounding for content with no ads. They could allow backgrounding of non musical content if you've watched the ad (they can detect if it's music and what song it is, that's how they pay royalties to labels on songs uploaded by random people). They could allow backgrounding of musical content if you've watched the ad.
YouTube doesn't attempt to handle any of these scenarios. Handling these scenarios would involve an investment of software and possibly legal resources — only to improve the experience of millions and millions of users, and to conserve tons of energy, without making Google any money, and even potentially reducing revenue to YouTube Red (or Premium or whatever it is next week). Why would a company in a monopoly position do that? I get WHY Google won't do this. It's not about licensing, it's about money and motivations. But I'm not going to pay into it.