I saw a talk by a product manager at the Guardian, and as I understand it, that transformation was a reaction to declining print sales, shaky ad revenue, the rise of Facebook & Google, the success of sites like Buzzfeed & Vox, etc. They've been losing money for a long time and have been slashing costs as much as they can, but that doesn't make a business sustainable.

Nowadays, most people will get to a news article from a friend sharing it on facebook, or from reddit or hacker news, etc, not the Guardian homepage, and that means clickbait pays the bills.