The beginning of the end was the release of the "New Xbox Experience" in 2008, which introduced a new rich and very cumbersome UI that was way heavier on resources, more confusing to navigate, and dedicated more real estate to ads and "sponsored content".
If Microsoft cared about speed and usability, they should have reversed course. Kill NXE, restore the original snappy Blades UI, and backport the one useful NXE feature (the ability to install disc games to the HDD). Nobody's going to care about 3D avatars if they make your game machine take three times longer to boot.