The extension ecosystem, tabs, plugins, and notably whatever effort they did behind the scenes to ensure that companies that did streaming video etc. would work with their browser all played out really well.
I think the ultimate problem is that Mozilla's mission of a standards-compliant web with open-source browsers everywhere ultimately did get achieved. The era of "Works with IE6" badges has ended and the top browsers run on open-source engines. Despite our enthusiasm at the time for it, I think the truth is that Firefox was probably just a vehicle for this, much bigger, achievement.
Now that it's been achieved, Mozilla is in the fortunate place where Firefox only needs to exist as a backstop against Chrome sliding into high-proprietary world while providing the utility to Google that they get to say they're not a monopoly on web technologies.
Mozilla's search for a new mission isn't some sign of someone losing their way. It's just what happens to the Hero of Legend after he defeats the Big Bad. There's a post-denouement period. Sam Gamgee gets to go become Mayor of the Shire, which is all very convenient, but a non-profit like Mozilla would much rather find a similar enough mission that they can apply their vast resources to. That involves the same mechanics as product development, and they're facing the same primary thing: repeated failure.
That's just life.