I think the primary factor was simply that Google switched from promoting Firefox to promoting Chrome after its release in late 2008, so once any inertia in the public mind wore off, they couldn't keep up. The decline in IE continued, but it started going to a different place.
The other dominant thing in that time frame was the rise of mobile. Mozilla didn't control a mobile platform, so they were second place out of the gate on Android in a way they never were on Windows where both Chrome and Firefox had to be downloaded manually. They also didn't have a well-developed account ecosystem, so as syncing between desktop and mobile browsers became a thing, that was another nail.
The trend line seems pretty unchanged since then.