1) It's simply not as responsive. They've worked really hard to keep up with Chrome, and I respect that, and the recent overhaul did make a big difference. But Chrome just feels like a piece of physical machinery, where Firefox still feels like a piece of software.
2) The dev tools are just not quite as good. As with general performance, they've always been very close, but there's just something about them that's off. Enough to disrupt my work.
3) Firefox mobile, at least on Android, has weird high-friction scrolling behavior that doesn't match the rest of the OS and feels terrible, and there's no way to turn it off.
I use Firefox for casual browsing on my gaming desktop, but that's about it. I'm rooting for them, but there are just improvements that still need to be made.
On the bright side, Chromium is 99% OSS (as opposed to Android, which is like 70% OSS). I think a fork could become a competitor if Google took the project in a direction that was just really egregiously bad. Let us not forget that Chrome's rendering engine, Blink, was itself a fork of Safari's WebKit.