There are some Firebase specific annoyances to put up with, like the local emulator is not as nice and "isomorphic" as say running postgresql locally.
But the main problem (and I think this is shared by what I call loosely "distributed databases") is you have to think really hard about how the data is structured.
You can't structure it as nicely from a logical perspective compared to a relational DB. Because you can't join without pulling data from all over the place. Because the data isn't in one place. It is hard to do joins both in terms of performance and in terms of developer ergonomics.
I really miss SELECT A.X, B.Y FROM A JOIN B ON A.ID = B.AID; when using Firebase.
You have to make data storage decisions early on, and it is hard to change you mind later. It is hard to migrate (and may be expensive if you have a lot of existing data).
I picked Firebase for the wrong reason (I thought it would make MVP quicker to set up). But the conveniences it provides are outweighed by having to structure your data for distribution across servers.
Instead next time I would go relational, then when I hit a problem do that bit distributed. Most tables have 1000s of records. Maybe millions. The table with billions might need to go out to something distributed.
Market gap??:
Let me rent real servers, but expose it in a "serverless" "cloud-like" way, so I don't have to upgrade the OS and all that kind of stuff.