Mnesia worked well enough for us at WhatsApp while I was there; although we didn't use it to store messages; long term message storage is on the end points (generally sqlite), messages in transit (offline) were stored in a file per user with the import/export written in C IIRC. We did add redundant in memory storage of messages in transit; but I don't remember the storage there; may have just been ets.

We mostly used mnesia as a replicated key-value store, but we got a lot of value from having the data and the business logic colocated. Other nodes would send logical operations to processes on the mnesia node and those processes could run each operation one at a time on the data. Any concurrent logical requests for a given piece of data were implicitly serialized by the process mailbox. But almost all of our data was easy to shard, no high volume operations needed to address multiple tables.

We heard a lot of things about mnesia scalability limits that just didn't match up with our experience; so I don't know what other people were doing, but you can get a glimpse of what we were doing in the Rick Reed talks at Erlang Factory. We certainly had some scalability challenges, but many (most?) are discussed in those talks; and my general recollection is that most of them were more like we were the only people running mnesia with tables of enormous size, so we had to make things work; but that's kind of how OTP is. The trickiest one to find, IMHO, was that IIRC mnesia_frag and ets (and our request sharding) all use(d?) the same hashing function, so adding more fragments would make distribution of keys per ets slot worse, ets wanted power of two slots, and would split based on average keys per slot, but we would have lots of keys on some slots and no keys on most slots. Changing the hash seed for ets was a 2 line code change that drastically improved performance on all of our sharded mnesia systems.

Another fun one is that if you use mnesia to store data for long periods, you have to be very careful with the binaries you store; it's easy to end up with refc binaries that have extra space for append operations; storing them in mnesia means that append space is allocated but unusable; you might also store a sub binary that's a small part of a refc binary, the underlying binary can't be disposed of until the sub binary is. For both of those cases, cleaning the binaries before storing them with binary:copy/1 can really reduce your memory use. There's probably some cases where you do want to store a sub binary though?

Mnesia doesn't (or didn't) include a good way to handle when two mnesia nodes sharing a schema disconnect and reconnect. We mostly solved that by ensuring our network was stable enough that that rarely happened. If your network is not stable, you will have a bad time with distributed Erlang in general, and Mnesia in specific.

If I were building Twitch but smaller, and on the BEAM, I would absolutely put account databases in Mnesia; but messages and media would probably live as files. I wouldn't tend to put those into a SQL database on a server either though.

That is not true. I dont use Mnesia because it is so different and thus harder for me than sql. But it is used. Klarna uses it. They are a pretty big company. Would I recommend it for most? No, but I also would not recommend cassandra and a bunch of other databases for most apps

And when you do have to jump off, you'll be screwed. Because the jump off is usually from in-memory Mnesia with guaranteed sub-millisecond responses to a proper database with at least a magnitude higher latency. And you realize that your app is completely dependent on sub-millisecond responses :)