Comment by oinksoft | Hacker News Reader

Comment by oinksoft | Hacker News Reader

Parent Full thread

You're underestimating the difficulty of scaling Mnesia the way WhatsApp did (one source says "In-memory Mnesia database using about 2TB of RAM sharded across 16 partitions to store about 18 billion records." [1]). The prevailing wisdom in the Erlang community was that Mnesia could not be made to do such things. The only well-known user of Mnesia at anywhere near this scale prior to WhatsApp was Klarna. To achieve this, Klarna employed at least one of the original Mnesia developers who wrote proprietary patches for their system [2]. So while you accuse parent of being reductionist, your views of WhatsApp's engineering challenges also seem reductionist. Erlang/OTP software doesn't write and scale itself ... large-scale distributed systems are decidedly non-trivial no matter what tools you've chosen.

[1] http://highscalability.com/blog/2014/3/31/how-whatsapp-grew-...

[2] https://www.erlang-factory.com/upload/presentations/601/EUC2...

When I checked this post, you were below zero - I'm not sure why you are downvoted.

In any case, my main point isn't that that WhatsApp is simple. When I said the parent posted was being reductionist, I was referring to Twitter's product offering. AFAIK (I could be wrong) WhatsApp is a messaging backend and multiple client frontends. The "50" engineers is deceiving becaus, despite the fact that Mnesia could or couldn't have scaled, there is a very non-trivial amount of engineers who worked on it and Mnesia is a good & proven system. In a similar light, Instagram was 13 engineers (or was it employees) and hundreds of django instances - and that maybe all you need for a photo feed, however Instagram didn't have a trending feature until earlier this year and how knows how many engineers that took.

If WhatApp hired another 200 engineers, likely 195 would either be twiddling their thumbs or working on new features outside what was needed in that acquisition to Facebook. Now we can make the argument that Twitter is bloated because they have too many products (Fabric, Vine, Crashlytics, and their numerous other acquisitions) - and the scaling back of those engineers may mean sunsetting those products - and thats fair. To imply they have 4000 engineers building a glorified 140-char mailing list is not.

Reply on news.ycombinator.com