Hacker News Reader

Comment by tekkk | Hacker News Reader

Interesting architecture. Asynchronous message passing at scale becomes quite a PITA very quickly. Wonder how scalable the code base really is. Probably Java isn't that bad of a choice considering the tooling although it's not my personal favourite.

Seems quite the ideal use-case for Elixir/Erlang but since haven't used it myself, don't know would it be that much better. Especially training/developer pool -wise.

And Vitess/MySQL for persistence? No Cassandra or something similar?

View on HN

Slack started out on LAMP stack, so a lot of their choice of technologies stem from there. There's a mention of using Hack in this article, which is a PHP dialect. People frequently reach for things more complicated than MySQL, but the truth is that there are likely very few use cases that won't work with MySQL. Vitess makes MySQL manageable at scale (YouTube uses it too) as long as you invest in getting the tooling & automation ready. Vitess itself is fairly straight forward to understand and operate. It's ultimately just MySQL + a side car service and a query layer that handles reading/writing to shards for you, and some other admin tools for managing shards, migrations, etc. If you need to debug something you can just SSH into a host and connect directly to the mysql instance. It uses standard mysql replication and you can tune it the same way you would tune mysql normally. There's no shortage of resources available when it comes to dealing with mysql.

> YouTube uses it too

Pretty sure YouTube migrated onto Spanner quite a while ago.

Vitess increases the number of things that can go wrong in a high traffic MySQL cluster by an order of magnitude. When you need it you need it but “straightforward to operate” only describes vitess in the most abstract of theoretical happy path situations.

That's true for any database at that scale

Discord uses Elixir. They're fans.

https://elixir-lang.org/blog/2020/10/08/real-time-communicat...

Notably, with regards to the hiring pool:

> None of the chat infrastructure engineers had experience with Elixir before joining the company. They all learned it on the job.

Seems to me like their Channel Servers pretty much does the work of what Cassandra would do, with Consistent Hashing and their Consistent Hash Ring Managers, although it's not detailed how the Channel Servers persist their data and if like Cassandra, they have replication to N Channel Servers. Looks like a specialized variant of how Cassandra works for their use-case, possibly without all the drawbacks of the underlying data structure Cassandra uses for persistence (SSTables), like needing exact ordering of clustering keys when performing queries or dealing with tombstones.

I could see these Channel Servers using something like SQLite for persistence, which would allow the full suite of SQL features without any of the drawbacks of CQL, of which there are many.

The blog post felt a bit high-level and lacking some technical/architectural details that would've given more insight, but nonetheless it was an interesting read!

Something along like this came to my mind but was unable to put it into words! Good you got it.

> Wonder how scalable the code base really is.

Are you for real? This isn't an hypothetical codebase. This is slack, used by millions of people.

I get it, they're using boring tech, but come on.

It’s a valid point. We might discover FTL and suddenly slack has to work at the federation level and serve trillions of people. This is such a poor design and won’t scale at that point.

You can scale even a stack made out of machine code if you just keep throwing money at it. I guess I should have phrased the question, how maintainable and tech debt free the code base is. It's obvious that scalable is in how many people are served, is apparent from the context. I didn't think there was ambiguity in that.

It is definitely an area in which Elixir excels. My personal feeling, however, at this advanced age of late 30s, is that I just can't do dynamically typed languages for large projects anymore. A statically typed Elixir would be a dream I think. There is https://gleam.run/, but I don't know how mature it is.

There is active work being done currently to add set-theoretic types to the core language. https://www.youtube.com/watch?v=gJJH7a2J9O8

Oh wow I wasn't aware. Thanks for the link!

MySQL can scale extremely well if you tune and use it correctly - Twitter was still backed primarily on MySQL at least through Obama’s second election, and I’m pretty sure for some time after that.

What makes you recommend Cassandra here over the other?

Reply on news.ycombinator.com