Hacker News Reader

Comment by pyrrhotech | Hacker News Reader

It's been done notoriously badly at Twitter, no offense if you are an employee. Sure it's gotten better since they switched to the JVM, but still scaling to that size has been done by hundreds of companies now. Once there are textbooks out describing the techniques, you are simply implementing known methodologies with perhaps a few customizations and tweaks. I don't really see much innovation there TBH.

View on HN

I'm neither an employee nor a twitter user. But I've built phone networks and appreciate the absurd complexity of very large systems. If you believe you can build twitter, you're either a brilliant programmer or ignorant of the real challenges. Look at Twitter's code on github and tell me that's just a simple tweak on a known methodology.

Twitter, Facebook, and LinkedIn (along with other big tech companies) all produce a lot of academic research that documents the architectures and protocols that they use, along with their performance. These companies innovate, and the material from their research often makes it into these "textbooks ... describing the techniques".

Lets not trivialize a truly difficult problem just cause mongodb has some huge numbers on their marketing literature. Its naive to believe that scaling to the level of twitter is easy.

Sometimes you have to be innovative just to implement simple ideas in those text books or to make it actually efficient. Its easy to use type 1 UUIDs as ids but if its not practical to store 128bit ids for every value or you need to pack it into 64bits or need better ordering guarantees. you build something like snowflake.

Reply on news.ycombinator.com