Hacker News Reader

Comment by pyrrhotech | Hacker News Reader

It's funny to me when people call companies like Twitter "tech companies". What technology has twitter invented? There are open source clones with 70% of the features in < 1k LoC. Bootstrap? give me a break.

It's time we got rid of this misnomer. Just because part of your company revolves around a piece of software you wrote doesn't mean you are a tech company. Writing software is not necessarily creating technology. Otherwise anyone who's written their own html site has just progressed technology.

View on HN

"What technology has twitter invented?"

http://twitter.github.io/ currently lists 101 public repositories. Highlights include a SPDY library for iOS, Zipkin (a distributed tracing system), flockdb (a distributed, fault-tolerant graph database), Finagle (their own highly concurrent RPC system)... a TON of stuff.

No offence meant but that is like saying "Kia is also a car, does the same work. Why the hell even BMW exists"

May be BMW is doing things on the back that make it a better choice. Same is the difference between twitter and its thousand clones.

Do you have any idea the amount of engineering effort required to handle the scale of Twitter's service?

Companies like Twitter, Facebook, etc, regardless of what technology they invent that is of use for the public, often healthily contribute to the open source ecosystem. Twitter has contributed a lot to the Scala ecosystem, and Facebook to the PHP pseucosystem.

They also fund/maintain OS project they themselves use. Facebook, through their FriendFeed acquisition, obtained Tornado [1] and have maintained it very well since. I use Tornado every day.

I get your point but your anger is misplaced if you think the large "SV tech companies" don't "invent". You want tech companies that don't contribute anything to mankind? Look at purely closed source hardware companies.

[1] https://github.com/facebook/tornado

You can write a superficial clone of any web app. But can you handle 8000 tweets/sec in real-time? The innovation is in the back-end.

It's been done notoriously badly at Twitter, no offense if you are an employee. Sure it's gotten better since they switched to the JVM, but still scaling to that size has been done by hundreds of companies now. Once there are textbooks out describing the techniques, you are simply implementing known methodologies with perhaps a few customizations and tweaks. I don't really see much innovation there TBH.

I'm neither an employee nor a twitter user. But I've built phone networks and appreciate the absurd complexity of very large systems. If you believe you can build twitter, you're either a brilliant programmer or ignorant of the real challenges. Look at Twitter's code on github and tell me that's just a simple tweak on a known methodology.

Twitter, Facebook, and LinkedIn (along with other big tech companies) all produce a lot of academic research that documents the architectures and protocols that they use, along with their performance. These companies innovate, and the material from their research often makes it into these "textbooks ... describing the techniques".

Lets not trivialize a truly difficult problem just cause mongodb has some huge numbers on their marketing literature. Its naive to believe that scaling to the level of twitter is easy.

Sometimes you have to be innovative just to implement simple ideas in those text books or to make it actually efficient. Its easy to use type 1 UUIDs as ids but if its not practical to store 128bit ids for every value or you need to pack it into 64bits or need better ordering guarantees. you build something like snowflake.

Surely you would consider at least some of Twitter's open source projects "creating technology": https://github.com/twitter

As much as I find many of the past decade of Web / tech companies underwhelming, innovation can come from unusual places. Duff's Device, a method for unrolling loops in C for better performance, was created at Lucasfilm.

The challenges of scaling to very high levels has presented engineering challenges and solutions. I'm not sold that they're net beneficial in the greater order of things, but to say that there's been no significant innovation would be false.

Twitter is one of the main contributors to Storm:

http://storm.incubator.apache.org/

Reply on news.ycombinator.com