Hacker News Reader

I only know Hadoop from reading about it on HN, and the blog post I remember the most is "Don't use Hadoop - your data isn't that big"

https://www.chrisstucchio.com/blog/2013/hadoop_hatred.html

TL;DR

> But my data is 100GB/500GB/1TB!

> A 2 terabyte hard drive costs $94.99, 4 terabytes is $169.99. Buy one and stick it in a desktop computer or server. Then install Postgres on it.

View on HN

There's also this [0] on how to outperform Hadoop with command line tools.

Obviously, you should look at the role I/O costs can play. A lot of RAM and some SSD drives might be a better idea than Hadoop at this scale.

[0] http://aadrake.com/command-line-tools-can-be-235x-faster-tha...

It remains true. 4TB now is like $100 too, and SSDs that size are around even if they're an order of magnitude more expensive. (And then there's Seagate's 60 TB SSD prototype, clearly the future is with SSDs.)

Reply on news.ycombinator.com