Ask HN: Setting up a cluster to learn Hadoop (using Amazon EC2)?
Thanks in advance!
Thanks in advance!
If you want to learn the "ops" side, EC2 is great for building clusters, tearing them down, testing different configurations, etc.
If you just want to learn how to start using Hadoop, writing jobs - don't even bother setting up a cluster. You can use it directly with Eclipse to run a single-node setup on your machine. Way less complexity and much easier to start learning with.
If you want to get practice with map-reduce patterns themselves, but not spend much time worrying about the ops associated with setting up hadoop clusters, you can play with Elastic-Map-Reduce (EMR) on AWS.
There is even a great python-based open source tool called mrjob (http://packages.python.org/mrjob/index.html) made by some folks at Yelp that makes using EMR really easy. There are a bunch of great tutorials already out there.