Tips for running HBase on low-memory EC2

dev.hubspot.com

18 pointsgsabo2 comments

Thanks for writing this. We're also running a large HBase cluster on EC2 and frustrated with the poor suitability of instance offering (there's literally nothing cost effective for HBase in the current generation).

I am curious why you chose c1.xlarge over m1.xlarge, which is cheaper and has double the memory. In my experience HBase tends to be more memory bound than CPU bound.

We did initially use m1.xlarge, a few years ago when we first started with HBase. I no longer have the hard numbers available, but while I agree HBase is memory hungry, our m1.xlarge servers were constantly pegged for CPU. For a while we ran both types, and eventually phased out the m1.xlarges entirely.

We run pretty frequent compactions due to our read and write patterns, so of course if you do not need to compact as often you might get much better mileage out of the m1.xlarge.

Reply on news.ycombinator.com