Hacker News Reader

Comment by adamseabrook | Hacker News Reader

My company runs a largish crawling infrastructure. We evaluated Amazon extensively but just could not make the price/performance calculation stack up. If you have lots of "always on" gear dedicated works out significantly cheaper. We have boxes with 256GB of RAM, 300mbps network pipes for under $330 per month and we don't have to share any CPU or Disk I/O.

View on HN

Likewise. My company decided to lease a rack at a colo, unlimited bandwidth (practically speaking) and we can add capacity as necessary.

We're planning to (eventually) utilize Amazon to temporarily add capacity when necessary, but it will never be our primary hosting solution.

That is the one thing we do miss. Elastically scaling dedicated boxes that can often take 12 hours from order to go live means we end up with a few hot spares laying about just in case but overall it is still cheaper.

Wow. That's a lot of bandwidth. I re-wrote the downloaders for siteadvisor a few years ago (hosted) and they didn't need anything close to that. They were just downloading EXEs and other content from pages the crawlers had already seen, but still.

Those servers run our ElasticSearch index so move TB of data around inside the network and out to customers. Our actual crawlers could happily sip from a 5mbps pipe.

Yep. Many businesses should (and a lot don't...) make this cost comparison. Also, 256GB of memory in a server is surprisingly not that impressive (even for a 1U server) these days! Crazy to think there are 1U machines you can cram 1TB of memory into now.

with amazon you're paying for software, not hardware. big difference.

What hosting provider are you using?

We use OVH in Montreal.

Reply on news.ycombinator.com