Hacker News Reader

Comment by kiwidrew | Hacker News Reader

Why would you bother going to all this trouble when you could just use Amazon's official Spot Fleet service/API?

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-flee...

He started before the feature was released and acknowledges they came up with an in-product solution:

"and it seems they now have a full fledged solution for the problem, based on pretty much a reimplementation of AutoScaling, using machine learning and with a beautiful UI and they are really successful with it. Funnily enough, they even contacted me to sell that solution to my company and we are seriously evaluating it"

He's talking about Spotinst.com, not Amazon's fleet solution.

Ah, okay, I missed that part!

Yes, but I actually did start a few weeks before the spot fleet was launched.

The problem with the spot fleet 1) it's kind of awkward to use 2) it has statically defined capacity so you can't scale it 3) it has a static bid price, so if at some point your are outbid on all the group's bids, you end up with no capacity 4) among other things it lacks integration with the ELB so you can't really use it for so many use cases.

My solution is simpler, better integrated with the rest of AWS and more resilient and once I iron out the bugs and get it production-ready, it should be a better choice.

Spot prices can spike above on-demand prices for some instance types (e.g. g2.2xl). If you wanted to maintain a certain capacity of homogeneous machines, you'd have to notice the failed spot instance request & increase the size of your on-demand ASG to compensate.

Can anyone comment on the crazy spot prices for g2.8xl ? they spike really high (they were $26/hr earlier? 10x the on-demand price. I'm guessing someone with enough market share has a job that doesn't want to be interrupted and they bid the spot price up to 10xon-demand, which seems ridiculous since presumably they could arbitrage /w an on-demand, but i guess that once they start the job they don't want to be interrupted? i'm not exactly familiar with the intricacies of the ec2 spot market. ). Also, are they any good for ML? I'm between machines ATM and would like to experiment with some deep CNN for face recognition. At $2.6/hr for on-demand its a bit more than I would like, but if the spot price was less i'd consider it.

I used them for a while for ML, works ok. I also noticed the these spot price spikes with both g2.8xl and g2.2xl. Not really sure why someone (or multiple players, since the prices spike) have set their max bid so high, surely it's worth running an on-demand instance at those prices.

Ended up going between two regions to avoid them but it's some hassle, just got a gtx970 in the end.

Can you checkpoint your work? Most libs I've used support that generally. That way you only lose a bit of time if kicked off, and you don't pay for the hour you get kicked off so it's even cheaper.

Reply on news.ycombinator.com