Startup now 100% on AWS (they're using AMQP, PostgreSQL, pgpool-II and Rails) | Hacker News Reader

Startup now 100% on AWS (they're using AMQP, PostgreSQL, pgpool-II and Rails)

aws.typepad.com

58 pointsspif28 comments

Believe it or not :) there are other uses for servers than web hosting. I can see why amazon publicise these stories, but it's probably not the idea use case.

If you're doing number crunching, where you need 100 machines to get results in an hour, and then you're going to spend 3 days thinking about the result, then EC2 is perfect - the key is that there is not cost difference between using 100 servers for one hour, and 1 server for 100 hours. Slicehost doesn't offer that.

Of course for this to be useful you need a problem that can use 100 machines in parallel, but if that's the problem you have it's perfect for EC2.

So, I guess I'm left wondering what makes AWS so much better for this purpose. Slicehost (and others) similarly offer Xen instances going up to 15GB in size. The big problem I have with AWS is the high cost of bandwidth.

With Slicehost, I can get a 2GB server with 80GB of persistent storage and 800GB of bandwidth for $130. With EC2, 800GB of transfer would cost $136 alone, plus $72 for the server, plus around $35 for the disk space and I/O charges. $243 > $130 by a long shot.

I really want to like EC2, but it always seems expensive. Plus, I always feel that elastic IPs are not heartbeat which other Xen providers do work with.

Maybe AWS is better because there's an implicit agreement that there will always be bandwidth or storage as needed when one thinks that's more of a question mark with other providers? Maybe there's an implicit thing that if you need to, on very short notice, fire up 10-15 Extra Large instances, the capacity will be there while others might not have the hardware?

Maybe someone has more experiences with high volume sites on these services?

Yeh, but if that $243 was your "base" price, it'd only cost $72 or what ever to add a new server, or of course you could just use it for a few hours when needed. Elasticity REALLY is the strong point of EC2, with the ability to create your application to automatically scale (and de-scale) on its own, and only pay for what you use.

You can cancel a Slicehost server at any time, but you pay for a day if you only use an hour, and the refund is credited to your account as far as I know -- if you need 100 servers for an hour, even 256 slices, its going to cost you $2000, with $1934 credited to your account afterwards, compared to $10 on EC2. If you have money to splash around (your not really spending $2000 for 100 server hours, but you still need it), you'd proberly be better with Slicehost (or some other VPS host), but if you don't, EC2 is a very good option for scaling on the cheap.

"EC2 is a very good option for scaling on the cheap."

Until you start using serious bandwidth. Then it's extremely expensive.

As you say, if you have a massive spike for an hour or something due to the nature of your service, maybe AWS makes sense rather than paying for the full day, but I think that's an extremely rare usecase.

Also if you're using a language/setup that isn't too efficient, then AWS may be a second best option to switching language/framework etc.

Sure, but that bandwidth is very flexible also; you GET 800GB bandwidth mentioned in the above plan, but what if you just need 400GB...or 1600GB? AWS will scale from your launch day till the day you go IPO, or at least are at the stage when you can buy terabytes of bandwidth in bulk, cause you know you'll use it.

Really don't agree with the last part though...unless your code is really slow, a rewrite is only going to waste your time and focus your attention on the rewrite rather than developing your business. Defiantly improve as you go...most of us are geeks, we'd have a massive grin on our face if we saved a few milliseconds from each request, but you should only do it if it's actually worth it! :)

I'm more meaning that changing architecture/language may mean you can run the entire thing on a single server, rather than 10.

Doing that sort of optimization helps a lot with scaling later on.

Not if that sort of optimazation means that you can ONLY run it on a single server!

Where do people get the notion that EC2 bandwidth is expensive from? Pricing is $0.17 per GB for first 10,000 GB and drops after that. $0.17 per GB is actually pretty cheap, and unless your shipping large amounts of high quality video, you are unlikely to rake up a big bandwidth bill.

Slicehost (which I am a customer of and have absolutely nothing against) seems cheap if you look at the bundled bandwidth, but they can do that on the basis that most people (e.g. me!) don't use anything like their quota. If you want more on top, their FAQ says that pricing for extra bandwidth starts at $0.30 per GB.

And linode is $0.10 per GB.

"Also if you're using a language/setup that isn't too efficient, then AWS may be a second best option to switching language/framework etc."

What? The monthly cost of a server is equivalent to a few hours of dev time. That's money better spent on making new things, not premature optimization.

I'm all for easy fixes that speed things up, but you rarely need to switch language/framework to do it. Doubly so because slower languages like Python and Ruby make it easy to plug in bits of fast C. More likely than not, someone has already done that, and you just need to start using a better library.

It depends what you're making. Some webapps might need 10 servers if you write it in php or ruby on rails, but may only need 1 server if you write it in c/java/something else.

For example, I could probably run Mibbit on around 50 servers if I just used apache/python/php/ror. And as far as I know that's pretty much what Meebo did. They had the money to throw at hardware. The alternative though is to re-architect and optimize at the early stages.

It's a longer term investment. I've spent a lot of time in my case, but I've saved a lot of scalability headache and cost, later on.

"Some webapps might need 10 servers if you write it in php or ruby on rails, but may only need 1 server if you write it in c/java/something else"

@axod, this statement is completely out to lunch. I don't even know where to start.

You don't believe a custom written lower level implementation would be more efficient than using a general purpose framework stack?

I think he believes network or disk i/o becomes the dominate factor long before cpu for 99% of webapps.

I'm not talking about the speed at which code executes, I'm talking about how many users you can support with a single server. Or how many servers do you need to support N users.

axod, you set yourself up with the 10 to 1 comment. I work every day in C# and we are currently moving all development to Java, at home I code in ruby. I will tell you from experience that your language plays a very small part in how fast your web app works. I will say without a doubt that programmer competence is the real factor. How the code is written will determine more how it runs. But for the sake of argument I can guarantee that if you wrote a web app the exact same way in Java and then ruby, or basically any modern language, one would not consume 10 times the resources of the other. Other factors will start to bottleneck you long before that, bandwidth, memory, cpu, etc.

I'm not talking about pure language choice. When you write php, you probably have to put up with apache. Maybe you also put up with using a database. It's those things that can limit your scalability.

As a slight tangent: for other purposes, one of the significant differences (not mentioned in other comments) is elastic block store (EBS) and free access to S3.

There's nothing akin to EBS at slicehost. The closest I've seen out there is the 300MB/s NAS at appnexus.com (which also runs Xen VMs).

It's hard to find anything like S3 (nirvanix is one, but they've had "issues" reported in the past). And sure you can access S3 from outside, but then you're paying the exact bandwidth charges you have at EC2 anyhow. Depends what you app does, S3 is not just for getting files to end users obviously.

If high traffic to the internet is your biggest problem, this $199 deal seems attractive: 4GB RAM, 250GB drive, 10 terabytes bandwidth a month -- http://www.10tb.com

Getting around whatever convenience VMs provide to take advantage of something like that seems worth it in some cases.

"it all depends"...

For our startup the ability to scale up and down on instances is what makes Amazon great. We often run just 4 instances, but ramp up to 10+ at times when we're busy. This dramatically reduces the cost of having hardware which is underutilized. Not sure of anyone else doing anything similar - is there?

We (Stormpulse.com) are on EC2 and love it for the same reason. Our traffic is insanely spiky.

linode is even cheaper for bandwidth. (1/3rd the price of slicehost).

AWS is cheap and fine for when you don't have much traffic, but I really wouldn't tie myself into that whole ecosystem so early on.

Well if you modularize your codebase, it's not a huge deal to tie yourself into a specific ecosystem.

And a problem with linode is that they skimp on the storage. We're paying $150/server at MediaTemple a month right now and to upgrade to anything past $60 right now, they want us to move to their $750 package. Ridiculious. Hard drives are so fucking cheap, there is no reason they shouldn't be giving 100GB+ for every package.

Maybe I'm a bit slow, but why is this special?

It allows them to leverage your contacts for marketing purposes?

"Allow us to spam all of your contacts so we can 'go viral'?"

In other news, I've now invested 100% of my money with this nice guy Bernie Madoff. He beats the market every year!

I've never been able to get Rails set up on AWS - had problems with the AMIs etc.

Can anyone point me in the right direction for a LAMP/Rails setup?

@goodkarma, you might try Rightscale management platform. It will give you access to a number of public AMIs with that stack.

Reply on news.ycombinator.com