AWS Elastic Beanstalk now supports Python | Hacker News Reader

AWS Elastic Beanstalk now supports Python

aws.typepad.com

146 pointsmza32 comments

A couple friends and I just spent the past 48 hours building an app that recreates a lot of this functionality. Some key things to note with this is that you're stuck with RDS, and locked in to amazons platform choice.

If anyone is interested in the work we did this weekend, https://github.com/jawnb/badatcomputers

Can't you use RDS like any MySQL database? And since you're writing a standard WSGI app, it seems like there is very little lock-in. This seems to basically automate the provisioning of EC2 instances for the web app and the DB. and since you just have a standard app, you're free to move it around as needed.

Or is there something that I'm missing with the Elastic Beanstalk? (I've only used raw EC2)

Yes RDS is like any MySQL db. The key word there being MySQL. If you want to use another data store like say, postgres or even some NoSQL solution. You're outta luck.

As for platform lock-in, my comments were less towards the web services level and more about system level concerns. It's not entirely clear how you would run persistent background tasks (read: celery). This doesn't even get into the headaches of learning yet another DSL to define OS packaging requirements.

That being said, IMO this would be great for simple apps, with no strenuous requirements. In addition, anything that allows more people to easily deploy their django apps is a good thing.

RDS isn't just MySQL - you can also use Oracle or SQL Server, if you don't want MySQL.

For NoSQL, look at the other stuff that doesn't have 'Relational' in the name :) (DynamoDB, SimpleDB, etc.)

Yes, RDS works with MySQL. And you're right about the provisioning for the EC2 instances for beanstalk, but it doesn't do much for RDS, that needs to be set up independently.

Has anyone here used Elastic Beanstalk? What were your experiences?

I'm using Elastic Beanstalk with PHP and overall the experience has been fairly positive. I don't really have any ops experience, so I wanted something that could get me up and running quickly. There are some gotcha's, but it's been great. If you use git, there's a git CLI that deploys directly to Beanstalk which is very convenient. Deploying an environment or deploying a new instance of your code to an environment is really a piece of cake. You can set autoscale rules, interface with load balancers, add some other configs, etc. right from the beanstalk interface in just a few clicks. Support from Amazon has been solid, and the documentation/forums are not bad.

Fair warning, Beanstalk is still young so there might be some growing pains, but for someone with little ops experience, it's been pretty awesome. One very important fact to note, is that it's just a layer on top of other AWS services (with no extra fees, you only pay for the underlying resources), so if there's something you can't do on beanstalk, you can do it with whatever resources are beneath it (EC2, RDS, Cloudwatch, etc).

I'd recommend if you're interested in it, or feel AWS is burdensome/complicated, it's worth a try. You have to scope out if it's right for you or your project.

I'm also using Beanstalk for a (big) PHP application, it was a bit of a mess rewriting all data saving to S3, email via SES, routing with Route 53 and such stuff.

Beanstalk is not 100% matured, you see this in some unexpected behaviors like :

- changing some environment vars give unexpected troubles with the load balancers in one environment, but not in another. - restarting an app server sometimes crashes the environment. -etc.

the automatic procedure is that beanstalk starts replacing it's EC2 instances from your Auto Scaling Group, which sometimes has to be repeated for 2-3 times in a row which can take up to 30 mins.

in the meantime you should switch to a backup environment ( which is a must have! )

but all in all im positive because it's getting better all the time and the benefits are despite the disandvantages great.

check out the beanstalk forum : https://forums.aws.amazon.com/forum.jspa?forumID=86

I've been using the Java version of Beanstalk to run a commercial API - http://www.degreedays.net/api/ - for about 12 months.

So far it's been working out very well. I love how you can dip into the EC2 APIs if/when you need to, but it's not usually necessary as the basic setup is excellent (a load balancer in front of one or more EC2 instances - your choice of number, type, and auto-scaling rules).

The tools for updating a Java app are great. Each deployed version of your app gets stored in the AWS system, and you can rollback to any version with a couple of clicks in the AWS control panel.

It's also pretty easy to set up a staging environment to test new versions before deploying them to the live environment. Only issue I have with that is that there's no way to pause a staging environment (like you can pause an EC2 instance and not pay for it while it's paused). So you either have to keep a staging environment running all the time (paying for its EC2 instances), or set one up afresh each time you need it. The latter isn't a huge hassle, but it would be much better if you could just set one up and keep it paused for most of the time.

I think development on Beanstalk is nicely approachable, and in my experience it's been pretty reliable as well, or at least as reliable as EC2. The current version of our app has been running well, with moderate load, for several months, without so much as a restart of an EC2 instance. On two occasions 6+ months ago one of the EC2 instances went a bit screwy, having problems connecting to external URLs, but in both cases we just terminated the problem instance and Beanstalk did the rest (provisioning a replacement and putting it behind the load balancer). We could probably avoid that sort of problem in future by improving our internal monitoring (in conjunction with the beanstalk health checks that ping a URL on each EC2 instance at frequent intervals to check for problems).

Can you tell me the reason you'd want to run a load balancer in front of a single instance? I read the feature spec for the elastic load balancer and can't it out.

If you have one single instance only, I'm not sure there is any point. But if you want your app to auto-scale upwards to add EC2 instances if the load increases, then the load balancer is necessary. It enables all your EC2 instances to be accessible via one public URL (as the elastic load balancer forwards requests on).

I set it up such that we always have at least 2 EC2 instances, in separate availability zones, so that, if one goes down, at least the other is likely to keep working. And then it auto-scales upwards from there, if the load gets high enough.

The load balancer is pretty much transparent as far as you the developer is concerned. You don't need to configure it - beanstalk does that - and it scales completely transparently (unlike the EC2 instances, for which you do need to define auto-scaling rules if you want that).

I may be missing some subtleties as I'm far from being an AWS expert... Just been picking bits up here and there through using beanstalk and a few of the other APIs for extra bits and bobs.

There's 2 main reasons. 1, if the AZ that your single instance is running in dies completely, ELB can help fire up a new instance in another zone quickly without having to handle re-routing DNS, etc. 2 if you need extra capacity in a hurry (e.g. you just got linked from HN and 10,000 people sign up), you can add more instances without re-achitecting.

Whether it's worth paying for those capabilities is always up to you though.. :)

This is a great point. Although it's a slight extra cost, we have adopted this a best practice because it's still cheaper than a more responsive DNS service.

Anybody else looking to move from AppEngine? I can't wait to try this out, AppEngine has become too expensive and AWS has more and better deployment options, and more and better associated services.

AWS doesn’t have anything even remotely approaching Appengine Datastore in functionality and scalability. S3 is also inferior in functionality to Google Cloud Storage, e.g. eventual consistency, no atomic operations. That’s actually a big deal if you’re building anything slightly more complex. OTOH, Google is now introducing their Compute solution, so you can just move parts that don’t fit very well within Appengine to that (if you have beta access).

What are some advantages of using something like Elastic Beanstalk versus Heroku, if any?

Price! Heroku is vastly more expensive.

Freedom from US-east. Heroku, afaik, only have instances in US-east.

Also, it's way less expensive than Heroku.

as the others have said price mostly. You are cutting out the middle man. Heroku has an awesome service, but they are just running on AWS. Elastic Beanstalk is basically the same Platform as a Service idea but you get rid of the middle man. I'm happy to see python supported as I was just getting started with a project and debating using heroku vs dealing with the hassle of doing it myself on aws. Now i get the best of both worlds.

Great day indeed to be a python developer. Greatly appreciated, thanks Amazon!

Now, if you could support PostgreSQL in RDS, that would be a nice feat.

Isn't Postgresql on EC2 called Heroku?

Is that really their official blog? I'm not usually one to care about a professional looking UI, but I feel like it matters here. A non tech boss is going to see this next to cloud.google and have a tremendous bias, regardless of what his/her tech people think

Hosted on Typepad, which apparently still exists. That blog template is probably 10 years old

Google's blog is on blogspot, and looks just as bad. EC2's main website is professionally organized.

I wonder if this will lead to Heroku reviewing their pricing plans sooner than expected. Elastic Beanstalk is catching up, AppFog had their public launch recently... they must be feeling the heat.

Heroku's main supplier is competing with them, so they need to provide a better value add layer than Amazon can to justify the price or get off AWS, do it themselves (not easy) and take that part of the margin

Does anyone know if this has explicit support for a task queue like Celery?

I never tried it but I guess you can use SQS for queues on AWS.

http://aws.amazon.com/sqs/

If you go with SQS, keep in mind it doesn't guarantee that messages will be received exactly once[1]. I've used SQS for large projects and this was killer because we had to implement message locking ourselves to prevent messages from being processed twice. Since then, this has been a deal breaker for using SQS again (now use Celery).

[1] http://aws.amazon.com/sqs/faqs/#How_many_times_will_I_receiv...

I was under the impression [1] that a message would only be received more than once if the first attempt at processing failed to complete in the lock time - for example if the queue consumer crashed. Is it more complicated than that?

[1] http://aws.amazon.com/sqs/#functionality

Sorry, but lol. You had to implement locking on top of a job queueing system to make it work? That's the whole point of a queueing system!!!

You probably didn't realize you had to extend the lock peeiodically.

Speaking of celery - I was looking at celery over the weekend. Do you know if its possible to submit jobs from languages other than Python (specifically, I would like to use Java)? I have some Clojure code running on Storm from which I would like to submit tasks to be executed in Python on other machines. Do you know if this is possible with celery? I couldn't really find any information on it.

Reply on news.ycombinator.com