Ask HN: Who's using Google App Engine?
What big sites use Google App Engine? Have you tried it? Is it flexible enough to run big, serious projects on?
Though it doesn't have the same HN buzz that Heroku does, App Engine has a fairly large developer community and a lot of applications using it. Like almost all PaaS product, there are restrictions that limit what you can do (library support, native code support, etc) and these limits frustrate developers that go in expecting the same experience as having a virtual machine. The limits mean that App Engine isn't the right solution for every application under the sun. But if your application doesn't have super unique requirements, the value that App Engine provides is hard to beat.
Ten second deployments, one click version management (and roll back), logs and monitoring, and autoscaling mean that we can be really agile pushing new versions of the application. App Engine's recent changes to the scheduler and Always On feature have really solved the slow instant start issue that still plagues a lot of Heroku developers. Last year the datastore had a ton of performance and reliability issues, but the new high replication configuration appears to have solved the problem. The biggest issue for us is the one that's mentioned here a couple of times and that's SSL.
For now, App Engine is nearly perfect for us. If we end up with unique requirements, we can definitely spin up a few EC2 instances and hook them up to App Engine's Task Queue. But I'm not excited about having to manage machines. Personally, if we hit a limit with App Engine, we'd be looking to another PaaS offering like Heroku or dotCloud, or Infrastructure as a Service like MongoHQ/RedisToGo before bothering to deal with machines again.
Have you launched yet? My experience is that it becomes very difficult to manage once you have customer data.
Pros:
- scaling is amazing; we've had sites go from 1 to over 1000 instances in a few minutes; from <1 to 1600 queries per second in the same time.
- reliability is pretty good now
- if you let go of the idea of doing everything synchronously life gets a lot easier, and deferred processing is really easy to do.
- the development process is great: download SDK, checkout code, point SDK at code and you're off. Then you develop locally and have it run exactly the same (with a handful of caveats) as it does appspot. It's a really low barrier to entry.
Cons:
- webapp (the default framework) sucks. Once you find something like tipfy, webapp2, or flask life gets a lot better
- changing your datastore structure can really suck with a live site. There's still no simple way of getting data on and off, or of doing backups. Messing with your live database if you need to fix some bug can be scary
- it takes some time to grok the right way to model data, and to use the datastore.
- there's a lot of "here's how to get started / build a simple blog" info, very little on best practices, how to model complex data, how to do maintenance...
- the datastore is extremely scalable if you use it as intended.
- your app can automatically scale up and down (as if an invisible genie were moving the little "workers" slider that Heroku uses)
- always on instances make your app mega fast, even at low volume (but cost a tad bit more).
- there are a lot of useful apis (memcache, queue, email) and a nice development environment.
- it usually "just works" and "just scales".
a few bad things:
- they had some very annoying downtime while they sorted out the datastore architecture problems (where theory and practice had differed with horrible results)
- the OSX dev tools don't bundle the correct version of python and associated libs, which makes it a pain to get a working dev environment on OSX without breaking your system's python.
- no naked domains, SSL support for normal apps that aren't priced per user.
All in all I'd rather write an app for AppEngine than any other cloud system, though Heroku is catching up -- both could really benefit from prioritizing 24 hour support via irc though.
As for SSL. I mean come on. It's 2011, it's absolutely unbelievable they don't support SSL for custom domains. Unbelievable.
Why do you say that?
It's a site that allows the public to submit weather observations from anywhere in the world. Since launching 31 days ago, we've got 1.7 million weather reports and probably spend about 0-30 seconds a day (on average) on maintenance.
App Engine has some big pains (SSL has already been mentioned) but if you don't require it (we don't), I think it's a fantastic tool.
The multiple versions, quick deployment (new build inc all testing&configuration can be completed by us in <5 min), automatic scaling and the fairly good dashboard are fantastic. Plus, our infrastructure costs in development (before heavy load testing) was $10 for 3 months of work - absolute bargain! Even after heavy load testing, it was a fraction of what a real server would have cost.
I'd say the biggest difficulty (or barrier to entry) is the datastore design. It's very easy to think you're in SQL land, and you're not. And because of that, it's very easy to burn yourself quickly. But, the design gives great scalability. We wouldn't be able to manage the size of the data we're expecting in 2 years+ in a typical environment.
So, I think it's great :)
(Disclaimer: I was part of the team at PA Consulting that built WOW)
Sure the fact that are some limitations (no IO and not many libraries) are a bit of a down-side, but the payoffs of using such a service instead of managing your own servers is really great. You're just able to crank a lot on the app code part without worrying at all about system administration work.
The simplicity of the datastore is also a great benefit, and it scales like nothing out there.
One thing not mentioned yet that is a legitimate concern applying generally to PaaS offerings including App Engine - that is vendor lock-in. There are some way to address that including choice of apis and frameworks and also projects like AppScale and TyphoonAE.
There is much to evaluate when considering App Engine for your projects. the flexibility of the platform from a technical perspective is one of things.
Also note, watch closely the changes that are coming in pricing and the way that Google is offering the service. You don't want to get caught evaluating the App Engine of 2010 for 2012.
The biggest benefits for us are the scalability (huge), ease of deployment, and the monitoring. The biggest cons have been lack of support for SSL on custom domains (HUGE hassle) and lack of official support.
The websites below has tens of thousands of registered members, there are also similar websites for some other European countries (Italy, Spain, Germany etc)
Getting British Businesses Online: http://www.gbbo.co.uk/
Getting Australian Businesses Online: http://www.gettingbusinessonline.com.au/
So yeah, App Engine seems to be suitable for big projects if you play by their rules. Only problem seems to be backing up data, as their App Engine's bulkloader tool has rate limitation, so it takes quite a while to download/upload all data.
This could solve the naked domain and SSL issues and enable other cool customizations.
Has anyone done this themselves? Just put Varnish or Nginx in front of GAE?
I'm sure there are many people who have done this. I'd be interested in hearing how other people have solved the lack of SSL...
I use appengine for all my web apps, it works for most of basic webapps, here are a few I have used appengine for: http://gramfeed.com, http://shaloc.com, http://checkinmania.com, http://misotrendy.com
Question: I was looking for a library for authentication mechanism other than FB or Google . Any suggestions ? (All I need is basic user/password)
Note: I noticed that some have accessed the site from this link. The site is beta and is newly launched, and for now we are only covering Europe. If you want to check out how fluid are the geo spatial queries, you can try checking out Portugal, where half our spots are now.
I've used it for http://www.minehub.net, http://www.quickcite.it, and a few others.
So what we have now is a product that runs both on App Engine and on DotCloud. Since DotCloud is awesome, too, and we know that we can talk to a specific person when something goes wrong, it's currently a front-runner for us. We're also looking at Joyent, too, as opposed to EC2.
But even if we switch large parts of our product to somewhere else, I wouldn't be surprised if we still used App Engine for some things.
I would still try it again down the road, but it's really not a good fit for what I'm currently doing.
The things I loved: - tight integration with Google accounts - easy integration with other Google apis (we used a few)
The frustrating thing was dealing with the black box. We used an api to get domain availability. Every time it returns more slowly than the equivalent call on our linux based cloud server.
With GAE, you're basically stuck with what you get. With a generic Linux based cloud server - you have more room to optimize your environment, or even take your code and run it anywhere... AWS, Rackspace, Voxel...
I still believe in what GAE is trying to do - and I'll definitely try it out again for future projects.
But I am betting on it because I think it will make my life easier in the future...
The memcache is ok to get.
The java subset is the part that bothers me as I had a lot of already made stuff I liked to use in my new site, and had to redo a loooooot of stuff to fit the dawn java subset.