Hidden limitations of Google App Engine

I have completely fallen in love with AppEngine since the beginning of this year. It is a brilliant dev platform for prototyping and the BigTable DB is a pleasure to work with.

My prototyping platform is essentially: tornado.web with the tornado request, response, escape, auth and locale. OpenID from web.py. My own WSGIApplication that inherits from the built-in so that it uses my Controller class (which has a plugin arch), Django for templates, JSON, BeautifulSoup, Markdown, a generic 'asset' module implementing something akin to ActiveRecord on the standard DB with memcache support, a generic HTML5 getting started template, a CSS toolkit (can't recall the name) then jQuery and backbone.js with some of my own JS glue code that maps the server models to client models.

Using this stack, I can prototype form-driven web apps in hours.

I wrote a relevance view for Twitter (with Twitter oauth support) in half a day. I don't think I will use the RDBMS even when they release it - BigTable is excellent (it will be confusing if you schema the same way you schema with an RDBMS).

It is great to be able to have an idea for a quick app, register an AppEngine app, fire the code up, commit and deploy and have the app live for others to try out within hours.

The admin interface allows you to spot errors and choke points in production and fix them quickly. The pricing is better than AWS or any of the others and the platform includes hidden gems such as the MapReduce lib, multitenancy, xmpp and Google data libs (including auth)

The only real limitations that I have bumped into are:

* The max request time is around 20 seconds, so you can't have open Ajax requests

* No HTTPS support as yet (coming soon, apparently)

* Limits on the number and size of files. It isn't ideal for serving a large number of static files, but it does have zipimport support for libs

* For best performance, you need to define the indexes yourself rather than relying on the defaults that the lib generates

* There are a lot of sys calls blocked, so you can't run Tornado or any cpython modules

AppEngine has made some great leaps in the last year. We built http://gri.pe on it earlier this year and have been impressed by the platform.

The last two months have seen the platform mature a lot as well. The latest 1.4 release has some amazing features: async datastore queries, app keep-alive, longer timeouts for background tasks, etc.

The architecture limitations exist b/c App Engine essentially scales effortlessly once you abide by them.

If you insist on using SQL, then use a company like Heroku, but realize that if your app gets popular you will be paying Heroku a lot of money for the same level of performance you could get for a few bucks a month using app engine.

After I wrote my first toy app a few years ago, App Engine doesn't really feel like it has any significant limitations other than the absence of naked domains and SSL. The recent reliability problems have been a nightmare, but as of Nov 6, I feel like they have been resolved.

I should add that I love app engine and use it whenever I don't need naked domains or SSL. (SSL is only available via .appspot.com domains)

The biggest limitation I had was while writing to the database, the database would sometimes return an error for no apparent reason. I'd then be forced to catch these errors, and resubmit. But if you had lots of traffic and resubmit too quickly, Google App Engine would throw more errors back at you. This was all over a year ago, not sure how it is now, I would never go back to using it.

From http://code.google.com/appengine/articles/life_of_write.html

There is an expected failure rate on writes as Bigtable tablets are sometimes unavailable, for example, when they are being moved or split. The presence of more indexes increases the probability of hitting an unavailable tablet as an exception will be raised if a write fails for any of the indexes. In those situations, your application will need to decide how to handle the exception. One option is to add a task to the task queue to retry the write at a later point in time. Another idea would be to respond with an error from the app and have the client retry. This tends to work with things like AJAX requests where there is client side logic which can handle an error message from the server.

Notice the date of this question is Feb 20th, 2009. Almost 2 years out of date. I imagine at least some of the limitations mentioned are gone by now.

If you're sick of these limitations (and use Django) head over to www.djangy.com and sign up for the private beta :-)

I wonder if the simplenote guys can provide some input?

http://simplenoteapp.com/about/

The biggest limitation is that they only support two languages - python and java.

Not that there's anything wrong with these particular languages, but I'd prefer more choice. It would be cool if they supported C++, PHP, Javascript (V8), Ruby, Erlang.