Hacker News Reader

Comment by jm4 | Hacker News Reader

I haven't used App Engine, but the BigTable thing looks pretty cool and I think a lot of us are starting to realize that SQL relational databases aren't very good for a lot of the things we do. That said, I think the main concern is lock in. The question I'd be asking is how easy is it to get an established application off App Engine.

View on HN

The main lock-in issue to me is the datastore. If you are worried about lock-in, you could try to use vanilla Django as much as possible. I think it's likely that an opensource datastore equivalent will appear, or at least something that's close enough like HBase or Hypertable.

Yeah, that's what I was getting at. As soon as you need any persistent storage you're chained to BigTable. That leaves a lot of code that needs to be rewritten if you need to move to a new storage model.

This isn't unlike the storage problems PHP had quite a few years ago. There was a separate module for every database that was supported and they lacked consistent naming conventions as well as consistent functionality. It was literally one set of functions you used to talk to MySQL and a completely different set of functions for PostgreSQL or SQL Server. This is still reflected in the documentation: http://www.php.net/manual/en/refs.database.php

For a long time there wasn't a common wrapper for all of these modules so if you started with MySQL and needed to move to Oracle or something you had to rewrite a bunch of code. I had to do that once and I never used PHP again.

It seems that's where we're at with App Engine and BigTable. I'm sure there's going to eventually be something equivalent or some sort of abstraction, but I wouldn't want to be holding my breath.

Isn't there already a decent abstraction, namely the Datastore API (http://code.google.com/appengine/docs/datastore/)? Current apps are written against that API and it's not a stretch to see an opensource BigTable on EC2 with a Datastore API. (See appdrop.com for an even simpler datastore replacement.)

Yes, that's BigTable specific. My point was that right now you have no option but to rewrite code if you need to move off App Engine and BigTable.

How is it not a stretch to see an open source BigTable on EC2? As far as I can tell BigTable has, up to this point, given Google a competitive edge. They are merely letting us come along for a ride and Amazon is a direct competitor in this market. Why in the world would they allow it to run on EC2?

If you are coding to the Datastore API, you won't have to rewrite that code when moving to another provider who offers the Datastore API. AppDrop.com, for example, was a proof-of-concept that let you run apps on EC2 unchanged. He simply replaced the implementation below the datastore stubs.

Google may not open-source BigTable code, but as I mentioned above, there are a number of open-source projects that provide similar features. Any of these can run on EC2 together with the open-source AppEngine SDK.

"The question I'd be asking is how easy is it to get an established application off App Engine." ... this is the most important question.

Reply on news.ycombinator.com