Google Jumps Head First Into Web Services With Google App Engine | Hacker News Reader

Google Jumps Head First Into Web Services With Google App Engine

techcrunch.com

139 pointsjsjenkins168113 comments

Open articleView on HN

update: this URL is now LIVE: http://appengine.google.com/

update2: yeah, waitlisted as well... http://code.google.com/appengine/downloads.html (where code samples are)

update3: API: http://code.google.com/appengine/docs/gettingstarted/

update4: everyone who signed up recently should have received an invite by now!

Lots of limitations:

- Python-only for now but the complete architecture is language neutral and they will add other language support (will rely on feedback on which language support to add next). Before they add a language, they need to 'harden' it (i.e. remove some features from it).

- Django is the only API that's currently supported, can upload other framework(s) but you're on your own (which will be an issue since there will be some issues with other frameworks due to other limitations)

- you cannot write to the filesystem - due to distributed nature of the system, you have no idea where the file will end up.

- you cannot open sockets! you can only use the limited API that they provide (URL & mail sending API). Forget about Twisted :(

- no threads (Google says they provide scalability in other ways)

- limits on how long an app request can run – forget about uploading large files for now.

- admin console contains version source deployment client (svn or git?) which means Google will have easy access to your source code. If you're competing with Google, beware... Google potentially has access to your source code (and data of course) so you will have to trust them and their legal agreement! Definitely some conflict of interest is possible.

Update: Here's the relevant info from the TOS: http://code.google.com/appengine/terms.html

By submitting, posting or displaying the Content on or through the Service you give Google a worldwide, royalty-free, and non-exclusive license to reproduce, adapt, modify, translate, publish, publicly perform, publicly display and distribute such Content for the sole purpose of enabling Google to provide you with the Service in accordance with its privacy policy.

Seems to me that there's a lot of magic happening behind the scenes but you get simplicity for that.

They demoed few interesting applications:

- Huddle chat - like 37sig's Campfire - client queries the server every few seconds to pick up deltas.

- Jaiku - twitter clone that Google bought was partially converted to Appengine

- some social app that allows you to pick a place to go out with friends (clubs etc).

I was refreshing the wrong link, http://code.google.com/appengine/

Edit: got an invite!

I signed-in and am waitlisted!? That was fast, unless everyone is on a waitlist and e-mailed in order (I hope).

It has to be that.. I got waitlisted at 12:02 (EST). No way 10,000 developers can signup that fast, or is there?

update at 12:55 (EST):

Thanks for signing up to try Google App Engine! Your account has been activated, so you can begin building applications!

Did anybody here get into beta?

edit: just did.

I did! :D

I just got an invite via email... now I can be the evil genius I always dreamed of! bwahaha!

Oh, whats that you say?.. "Don't be evil".. nevermind. :P

Got mine.

"... Thanks for signing up to try Google App Engine! Your account has been activated, so you can begin building applications! ..."

I'm in.

"Google potentially has access to your source code so you will have to trust them and their legal agreement!"

I would be far more skeptical of what they do with your data, which would live in bigtable.

I see some SDK and download links here: http://code.google.com/p/googleappengine/

Looks like the SDK has been live there for 2 days. Wish Google would have an RSS feed for new things on code.google.com (if they do, let me know)

isn't huddle chat identical to campfire? http://blog.searchenginewatch.com/blog/080408-021637

One thing I haven't seen anyone mention: How this affects one's ability to be acquired by anyone other than Google in the future! I think it safely rules out being acquired by Microsoft, Yahoo, and the like. But what about other customers who want to integrate your apps with theirs? I think that then rules out MySpace, Facebook, and plenty of others. That brings us to companies that don't really care about stuff like that. The list gets small pretty quickly. With the exception of Google -- and they don't have to acquire you because they already have you locked-in! :-)

This isn't an issue with EC2 because EC2 apps can run anywhere the proper distro is located (assuming you don't rely too much on other AWS services).

How do you figure? What web app isn't required to port to new infrastructure when they're acquired. Sure MSFT is at odds with Google, but they were at odds with Unix when they purchased Hotmail (ran on Unix infrastructure for awhile after acquisition even). Then they ported. Assuming the port goes well, it probably makes a nice press release also.

The code and user base (the two most valuable parts of a startup) will still exist and could still be ported to another hardware architecture.

with the users api your users cannot be transferred, they are google users who can use your application, it isnt openid, you cant chuck this on another computer and have remote authentication.

anything to do with your code that stores data, interfaces with external data / services, runs new threads etc pretty much everything that makes it useful, cannot be ported, by 'hardening' the language sandbox, they tie you into their api. their query language is even google specific.

any application written on this platform looks to need to be entirely rewritten from scratch to port to another architecture

with the users api your users cannot be transferred, they are google users who can use your application, it isnt openid, you cant chuck this on another computer and have remote authentication.

I wonder, though -- what if you used something like a remote REST interface to do authentication, passing MD5'ed or SHA'ed passwords as one of the arguments, and then based auth on that? Google wouldn't like it, but technically it appears to be within the scope of the TOS.

Very, very good point, though... users and data are basically the only things that matter for this type of application.

I did find one of your other points spurious, though:

their query language is even google specific.

Oh for crying out loud -- if you can't write a Perl one-liner to convert every instance of GQL to SQL with bind variables, or if you don't already use placeholders in any instance of raw SQL in your own code, you've got worse problems than that. This is the least of the problems IMHO.

Something has occurred to me regarding NumPy, SciPy, R, etc. Since it's Google's CPU time, it's not my problem if a loop or matrix calculation is inefficient. I'm going to deploy some grad students to stress test their TOS... :-)

I havent looked deep into it, buts the docs mentioning that it was a crud datastore engine, which means there is no 1-1 relationship with sql, by the positioning of the rest of the api, I dont imagine the more complex functionality google will introduce will align with the current standards either

* I dont imagine the more complex functionality google will introduce will align with the current standards either*

Probably not, but again, you don't necessarily have to use the bits that lock you in :-)

Nothing new under the sun here...

I'm speed-reading my way through the docs right now, and I just had a realization... I'm betting that a huge driving force for them to release something like this is to lower the cost of acquisitions. They are opening up a lot of the tech that (AFAIK) is being used on some of their apps internally.

It would be vastly easier for them to get at the creamy nougat filling of acquired startups if said startups were already using the google pixie dust.

Not just "easier to get at" but also "locked in so M'soft and Yahoo! can't as easily buy them."

There seems to have been the same level of hype over Facebook's apps and I haven't seen many Facebook app startups getting acquired by Facebook. Granted the Google App Engine is whole other level of app hosting but compared to the scale of Google could any single Google App Engine app grow large enough to worth acquisition by Google? The best offer an App Engine startup may get from Google is maybe a job offer and nothing more.

The service is completely free during the beta period, but there are ceilings on usage.

I understand these articles are written in a rush, but this is plain wrong. From Google's site, it seems pretty clear to me that this will be true out of beta:

Every Google App Engine application can use up to 500MB of persistent storage and enough bandwidth and CPU for 5 million monthly page views.

I'm surprised nobody has mentioned this - does this mean the tech cost of a web-based startup is now $0?

The operational costs of a web startup are approaching the cost of connectivity. But, the people to wire it together will still want money.

And as a number of people are gleefully pointing out, Appengine comes with a strategic cost that is not trivial.

After playing with the .zip file they provide, I think fears of lockin are overblown. It should be relatively trivial to port existing apps to GAE, and apps developed for GAE should be easy to port to a WSGI enabled server.

Google will be a good place to run the free version of your web app that serves as an ad for the premium version that does the same thing with crypto, or with C extensions that add value to the transaction, you'll be able to use the same core code in relatively similar application containers.

This is what I was thinking as I played with it last night.

I'm definitely going to use it for that. And if I can get away with it, I'm going to try doing some unclever implementations of semi-supervised ML, maybe a few matrix calculations, and see what I can get away with before they kick me off their grid :-)

It occurred to me that, since no one is paying me to optimize the usual tight loops that crop up, or write a C extension to speed them up, I can try just running a shit load of them in parallel and storing the results in their ObjectStore. Now I just need to randomize the ''request'' IP blocks and we're all set for the billion or so simulations that my officemate wants to run.

It was a long time ago that the "IT" cost of web startups was anything at all worth thinking about. About as relevant as the cost of a water cooler (less so, probably)

"Another serious limitation is the requirement that applications be written in Python, a popular scripting language for building modern web apps"

Sounds like a feature to me.

It would be awesome if they ported the Google Web Toolkit from Java to Python.

There's an open source project attempting to recreate GWT in python: http://code.google.com/p/pyjamas/

Google account sign-in: let your users sign in to your application using a Google account and password

This worries me. It sounds like Google is trying to become the new single sign-on, completely blowing off OpenID, a perfectly good open solution. Microsoft Passport all over again...

This is insane. Am I the only one having deja vu about Windows versions and locking in the 90s?

This is (potentially) a much much deeper locking to googles cloud (its not just an OS, its your hardware too !).

Of course, it seems django - like, maybe that is enough to avoid the trap?

It supports django templates.

that is encouraging I guess. I was a bit worried it would be even more proprietary then it is. It looks like the main way they will lock you in is with the google apis to google services, which isn't related to them hosting your app, it seems.

Lock-in? Google "obtains no right, title or interest" in what is created; the creator is responsible for what is created and can take it down and host it elsewhere.

From the "Google App Engine Terms of Service" (http://code.google.com/appengine/terms.html):

"6.3. Except as provided in Section 8, Google acknowledges and agrees that it obtains no right, title or interest from you (or your licensors) under these Terms in or to any Content or the Application that you create, submit, post, transmit or display on, or through, the Service, including any intellectual property rights which subsist in that Content and the Application (whether those rights happen to be registered or not, and wherever in the world those rights may exist). Unless you have agreed otherwise in writing with Google, you agree that you are responsible for protecting and enforcing those rights and that Google has no obligation to do so on your behalf."

"GQL is a query language that is used with the Google App Engine datastore."

"Our Users API allows you to authenticate users with Google Accounts"

from some of the quotes earlier,

"you cannot write to the filesystem - due to distributed nature of the system, you have no idea where the file will end up."

"you cannot open sockets! you can only use the limited API that they provide (URL & mail sending API). Forget about Twisted :("

"no threads (Google says they provide scalability in other ways)"

To me looks very specifically that applications built on this framework can not be transferred without a major to full rewrite, thats very much lock in, it doesnt make it easier for google to buy startups, it makes them easier to no longer need to buy them. your users become their users. your applications become part of the google infrastructure.

This looks like the most dangerous and hopfully damaging thing google have done so far

I agree. But it also one of the most tempting things. But this is platform lockin for a new generation. Had to happen sooner or later.

"Our Users API allows you to authenticate users with Google Accounts"

It doesn't in any way disallow you from using something like openid, but your post seemed to imply that.

s/Google/Microsoft/ s/Google App Engine/Windows

and you will see the problem ;)

I think the bigger implication of this is not that the costs for startups are now 0. The couple of bucks for a hosted server should not have been that much of an impedient for a startup.

Rather, this enables a new category of applications, personalized web applications for non-programmers. Instead of getting a blog on wordpress.com, in the future I will upload a wordpress image to google. Eveybody can have their private bug tracker easily now. And so on. I think there will be opportunities for new applications, or for enhanced user-friendliness of existing applications.

An important question at least for me is what sort of libraries will be usable from these python instances? Like, I would like to have NumPy and SciPy libraries available for a lot of the heavy computations I do. (in particular, linear algebraic representations of graphs for heuristic solutions to various planning and image processing problems)

I am wondering the same. I have a lot of SciPy and NetworkX based simulation code I have been meaning to plug into web interfaces. This could be extremely useful for academic research.

fat fucking chance at present. Processes must return data within a few seconds, so unless I can take up an arbitrary amount of CPU-hours in those few seconds, there won't be an awful lot of simulation or analysis going on until the TOS changes.

Not that I expected otherwise -- there are plenty of people (like, say, my officemate) who would pay a few bucks to run a few million simulations on Google's infrastructure. (I would too, but it'd have to be a lot of nodes, since I have my own farm of client machines due to the way I arrange consulting)

I hear they are actually CPU starved these days, which quite literally boggles my mind... if Google can run out of cycles, anyone can. But that informs the current deployment and limitations if it is true.

Some aspects of SciPy are a bear to build (I maintained the binary distributions for a couple of years), with a huge dependency chain. I wouldn't bet on being able to install it without a lot of trouble. NumPy isn't so bad.

well, as long as they have NumPy, I can deal with rolling my own ontop the rest. OTOH, what would be amazing would be to have some of these hosting services have a setup where you can execute matlab code and extensions on top! I suppose that that would require a pretty complicated licensing setup though...

You may not have much luck... from the FAQ: For security reasons, most C-based modules are disabled

So, it seems that EC2 is still the way to go for that kind of problem.

Once Google App Engine is fully released, you will be able to acquire additional computing resources at market prices, and set your own automatic resource limits.

Market prices? What exactly does that mean, will it be determined by supply and demand, via an auction like AdWords perhaps?

What makes it really interesting is the last point from Google:

- Easily integrate with other Google services. It’s unnecessary and inefficient for developers to write components like authentication and e-mail from scratch for each new application. Developers using Google App Engine can make use of built-in components and Google’s broader library of APIs that provide plug-and-play functionality for simple but important features.

One of the biggest things that I'm interested in, is being able to use Google checkout. If they can provide an easy API integrated to their hosting, it's going to be killer.

That's what I heard in conversations with several people. For my application (ezLearnz.com), I'm really interested in using google documents instead of rich text fields, since that solves a whole host of problems (adding images, revision history, sharing editing capabilities, etc). But I think that's a ways off.

is everybody else continuously refreshing http://code.google.com/appengine to be amongst the first 10k to sign up?

word on the street is it is going live at 9pm PST, but I have also seen 10pm, so we'll see.

Any tips on how long the 10k places will last? I'll go 30 mins

3 mins is more like it. 12:02am here and it still offline :\

I wonder how Steve Yegge's Rhino on Rails might fit into this...

http://steve-yegge.blogspot.com/2007/06/rhino-on-rails.html

It would be super fun to play with a well-supported server-side JS framework. He mentioned open sourcing that code eventually too :)

This is a huge announcement, although I am surprised that it is not designed to support anything other than Python. This hindrance might keep more people using AWS than they might realize.

i agree, what could possibly be the technical reason for only supporting python?

one thing I know is that Google loves python, so its a natural choice to start with. No doubt they'll be adding support for other languages as time allows.

I would assume that google built a sandbox (probably like appjet), for python. It's not trivial to build a sandbox for a language, and it's certainly not trivial to build multiple all at once - which is why they are only launching with one (python).

Read the first post. They had to harden Python to allow third party code to run on their network. Other languages (once hardened) are coming...

I wonder if this -- coupled with Django going 1.0 soon -- will create even more buzz for Django?

"... I wonder if this -- coupled with Django going 1.0 soon -- will create even more buzz for Django? ..."

Prophetic suggestion ~ http://code.google.com/appengine/articles/django.html & http://news.ycombinator.com/item?id=157648

"... although I am surprised that it is not designed to support anything other than Python ..."

Why? The Google "big 3 language" choices are Cpp, Java & python ~ http://panela.blog-city.com/python_at_google_greg_stein__sdf... You may see perl or other languages but the "big 3" are the officially supported ones.

"... python ... but it will be a non-starter for many developers. ..."

I find that an interesting statement. Why say that? Python is a pretty easy language to learn & use.

Its a great language and easy to learn and use, but it might greatly limit their business relative to AWS.

I polled HN about a month ago to see what percentages of HN readers used what languages for web development, and while Python was the most common, its still only used by 22% of those polled. Take this data with a grain of salt, but still an interesting statistic:

http://spreadsheets.google.com/pub?key=pxT7jIffmj3lGdXFcn9hK...

It's not even the choice of python that's a limiting factor, its that you don't have a full server, just a language sandbox. Still it might be a great place to start a web application.

This may be the move for Python to overtake Ruby in web app industry..This seems even more revolutionary than what RoR did before 3 years ago..

pyRuby anyone :p ??

This is all just wild speculation but I wonder if some hosting company will come out with a "compatible" api for when you want to leave Google.

Or an open source project to create a compatible framework... That way you could choose from many hosts :)

Maybe Google could speed up the process and make the app engine framework open-source :~).

That would be totally awesome, and you could make the case that it benefits them to have their App Engine platform become a more standardized/open framework!

better to let us white hats find the flaws :)

no buffer overflows or fork bombs in this sandbox baby.

I think they already did (kind of). As far as I can tell from a quick glance, this is Django on top of Google's infrastructure. But they aren't about to open-source their infrastructure.

But they aren't about to open-source their infrastructure.

Wouldn't matter if they did -- you can't open-source a bunch of contracts with trucking companies, nor would any sane company want to open-source their testing routines for the hardware they build in house to manage the mess. Parts of the infrastructure are, however, published as patents :-)

Google's operational efficiency in deploying new capacity is the crushing advantage they possess over 99.999% of the companies on the planet.

Interesting that this lets apps integrate with Google user accounts. It's pretty cool, but I don't know how I'd feel about typing my Google user name and password into a site which isn't obviously owned by Google.

Huge news for python ! Cool. Of course, who in their right mind would trust their business lifeblood to something proprietary to google only? (or maybe its not, we will see).

The code written and the content created is not the property or problem of Google. Read the Terms of Service (especially section 6) at:

http://code.google.com/appengine/terms.html

You are totally missing the point - doesn't matter about property, if you can't take your app anywhere else without rewriting it, that is lock-in. Do you think writing for Windows locked you in content wise? Of course not. Totally irrelevant.

Google, the deadline for the Android Challenge is just 7 days, 1 hour, and 56 minutes away.

Couldn't you have waited a bit over a week to release this? :P

this is good news for heroku - Amazon will probably be interested in them

Interesting side note: it appears they've reserved all Gmail names for application identifiers.

It wouldn't let me pick my Gmail name (which is fairly obscure) but appending "1" to the end worked...

Looks like this will put a lot of web hosts out of business...

Looking closer at the Django support... it's Django except for the ORM, Sessions, and Admin! All that's left is routing, forms, and the templates, all of which have better alternatives. No out of the box Django apps will run without the ORM, and there are viable alternatives to the remaining pieces - FormEncode and Genshi.

But hooray for the limited transaction support. That was a real limitation in Amazon SDB.

Has anybody been able to put their app online?

I keep getting Application Identifier: App already exists. Even when I pick random App names. Odd.

Are you going through the web page to create your identifier first or just setting it in the app.yaml and attempting to upload? I had no problem setting mine up through the site first and then using the appcfg deal to up it.

I couldn't create the identifier from their web page, it looks like it needs to be at least 8 characters long.

It's weird, because I created an identifier with 4 letters and it worked, but only after my first choice identifier, also with 4 letters, didn't work and it said it wasn't avaiable.

This will probably end up being like heroku, but for django... hopefully.

Mail API supports sending mail, but I don't see a way to receive mail to the app's domain... am I missing something obvious?

It looks like it's not yet supported (that I can tell). I need this functionality as well, so hopefully we'll see more soon.

I'm sure that you can use a google apps for your domain account to receive emails back. It only costs about $50 a year.

There is a new channel #appengine on freenode

It'll be interesting to see if BigTable is required. Sounds like they've integrated a full stack and require python, so it wouldn't surprise me if the only DB available is BigTable. If so, which python framework is most easily decoupled from traditional DBs?

Anyone buying put options on Amazon? Don't think it'll affect them short-term, but long-term is a whole another story (after Google's beta period and after other languages are supported). Free usually wins.

AWS only generates about 2% of Amazon's total revenue so I wouldn't be buying puts any time soon

I doubt this service will be free once it comes out of beta. Google will definitely charge for it. They're using the free aspect to gain some traction.

I'm willing to bet there will be a level of service available for free, just like they do on Google Apps for your Domain. You can host your mail and a handful of accounts for free, but at a certain service point, it becomes 50 bucks a user.

It'll be free unless you go over the limits. The limits are high enough for most.

This is really exciting. It'll be easier than ever to create scalable web apps. Choosing python seems sensible, I don't see what the fuss is about.

well one fuss i see is that there will be no red5 on google web services it seems...

not a big deal to most people... but from what ive read this seems more like a competitor to heroku/appjet than ec2/s3 as others have mentioned

but as a django user, i am pleased

It is perfectly sensible if you are starting a project from scratch or have already built something purely in Python. For everyone else looking to host something already written in another language, it is a problem.

I'm sure Amazon doesn't mind though, they'll just continue to get all of this business.

Good point. For existing codebases it means porting code, which could be a pain.

I did just read that Python is just the first step, other languages will be supported soon. :)

Hmm, there are a bunch of sample apps available now: http://appgallery.appspot.com/

yeah I was on 9:02 and got waitlisted. I am stoked to bust out some python code again, I don't view it as a limitation at all! The only reason I don't use python more often is I cant be bothered with the added hosting issues it brings (as opposed to uber-cheap/readily available php/apache space I have all over the place).

Everytime I login for HuddleChat it says "Page you requested is invalid". Has anyone tried ussing that app?

Damnit, the one night of the week I wasn't online. Just signed up for the wait list.

Just got my invitation! Looks like they're opening the flood gates! Woohoo! :)

the sdk is now up/available for download. http://code.google.com/appengine/downloads.html

Just got the invite - 12:03am Central.

bah, we can't do any NumPy on appengine!

the install and hello world was easier than arc.... just saying. No seriously my only problem with arc was getting some mzscheme/cygwin stuff set up in windows but thats my fault for not wanting to dual boot my laptop.

Arc's goal is to be a language used by people 100 years from now, and only as a side-effect, something neat for you to build your web app in.

if google owns your users, content and code, what do you own?

your brand, and a copyright, if you have any sense

If they fuck this up, and lose developers' trust, they're done. I don't think that's the plan. I could be wrong.

Reply on news.ycombinator.com