Happy Holidays from the App Engine team - 1.4.0 SDK released | Hacker News Reader

Happy Holidays from the App Engine team - 1.4.0 SDK released

googleappengine.blogspot.com

This is great. We just turned on reserved instances for http://gri.pe (select "always on" in the billing section). It costs $9/month to keep three instances warm all the time. That's peanuts compared to saving the 10s warmup time if your traffic drops off for a minute.

Another huge item: up to 10 minutes of background processing on a single task queue callback:

  No more 30-second limit for background work - With this release, 
  we’ve significantly raised this limit for offline requests from 
  Task Queue and Cron: you can now run for up to 10 minutes without 
  interruption.

Make sure you turn on the new warmup handlers for your applications too. With this, AppEngine will deliver a request to /_ah/warmup and let it complete before serving traffic from a new instance.

XML:

  <inbound-services>
   <service>warmup</service>
  </inbound-services>

YAML:

  inbound_services:
   - warmup

Thanks for the price information - that is reasonable!

Can someone explain this "always on" in more detail? I don't have much experience with Google App Engine, does this mean apps not used frequently have a startup time? So, if noone accessed the site in the last few minutes, the first user accessing has to wait some time for use?

Exactly right. Especially with larger apps, the "first" user often sees some latency while the app is loaded into memory, static data structures are re-created, etc. The always on feature should keep this from happening.

Perhaps I'm being naive here, but doesn't an uptime-service like Pingdom do the job? They basically ping the site every 5 minutes which should keep the instance alive.

This release solves almost all problems we had with App Engine. Users now don't have to wait for an instance to start, so the site will always be fast. Also, DeadlineExceededError now can't lead to mysteriously broken instances, anymore (thanks Guido!). And with the Channel API we can build real-time collaborative apps.

Now we only need SSL support and here we come, blasting everyone away who wastes time on maintaining his own servers. :P Well, at least until the next feature is needed, but not available, so let's hope this won't be the case. ;)

Also, DeadlineExceededError now can't lead to mysteriously broken instances

Note that the increase in the deadline to 10 minutes only affects background tasks (ie, the Task Queue and Cron jobs). It's important, but won't fix those DeadlineExceededError on a user-facing page errors on it's own.

What the parent is referring to is that when a Python process gets the DeadlineExceededError, it will now be recycled. Previously it would be re-used, and this would break things if the error happened in the middle of a module import (because the module would not be re-imported). Django was especially vulnerable to this.

Thanks - I wasn't aware of this (I do mostly Java stuff).

I'd imagine that the "warmup request" thing is especially useful for avoiding this.

Indeed. Pre-warmup request, GAE would start sending requests to instances that were started but weren't fully loaded.

The pricing power this reveals Google to have is fascinating. On Heroku, which runs on top of EC2, 3 dynos running all the time cost $72 a month. Google is charging $9 for what seems to be the same thing. Is Heroku really charging 8 times as much, or are their dynos not directly comparable to Google's app instances?

http://heroku.com/pricing#3-0

Heroku is selling a lot more than Google. With Heroku there are few constraints on your app other than Ruby. You can do very traditional web development (SQL among other things) and there are a lot of conveniences available for you. You should be able to build your app faster and launch 1 instance for $0, and then pay Heroku when it needs to scale. Google will force you to build in a more cumbersome manner, but it will scale much better so you won't have to pay them as much. You are always going to pay more for something like Heroku that is very much optimizing developer time.

I think Heroku's pricing contains both the RAM and CPU cost for the instances. The Google price only includes the RAM cost of the servers, the CPU is still charged per request.

$9 is for an always-on instance. It includes no CPU or bandwidth.

AppEngine pricing is quite complex. You need to look at the (very generous) free quotes first (http://code.google.com/appengine/docs/quotas.html#Resources), and then look at the pricing (http://code.google.com/appengine/docs/billing.html#Billable_...)

Unless you have a very CPU-heavy application it's unlikely AppEngine will ever cost more than Heroku. 8 times at much seems a lot, but 5 times as much wouldn't surprise me.

Thanks to all three of you for the answers-I didn't realize AppEngine's pricing model unbundled warmed-up app instances resident in RAM from CPU time and bandwidth, which makes their offering not directly comparable to Heroku.

Their implementation of the channel is interesting compared to say, socket.io. Its essentially a one-way-only pipe where data can flow out of the server, to a client. The server only sees it as a pipe, and never gets any data back through it, not even connects or disconnects.

Any server-bound data must be sent through a REST interface, and the corresponding state retrieved through memcache or a DB request. To track connections, the client must periodically send notifications to the server to keep it alive: "If the application needs to keep track of which clients are connected, one solution is to set the socket’s onopen property to call a function designed to send a POST message to the server at some regular interval in order to notify the server of the client's state."

This is like waking up and finding the bike you asked santa for next to the christmas tree.

Awesome!

* Now please let us use naked domains, I badly need it for my startup, I hate www to death.

"I badly need it for me startup" and "I hate www to death". Do you really need it that badly or do you really just want it?

I don't disagree, I would like to have it. Also SSL for my domain without having to use appspot.com. But I'd put SSL as a much higher priority.

An url shortener would benefit more from a naked domain than from SSL.

* Not that I am working on one, just that you never know the poster intentions.

Oh god no more URL shorteners.

I hate the www thing too. The explanation is that you cannot have a cname on a root domain. They coud get around e problem by just hosting dns for you themselves.

The other issue is that you end up having t have yet another service to redirect dom.ain to www.dom.ain as well. Sometimes dns providers will do this, but not always. (Dotster doesn't do the right thing, for example.)

Can maybe someone comment on the current reliability of GAE? What people write on this topic, for example in the comments on http://www.carlosble.com/?p=719&rewrite, sounds pretty scary. Apparently the status page at http://code.google.com/status/appengine doesn't reflect people's experiences.

Is it possible to work around the reported reliability issues? After all, what good is scalability without reliability?

This has been discussed a lot on HN lately - it was quite the meme for a while.

I did a quick !hn search for you[1] and saw these three popular submissions:

http://news.ycombinator.com/item?id=1934563

http://news.ycombinator.com/item?id=1931456

http://news.ycombinator.com/item?id=1927903

The general impression, as I've understood it so far, is that App Engine has its limitations, but Google makes you well aware of them, so adapt to them and don't get frustrated or surprised by them two months into development, or host your service somewhere else like AWS.

[1]: http://searchyc.com/submissions/app+engine?page=1&sort=b...

As a reference point:

Our service (getmetricmail.com) which is running on GAE had an uptime of 99.97% in November.

I just turned on the "always on" feature for http://www.quickbrownfrog.com. Worth spending $9/month on. I imagine this will lead to lots of instant cash for Google from frustrated developers.

Now, if only we could have SSL...

Acording to the roadmap SSL should finally be here within the next 6 months. http://code.google.com/appengine/docs/roadmap.html

App Engine is great if you design your app within its limitation. It's just a different architecture to work with.

I haven't found the cold start or 30-second limit a problem yet. I have a cron job running periodically hitting the app to schedule tasks, that seems to always keep an instance running. The tasks were designed to do small things and finished in a short time. That actually helps to spread the tasks to more machines.

It does seem App Engine is running faster in the last week or so; it was kind of slow before. The new Play Back feature in my app http://www.previouslook.com actually works without much hiccup now.

i really wish they had a fulltext indexer available.

Hmm, I wonder if Google has tech to do that. Thinking...

Try websolr.com! Use coupon code SILVERTRIAL. If you hit me up at kyle@onemorecloud.com after you sign up, I'll switch you to a comped account.

I'll check it out!

Nice..

After writing a whole lot of scripts to do this myself, I'd been thinking this was a great market too.

Are you doing multiple Solr cores per JVM or does everyone get their own JVM?

Is i possible to try it out first, before commiting on the payment details? Like an index, with only 100 docs etc. I just want to get a feel for it would all tie in.

Channel API: “There is no concept of “presence” for clients connected to a channel. This means the application cannot receive a call when a client connects to or disconnects from a channel.”

Sigh... this reduces the awesomeness quite a bit :|

We'll get there.

  The developer who uploaded an app version can download that
  version's code using the appcfg.py download_app command.
  This feature can be disabled on a per application basis in 
  the admin console, under the 'Permissions' tab. Once 
  disabled, code download for the application CANNOT be re-
  enabled.

This sounds awesome too.

(source: http://code.google.com/p/googleappengine/wiki/SdkReleaseNote...)

This sounds nice, but it responds by:

    Error 403: --- begin server output ---
    You do not have permission to download this app version.

even though I am the only developer. I wonder if this applies to downloading apps uploaded by this version.

I just created a new throwaway site to confirm this. It seems you can't download your previously deployed apps, but only applications uploaded by v1.4.0.

This is not true (meaning, it worked for me). Are you sure you used a command something like this?

  $ appcfg.py download_app -A <app_id> <some_folder>

If it doesn't ask for credentials, you should probably add the --no_cookies flag.

Yes, I did just as you suggested. Strange. Still no go.

That is the really great part of the google brand. You can always bet that things will get better over time.

Yes! I've looked longingly at App Engine several times in the past but the 30 second limit was always a big fail. To me it's always been what separated App Engine from a platform for serving AdWords to a platform I could put a business intelligence app on.

Now if they could just put node on there!

If I remember correctly they had a cap of 30 seconds even for processing requests. Has this cap also been removed or it is only the cap on the task api that has been removed?

It's been only removed for Task API... But why would do you need more than 30 seconds to process normal request?

I was using it initially to do some scraping and it used to fail. But now I guess it can be done using the task api.

How often does one need/want requests to take longer than 30s? If it takes me more than ~5s to load a site, I typically go elsewhere.

AppEngine's task queue uses http requests to do offline processing.

In fact the new limits only seem to apply to these types of requests — not user facing requests.

http://code.google.com/appengine/docs/python/taskqueue/overv...

where is this Channel API I googled it didn't see anything.

I found short preview coverage at http://bitshaq.com/2010/09/01/sneak-peak-gae-channel-api/

The docs haven't been updated yet. Here this years IO video and pdf presentation:

http://www.google.com/events/io/2010/sessions/building-real-...

and here's the sample app (java):

http://code.google.com/p/dance-dance-robot

Live demo: http://dance-dance-robot.appspot.com/

Docs are on their way; just waiting for the push to complete.

Docs are here:

http://code.google.com/appengine/docs/python/channel/

http://code.google.com/appengine/docs/java/channel/

Its in the official docs http://code.google.com/appengine/docs/python/channel/

The Java Channel API documentation: http://code.google.com/appengine/docs/java/channel/

Reply on news.ycombinator.com