Background processing coming to Google AppEngine

zef.me

17 points·mdasen··5 comments

Great news, I have been waiting for this. GAE still has a long way to go, but this is a feature that I'm sure a lot of developers were waiting for.

They're slowly getting there. But I think Bulk Data operations are a lot more important than background tasks, and it's sad that there's still no progress there.

Until you can do "delete from Stapler where color='red'" or "update Building set onFire=false where isMine=true", AppEngine is still basically unsuitable to the task of hosting web applications.

I would say this solves this by allowing you to batch segments of that bulk... a streaming bulk transfer may not be possible because you don't know what server the data is on, and what order it comes in, etc... Google has had a lot of success building web applications using this generic, split-up way of storing data... it sucks for one box, but for their cluster of so many, that makes up for it. I haven't had problems with the example queries like what you're asking for, if the dataset that it modifies is small.

It only really becomes a problem when you add a column to a table and want to fill in values for all the existing data. Or when you want to truncate a table.

This has been missing on Heroku as well. I haven't needed it myself, yet, but it's been discussed frequently on the mailing list.

Reply on news.ycombinator.com