Hacker News Reader

Comment by bsaul | Hacker News Reader

Has anybody had successful experience deploying docker containers on appengine ? Last time i tried, i had such a bad experience in terms of deployment speed ( time to build the image, then upload it, then waiting for the stuff to deploy) that i reverted to managing my own gce instance.

But maybe i had bad luck..

View on HN

I just tried to move a rails app to appengine. It uses the flexible environment (eg, docker). It took like 10 or 15 minutes to deploy, each time.

Heroku does my deploy in about 5 mins.

https://groups.google.com/forum/#!topic/google-appengine/hZM...

GCloud is cheaper though.

Also, VMs spin up in GCloud amazingly fast. Like 5 seconds. Feels like somebody at Gcloud just needs to go and fix this. No reason this is so bad...

Deploy is 15 seconds on GKE. Wonder why app engine is so bad.

With GKE you don't necessarily update your load balancer rules with a deploy correct? The linked thread points the blame for app engine deploys on waiting for GCLB to update.

Correct, you have software based load balancing under the covers.

My Google Load Balancers never move.. It is a single thing that points each node (physical machine) in the cluster, and distributes traffic between then.

Each node knows how to route traffic to each app. So when I deploy that app, the software load balancer at the node level will slowly move traffic over from old app to new app. Entire thing is MAGICAL. And 0 downtime, very very fast deploys.

Edit - But yes this explains iy. Changing the google load balancers is like a 5 minute ordeal. Total pain. Nice that with GKE you only need to touch them when your node count changes, which can be very rare (~monthly for me)

Yes uploads are fast and you can spin up a VM directly from an image now

How many minutes/seconds do you mean by fast? Can you point to guidance on how to make that happen because I am seeing 8+ minute re-deploy times.

Funny you mention that, I was just experiencing this yesterday in the flex environment. Re-deploy time for a 3 character text change on one page is 8+ minutes for each re-deploy. This is insane.

Does anyone know how to speed that up?

(Google cloud support) Unfortunately, this is a pain point that has no easy solution at the moment. It doesn't matter how trivial a change is, it isn't the docker deployment that is taking all that time. As mentioned above, the bottleneck is updating the GC load balancer with the new routing rules, which takes time to propagate throughout the system. This is a high priority issue internally, but updating the load balancers is no trivial task, and will take a lot of time and testing.

In the meantime, I recommend the following mitigation strategies:

1. Try to get into the habit of carefully reviewing and testing new versions locally before deployment. Client libraries should still work if you have a valid default application credential set up. I say this because I have a hard time remembering to do this as well.

2. Static content and templates for your site should be hosted on GCS, not deployed with your app in a "static/" folder or something. Easy to fix typos, HTML/CSS, and JS errors by simply using gsutil to copy the fixed file over, takes only a second.

3. Always keep a stable version of your app available in case you broke something in a new deployment. It's quicker to route traffic to an older version than it is to track down a bug and wait for the fix to finish being deployed.

Not ideal, but again, Googlers have to suffer this too, and are very motivated to find a way to fix this.

Great answer, thanks!

Reply on news.ycombinator.com