How Google Is Challenging AWS
stratechery.com
stratechery.com
Amazon really puts customers first. Their platform and organization are made up of small teams that own services with well-defined interfaces, accountable for customer metrics. All profits are reinvested, so resources and perks are scarce, efficiency matters, and management is tight. The platform emerged because internal teams thought of their infrastructure services as products with customers.
Google really puts ideas (or technology) first; it aims to hire the smartest people and rewards them for launching new things and solving complex problems rather than optimizing UX or making customers happy. Resources are ample and management is loose, so individual contributors can try new things with greater leisure. It's been compared to grad school. But simplifying customer experience is less of a priority, so the internal infrastructure was notoriously complex and hard to use. They're now learning to prioritize customers, but it's hard to change culture.
Of course, both companies are huge and diverse and evolving, so you'll find plenty of variance.
App Engine wasn't evidence of Google being a product company, nor does it exemplify the company's strategy. It was a grassroots project that for years didn't receive much leadership support, but was still allowed to launch and grow.
as someone who's dabbled a bit w both, I found GCP UI/UX to be far simpler than dealing w AWS
When dealing with a platform, this user experience coordination becomes much more important. (Think the old Apple ecosystem versus all the crap that came pre-installed on the Wintel systems)
This new Org stuff might be a recent exception though; I haven't read through it all and tested it to know. Hopefully.
Amazon gives each team independence. Therefore it is virtually impossible to insist on consistency between what different teams do. Each team makes sense on its own, but the whole can be very, very confusing.
Google has a process that results in much greater internal consistency. It may not be a great UX, but it is consistent. Inside and out.
For small systems, Amazon is going to give a better UX. But for a complex system, I prefer what Google will produce.
But Microsoft is far worse in that department.
Having worked there in teams near to their tablets I really think Amazon would have a hard time producing software of the complexity of Android or Chrome.
I found this quote from SEC filings. Jeff Bezos says:
> Service-oriented architecture -- or SOA -- is the fundamental building abstraction for Amazon technologies.
This was in 2010. https://www.sec.gov/Archives/edgar/data/1018724/000119312511...
It sounds like they have been committed to it at least since 2005? https://s3.amazonaws.com/aws001/trailhead/MigratingAmazonCom... I'd imagine internally it'd have to be a lot sooner than that because I think they were ready to release AWS by 2006.
Sorry, I don't mean to start a holy war. I just wonder if there's any connection... What do you think?
Silk is a different story.
App Engine was a precursor that came 5 years too early.
The hype about serverless is only starting now.
Choice of languages -- initially just Python and Java? Fiddly APIs, different from competing platforms but not actually super-simple for simple tasks? Lack of a straightforward way of running background tasks (still a bit of a mess)? Lack of management support? Maybe just underpowered at first for large sites, and insufficiently compelling for small sites to build a loyal fanbase?
I just built a new, very small project in App Engine (Python, standard environment). It works fine but the tools are quite fiddly. There are plenty of docs but they're a bit of a trainwreck, in the classic Google "the old way is deprecated, but the new way is still in beta" way (e.g. standard versus flexible environments).
I think a little time invested on customer service and user experience would have gone a long way.
That doesn't help though, as Googlers learn to write apps the Google way (massively horizontally scalable, managed NoSQL data service), which looks very much like App Engine. Outside, people still wanted to run their relational databases and large VMs, and App Engine didn't let them do that. That's why we came out with Compute Engine.
They had an issue where outgoing emails just disappeared, without errors, and no way to debug what was happening. Support denied the issue for quite some time. Wasn't fixed for 2 or 3 weeks.
I've been surprised how few people understand the value proposition, and how little competition there is. When I first heard about Azure I expected it to be PaaS, but it turned out to be Windows-first AWS.
With Appengine, you don't know anything about the hardware of software your application is running on - you simply upload your app.
I haven't used Azure, so please correct me if I'm wrong. I believe AWS Lambda is the beginnings of a "serverless" environment - maybe Azure has an equivalent.
Yet then you get into the trenches of it and (IMO) you realize the sum of its parts is much less then the value of the individual pieces. You feel the pain of the documentation writers who had to transcribe examples and helper libraries to ten languages, "beta" features that have been out for years, "examples coming soon" in README's that are two years old.
Want to use python3? That's cool, use the flexible environment. But it doesn't support taskqueues or many other features.
Need websockets? Thats cool, we kinda have this socket API and similar for some languages and environments. It doesn't really work in the flexible environment though sorry :X.
All our python examples are in framework X, that's sufficient for everyone using framework Y, right?
Don't get me wrong, my company uses GAE and its benefits outweigh the costs for us. But there is a very real "Googliness" to the failings of the platform. The shear breadth and requirements of "fixing" and iterating on GAE must not be a very fun project to work on.
I work on GCP on the Python samples.
We generally pick Flask, since our thinking is that for many API calls, it is pretty much identical code in other frameworks, and Flask has minimal boilerplate.
We have quick starts for Django for all our platforms. I think Flask + Django covers a huge chunk of Python frameworks people are using.
If you think we are missing important Python samples, you can file an issue here:
https://github.com/googlecloudplatform/python-docs-samples
There are a lot of potential samples to write, so any guidance on which ones people want is always helpful to us.
I keep a local branch of the python-docs-sample repo and just took a gander to refresh my memory. Specifically I see a quite a few examples using class based views based from the webapp2 package. I don't think it's unreasonable to have this as a major reference point, but it does require some extra documentation reading when converting to say, function based views in Flask.
Our personal use case is python3 in the flexible environment and I'd like to point out two notes while I have you here (if it's appropriate):
1) Are task queues coming to GAE Flexible environment - python? (and more over is feature parity coming between the google.appengine and google.cloud packages)
2) It's undocumented that the flexible environment of python requires a specific configuration variable to be set in order to make a connection to cloudSQL. I raised a support ticket for it a few weeks ago and the documentation hasn't been changed. It took me a few hours to debug it personally and I would like to save others the effort, can users make a pull request on the docs directly? For reference the variable is "beta_settings: cloud_sql_instances:" in app.yaml (it's present in the python-docs-samples but has no comment explaining its significance/requirement).
EDIT: I can no longer edit my original comment, but it seems GAE flexibly environment for python does support web sockets, though I would question the effectiveness of stateful servers in GAE. Of course thats an implementation problem and not one with GAE.
You're right my original comment was misleading, most of the GAE Standard samples are webapp2, but that's because it comes built-in to the platform and can be specified in the app.yaml, so webapp2 doesn't require people to `pip -t` to vendor Flask into the project. It might be worth revisiting if some of those samples should be in Flask or in both.
1) Guessing you already know you can use Pub/Sub for background tasks, examples here in our Bookshelf app:
https://cloud.google.com/python/getting-started/using-pub-su...
I think Product knows that the developer experience for tasks could be better and closer to Standard, but we haven't announced any public roadmap for task queues on Flexible.
2) I see references to that variable in our docs so I'm not sure where you're saying it's missing. Unfortunately you can't submit PRs for our docs, I wish you could.
Thanks again for the feedback, getting pretty off-topic so maybe good to take any further conversation to #python in GCP Slack?
https://googlecloud-community.slack.com https://gcp-slack.appspot.com/
What was the specific Python 3 issue? I didn't see anything in your post specifically about Python 3.
As far as I know there is no planned support for Python 3 in the standard env. I've been using GAE since 2010 but I'm a little uncomfortable continuing to writing new apps in Python 2.7 when they have a clear end of life date set now.
Given it might take you guys a year or two to support 3 after you decide to do it, then a year or two for me to port my apps over to python 3, my apps might be running for a while past the end of life date for Python 2.7.
Besides, I can't just keep writing 2.7 apps forever, so either I you guys have to update the SE to 3, or I need to start evaluating and comparing the flexible environment to what everybody else offers.
(I work on the Python developer experience for Google Cloud Platform)
You are using the terminology of products and platforms with the exact opposite meaning of the article's author.
Those "services with well-defined interfaces" become a platform others can use to build their own products. Similarly with the e-commerce and fulfillment infrastructure third party sellers can use. And maybe the transportation infrastructure in the future.
I also think you vastly underestimate the quality of Google's UX. Type any thing you can think of into one simple box, and you get surprisingly useful auto-completions, relevant spelling corrections, and almost always find the answer to the question you had when you started typing. There are any number of complex back end services working in parallel to resolve your query, with results gathered, ranked, merged, and rendered in a fraction of a second. Pretty hard to beat that user experience.
This is how the author defines product focus, emphasis on combining many components internally to provide an outstanding end user experience, without necessarily making the components available to be used by others to create their own user experience for their customers.
Google does make great products, especially when it's a matter of presenting a simple elegant interface to a complex internal system. I'm a big fan of Google Search and Maps UX, and Google Now.
But the UX of a product isn't just its immediate interface, it's also all the interactions you have with support, documentation, and change over time, and trust. The cultural differences are more evident there, though some teams at Google are getting pretty good at these things as well.
1 - Prediction API - you provide your own data there, so no data advantage from Google there.
2 - Cloud Natural Language API - the effectiveness really depends on what type of text you want to understand. If Google’s training data includes information about my type of text application great, but if it doesn’t then what? How do I know that?
3 - Cloud Vision API - likewise. Can I subset the training set? Provide my own examples? If they subset, can I inspect the examples?
4 - Translation API seems like the exception here, mainly because the odds are that customers of translation service are unlikely to have collected language pairs and this collection is more highly specialized. But it’s unclear that this one API would be the deciding factor for many companies choosing which cloud vendor to use.
ML services as a differentiator have yet to be proven out. I am highly suspect. Yes, some big general data sets will be better on some applications than others, but an enterprises’ own data about their problem will always be better than a huge, general data set. And if you’re using your own data anyway, you’re going to care about all the platformy things Amazon has already been winning with.
Barring proprietary breakthroughs in unsupervised learning, I don’t believe that this strategy as outlined will work in practice.
However, I'm not sure who's in the leading edge here while Microsoft seems to suggest as the leader, but in the end machine learning, deep learning will be commoditized with enough training already baked in.
ex. Need your web app translated flawlessly into Farsi? Just drop in farsi.js to your </head> and etc.
You can control the app, but you don't have control over the model.
It will be useful to some apps but I don't think it is going to be the secret weapon in Google's fight against AWS as Ben's article suggests. It's a neat argument but I think it ignores the reality of applications and machine learning.
If you have specific needs, then you can use TensorFlow running on the app engine (as they will soon be providing hosted and GPU-accelerated instances), which at worst makes it equal to Amazon offering... but something tells me the vast majority of Google Cloud customers will be satisfied with pre-trained models that can be applied on a very large swath of problems.
In this case you already have a bunch of historical labeled data and a pre-trained model is useless to you application. It doesn't help you that the pre-trained model can recognized 10 different types of cats, you need a model trained on photos of damaged cars. Obviously the insurance companies own photo data will be more useful here because it's data about the application domain.
Google has collected a ton of photos for the purpose of image search and consumer photo organizing and that models utility has been tuned to those application area.
The key question is what is the overlap between all applications of images models and what photos Google has collected.
There will be for some but my guess is that those are the mission critical, I can only get this performance from Google cloud are few and far between.
I'm not saying there aren't any. Ben's article suggests that Google's data is somehow going to be a mission critical asset for all applications areas. Which I think is a terribly naive idea when it comes to ML.
I'd argue that Amazon isn't a successful company, they are a popular company with a few large successes surrounded by decaying and decrepit failures that won't die. But then again I'm biased.
As far as Amazon's AWS strategy, I can't comment (I worked in the retail business side). But I can comment on a relatively small aspect of management that I witnessed. At one point in time I had a very strong need for PostGIS, and I lamented on an internal email list about AWS not having a Postgres version of RDS. I received an email directly from Raju Gulabani, VP of databases in AWS. He scheduled an appointment with me, him, and two product managers. He asked me pointed questions about why I wanted Postgres over the other options, how I would be using it, what extensions I wanted, and what features were important to me. He thanked me for my time, and less than a year later it was released to the public.
In the retail business side, I never had more than 2 minutes at a time with someone at the director level, and not once had I spoken to someone at the VP level. Literally zero communication from the bottom up, everything was top down. Whether AWS had already been working on it or not I don't know, but they definitely took the time to hear my case, and when it was released it was almost perfectly as I had asked for. And that, IMO is waaaay more important than anything regarding the size of a team or whatever the fluff pieces have attributed.
Couldn't you say the same thing about Google, or Microsoft? I think the point of 'success' is that your big wins outweigh your failures as determined by your revenue. Is it not?
Creating the largest retailer in the world and the largest cloud services platform in the world seem like two pretty big wins. Either one on its own would be an extremely successful company IMO.
Of course it is. And by that definition, Amazon isn't successful, and nowhere in the same league as Google or Amazon. Not yet, at least. They've managed to break even more or less, but it is still yet to be determined if they can become the wildly profitable company that their stock price suggests they can become. My opinion after working there is that AWS is to Alibaba as Amazon is to Yahoo.
Breaking out investment vs administrative cost is actually very hard to do. Is a programmer working on a new feature an R&D cost, or an administrative cost? What if it's a new service? What if it's a new product? What if it's a bug fix? What if it's a critical vulnerability? What if your programmer does all of the above at different times of the year? It's pretty much impossible to separate administrative overhead from research and development in tech companies, which is why they tend to not do it unless they are forced to. It's up to their shareholders or the SEC to force them to do it if it happens, which hasn't been the case for Amazon yet.
It is assumed that they would be turning a profit if they decided to just keep the lights on and not invest in the future. That's what they tell us, and that's what we see (new product and service announcements tell us as much). What we don't know from public information is whether they would be 10% more profitable or 10,000% more profitable. And that's before we know if their investments will pay off or if they become another perpetually subsidized program like Amazon Fresh. That's why Amazon stock is considered to be a speculative investment, whereas Google and Microsoft are more in the blue chip camp. My experience and hunch tells me that Amazon stock prices are at least 50% undeserved hype.
- Price/performance is better in some/many cases for VMs
- It's easy(ier?) to use
- Clear technical advantage with some of their other services e.g. Load balancers
- Customers prefer when there are multiple companies competing for their business
I get that there are long-term strategies that involve the likes of container services. But just the fact that they are better in some areas will help them get traction. Plus they have a fantastic brand name.
Distributing apps across multiple clouds is easy to say, hard to do. Each IaaS has peculiarities and wrinkles, different tradeoffs in performance and cost and so on. It takes a moderately tricky scheduling problem and turns it into a much gnarlier one.
What's easier is using high-level tools like Terraform and BOSH to manage installations on different IaaSes, and pushing apps to whichever one you like as you like. I can easily imagine setting up round-robin deploys.
That said: data has inertia. Any sensible architecture has to bear that in mind; typically apps will wind up living close to their datastores.
Disclosure: I work for Pivotal, we're the majority donor of engineering to Cloud Foundry.
"Do one thing and do it well" is underselling Heroku.
PaaSes have to do a lot of things and do them well.
The author loses his creditability since here.
`Transfer data to your Cloud Storage buckets from Amazon Simple Storage Service (S3), HTTP/HTTPS servers or other buckets. You can schedule once-off or daily transfers, and you can filter files based on name prefix and when they were changed.`
(I work for AWS)
Is one less informed than the other or what am I missing something? Why such disparity in opinion?
I've work on high performance networking file transfers. my experience is that most people who move data get very low utilization compared to the actual throughput of the network. People typically use one TCP connection, one process. high performance data transfers use thousands of TCP connections and thousands of processes.
Many other people underestimate the time/labor effort of dealing with a snowball.
You basically just proved my point: things aren't cheaper if you have to factor in employee cost.
But, really, I'm not trying to prove or disprove your point. Just noting that there was a situation for us where disk made sense, and we were satisfied with the outcome. Spending 4 hours of person time to save a thousand dollars was reasonable for us in a way it probably wouldn't be for many real companies, because we had comparatively little money and we're willing to work for peanuts.
(Note that I actually share your bias in this one. I both use GCP for my personal stuff and I'm writing this from a Google cafe. :-)
I've actually only heard positive things about snowball. I would be very interested in negative feedback.
(I work for amazon)
How is that supposed to work? Do they mean several 40gbit/s connections? Or did I misunderstand sth here?
At meaningful volumes, literally months faster than using tier 1 backbone, and a fraction of the cost as well.
So a back of envelope calculation says that it would take around 10 days to transfer 100 PiB over Gigabit ethernet. When you say immense, do you mean faster than that?
Have a great day.
Is there any validity to this? I don't do much OS-level programming, but is the Win32 API really that much more powerful and extensible?
I certainly don't think so (although there are some nice things about the Windows kernel, if one comes from a VMS background), and it's certainly not the reason for the success of Windows. Remember that MS-DOS beat Mac OS; Windows 1 beat Mac OS; Windows 3.1 beat Mac OS; Windows 95 beat Mac OS. None of those was technically any good whatsoever, and only one of those had a UI that was worth shaking a stick at.
The reasons for the success of Windows are non-technical: Bill Gates's mother served on a charity board with the chairman of IBM; business bought IBM computers because no-one ever got fired for buying IBM; IBM clones were cheaper than Macs; IBM clones were more extensible and hackable than Macs; Unix workstation vendors thought they could keep milking their cash cows; Microsoft engaged in noncompetitive behaviour; random chance.
Are Explorer and Finder "core"? Probably.
GCP Gold and Platinum support packages include phone support: https://cloud.google.com/support/
I think you get a lot more "kick the tires" flexibility with a moderately large up-front credit.
(affiliation: I'm an engineer on Compute Engine)
I will give Amazon that theirs extends out to a year, but summing up the costs of everything included in the free tier if you use all of it, I think it may still come out to less than $300 (or the equivalents on GCP would, anyway). For example, running an f1-micro for a year will cost you a bit under $60. If you add in another for Google Cloud SQL you're up to a total of about $150 over the course of a year. What they offer you in S3 is basically free (<$2 over the course of a year).
It's possible the other services are a better deal than compute and storage, if you have a use for them, but the GCP free trial lets you allocate that $300 however you like. You can scale up more in that 60 days than you can within the AWS free tier. To me, personally, this strikes me as more valuable if I'm trying to sketch out a new product — I'd rather not try to figure out how to fit inside the AWS free tier resource envelope and instead understand how my costs are scaling with the resources I'm using while not being on the hook for those costs (up to the $300 size of the credit, obviously) for the first couple months. Especially to absorb things like shaking out automation -- go ahead, spin up a GKE cluster, scale it out to 5x the size you currently need, run some quick load tests, and then scale it back down 30 minutes later (and only pay for that 30 minutes).
(affiliation: I'm an engineer on Compute Engine)
If you told me to use Azure two years ago I would've laughed you out of the room. But here I am in 2016, using Azure, using ASP.net + IIS on Visual Studio. that's some powerful shit and currently AWS has cost leadership and perceived switching cost as their edge.
By introducing a layer of learning curve, you lock in your customers but eventually the other guys will race to lower that curve.
However, Google Cloud really hasn't entered my mind as much as Azure has this year. AWS has always been there. I'm not sure as to why this is maybe I have built a perception that GCE is more expensive and my documentation experience with Google wasn't anymore smoother than AWS.
One thing for sure, Build 2016 earlier this year was one of the key driver for my conversion + Nadella's leadership.
Google Cloud has always been obscured by AWS and now Azure in my mind and so far, they've yet to really jump out at me like AWS does on HN regularly which Azure is now catching up.
This takes a huge cognitive load off the developer who won't have to do context switching between portal.zure.com and VS.
But portal.azure.com is a much better, smooth, streamlined & intuitive interface than AWS, and I find myself wanting to work with Azure more and more.
There is still a stickiness to AWS and stuff like Cognito seems way more less intrusive from brand point of view (MS AD redirects you to onmicrosoftonline.com when logging in user)
Ah, the interface that makes you scroll in all 4 directions on a small screen. I get slightly dizzy using it.
[1]: http://thenextweb.com/dd/2016/07/14/amazon-buys-cloud9-aws/
a cloud based IDE is certainly lighter than VS. the fact that it's accessible anywhere is a huge plus.
I would still like to see a desktop version of C9, sort of like Atom & Microsoft Visual Code.
It's going to be hard to justify using Google Cloud after this when I know I'm most likely to ditch Azure in the process as well.
When the answer to your tech problem is not ASP.net/SQL Server, you are going to find their services much more difficult to put up with compared to the competition.
Before someone jumps in with ".Net is huge in the X space" thing: There has been double digit growth of close to a decade now of platforms that don't run .Net. That ecosphere was a giant. Now it isn't.
As you do this, I personally think that you are much better off getting the JetBrains all-in-one subscription (if you can afford it) over continuing to use Visual Studio. There are a lot of things happening on the data science front which are just that much harder to do within the MS stack.
Actually I really love the C# language, and wish there were .NET ports for Solr, Hadoop and NLP libraries etc. But it just makes more sense to get into the native ecosystems for these libraries (Java, Python etc.) with the most convenient IDEs for those languages (e.g. IntelliJ, PyCharm) and not struggle with trying to get all your NuGet ducks in a row.
https://cloudplatform.googleblog.com/2016/10/introducing-Goo...
Google Cloud UI is vastly superior to AWS. It's clear to me AWS didn't put a lot of effort into their interface, Google console is nice too in order to quickly experiment with the platform. On the other hand, it seems to me that AWS is still cheaper than GCloud right now.
How so?
I thought this was brought up just a week ago saying GCloud is consistently cheaper:
(I work for AWS)
Is that not true?
@ranman: You could start by disclosing that you work for AWS ;)
I'll stand by my word. I can present the pricing comparison at reinvent 2017 if you'd like.
1) The equivalent instances are cheaper on Google.
2) Automatic discount is significantly superior and more customer friendly that reserved instances.
3) Google is faster or more flexible, usually both. That means that when you have specifications to achieve (IOPS/bandwidth/SSD), AWS has to get severely over-provisioned compared to Google cloud.
1 + 2 + 3 = https://thehftguy.files.wordpress.com/2016/11/aws-vs-gce-pri... the actual difference I expect on the bills depending on the type of workload ;)
>Pokemon Go was able to scale to facebook level user engagement in a mere month time period with 4 backend engineers (and of course with lot of help from Google).
I mean... did they? I was trying to play for weeks and I couldn't even login. I loved the game when I was able to play but I don't think they scaled seamlessly. Maybe that had nothing to do with the cloud provider and had more to do with the application itself (I don't know) -- but I wouldn't personally use Pokemon Go as an example of successful scaling.
>Things like these are impossible to achieve with AWS.
Twilio, Slack, AirBnB, lyft, duolingo, FINRA, yelp, pinterest, foursquare, adroll, shazam, supercell, etc. etc. etc.
https://aws.amazon.com/solutions/case-studies/
I'm curious why you think these things aren't possible on AWS? They really are... and I can think of hundreds of examples.
Regardless, it's important to recognize that the cloud provider is only ONE piece in your ability to scale. I have examples of failures on GCE and on AWS. Your application architecture is far more important than the cloud provider you choose when it comes to scaling quickly like this. Sometimes it's not worth the dev effort to be prepared for these things.
>"Fraction of the time", "half of the AWS's price"
Nope. As outlined in my linked post above, the comparisons in the article are not accurate.
James Watters, Pivotal SVP says they saved 50% by moving from AWS to Google Cloud. For an 8 figures bill, that's a lot.
He said this 2 days ago at re:invent, FWIW.
Yes. This is exactly why I keep pushing for Google Cloud.
I've found a way to talk about costs with understandable graphs, I didn't find a way yet to talk about the huge gains because it's easier to manage.
tl; dr: Even matching instance types more closely, for on-demand pricing the 50% number held up remarkably well, although I'm worried that I got something wrong with my math on provisioned IOPS pricing, as a 10x difference seems unbelievably high.
Executives love slick UIs and flashy dashboards. Executives have a lot of power.
What world do you live in that a random dev picks the cloud provider for a company?
If we're profitable, we're going to invest in a better working experience so that our employees spend minimal time 'dealing with' a tool's 'issues'. -because we can.
If we're a smaller pre-profit/cash-strapped/VC-beholdened company everyone is going to be expected to tolerate working with 'rough' tools if it can save a few bucks.
GCE Pricing Page: https://cloud.google.com/compute/pricing
Google's most prominent products are possible only because they could deliver those with the unprecedented computing efficiency. This is not to be found anywhere else. For example, search, Gmail, Youtube.
Before AWS, Amazon care about efficiency. But fundamentally, Amazon do not depend on computing efficiency to support the company's bottom line, or at least only in a degree that is far less important compared to Google.
Most modern companies don't need cheap computing. They have a lot of users generating revenues, and they usually need little computing resources.
Google always worked by giving everything free to people. (Google, Gmail, Maps, Youtube, Android, Chrome). And they have extremely infrastructure intensive applications to run (e.g. just gotta copy the entire internet to index it + serve years of videos per second :D).
For every paid click/page a user will see, he will go through hundreds of page paying nothing, and Google will have to make thousands of pre-computations to be able to serve it in the first place.
That's the world Google lives in. They had to be hyper efficient since day 1 or they couldn't survive.
(Also note that they started > 15 years ago. The available hardware was 2^5 smaller at the time).
To be serious this is a catastrophic failure. We can not deprive people of there freedom because of a glitch. This system doesn't have fail safes.