Google confirms an issue with WebView is crashing many Android apps (Fixed) | Hacker News Reader

Google confirms an issue with WebView is crashing many Android apps (Fixed)

xda-developers.com

88 pointsneversaydie43 comments

There is a misconception that removing Android Webview is "fixing" the problem. The proper word is that removing the updated Android Webview is "a temporary workaround" until an updated Android Webview is released.

You cannot remove Android Webview because it is an essential part of Android. By "removing" the package, you are removing the latest update and revert back to the old (and probably insecure) version of Android Webview that came with the initial Android version of the phone.

In the end, Google eventually released a new package now, so you can perform the update of Android Webview to the latest version.

Dont see an update option for this app in the app store. It says last release was March 12th. How exactly do I get this update?

I was able to just do a Chrome update to solve the issue on my phone.

No action required if you’re not experiencing issues.

If you’re affected and not able to update, trying to “uninstall updates” for “Android System WebView” will revert it to ROM-included unaffected version and solve it.

That sounds very risky, since untrusted code shown in any webview can then exploit long fixed chrome bugs.

I uninstalled the updates and was then able to install the March 22 version. It didn't offer any update until I did that.

I assume that this is uninstalling updates, which means that you are back to the WebView that came with your phone. This obviously opens a whole new can of worms as now you may be years of security updates behind.

I really wish that popular stores had an option to roll back one version until the next update came out.

This is why auto-update is turned off for me. You just cant trust blindly updating things because there is no undo button for breaking changes.

How do you evaluate updates? And how do you rollback? It seems like you are still basically relying on "find the APK somewhere" for rollbacks still.

Really if I do not need new functionality and it's not a sensitive app (i.e. one that deals with funds, email, text, etc..) I just dont. Some of my APKs are REALLY REALLY old. If i get a new phone, I save them off and roll them over.

If its for an app that has potentially significant security issues I'll wait a while and monitor the reviews and check the web before I update (I'll also backup the installed APK before updating) so I can rollback if I need to. But the number of apps on my phone for this is very slim.

It is definitely fixing the problem. As is the entire phone is unusable because you cannot open many of the most important apps.

Actually just noticed that on my wife's phone:

03-23 14:47:11.393 21102 21102 F DEBUG : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0

03-23 14:47:11.393 21102 21102 F DEBUG : Cause: null pointer dereference

03-23 14:47:11.393 21102 21102 F DEBUG : x0 0000000000000000 x1 00000077a53d71ef x2 7265646f63654400 x3 726f7463656c6553

[...]

03-23 14:47:11.395 21102 21102 F DEBUG : backtrace:

03-23 14:47:11.395 21102 21102 F DEBUG : #00 pc 00000000034521d0 /data/app/com.android.chrome-xLjRNvGCAeF2ZEpHu2gC4g==/base.apk (offset 0xa0f000)

Pretty bad. Affecting various apps including GMail. Solved by updating Chrome. Luckily for them (and us) Google Play was not affected.

> Cause: null pointer dereference

that is actually pretty funny :')

Well, I'm glad to see this posted here... I was concerned that somehow my phone got infected with some kind of malware after a reboot didn't fix it. It's concerning to me that an update was pushed out that apparently wasn't tested very well.

Still seeing 100x the normal crash rate across all our Android apps that use Webview on devices that have either com.android.chrome or com.google.android.webview 89.0.4389.90 installed.

Started yesterday around 2 PM US/Eastern.

How does something like this make it to production? I couldn't reply to any work emails for the greater part of the evening yesterday.

Adds more evidence to the case of "never use Google for your business".

I guess the team has Q1 OKR to meet and/or error budget to spend? Either way, it's bad.

I am not saying the bug would have been prevented if the team operates differently. Brown paper bag release happens. However, the way this incident blew up suggests that the bad release made it to too many users in too short of a time, exactly the behavior incentivized by meeting (or exceeding) an OKR while having some spare error budget.

What do you think "error budget" means? (Hint, it doesn't mean "amount of intentionally deployed errors we allow".)

Do you believe WebView should never update because it's never OK to spend error budget?

That is a strawman argument. Of course WebView should update. The question is whether the structure of incentives contributed to a risky release.

If you "play the game", an error budget causes your releases to become riskier and riskier as the budget grows. This is optimal because a riskier release allows you to accelerate changes, allowing you to hit other OKR.

The issue is that this will eventually cause very risky releases (where you would slow down otherwise). In a sense, an error budget does eventually become "amount of intentionally deployed errors" because you are incentivized to increase risk until an error does appear.

Interestingly, the Google SRE book[1] mentions scenarios where they practice exactly this:

> The solution to this Chubby scenario is interesting: SRE makes sure that global Chubby meets, but does not significantly exceed, its service level objective. In any given quarter, if a true failure has not dropped availability below the target, a controlled outage will be synthesized by intentionally taking down the system.

I doubt there are any customer-facing systems that practice this technique. Just super interesting that it exists.

[1]: https://sre.google/sre-book/service-level-objectives/#xref_r...

You never made a mistake that made it through your test suite to production?

Not on something that everyone depends on that increased crashes by an order of magnitude?

Even some basic testing on a real android device would've caught this

Google is known to A/B test about everything, also, they very often release new updates to core features like this to 0.01% of users first.

So it is very weird that they released this so (apparently?) a big chunk of their users

Google products don't really need to move "fast and break" things anymore, they just need to work.

Maybe it didn’t make it to production? The affected users might have been in the canary population?

The problem is, I didn't issue an update on my android phone this week. I don't even have a Google play account and use ApkPure.

Yet this morning my applications started to break.

How did the update occured on my phone ?

IIRC system components can be OTA updated in the background. The only way to get around that is to use a de-googled android distro.

At least you can write/bundle your own render engine on Android, something which is not possible on iOS.

But if your app is now 200MB instead of 2MB, know that there are a significant number of users that simply won’t install your app (or will remove it) because they don’t have either space or bandwidth. Cheaper Android phones regularly have almost no spare space (like, well under 2GB available to user space when empty, even though the phone may have been advertised as having 8GB, and even that 2GB may well be partitioned stupidly so that you have less than 1GB available to install apps on).

Isn't the whole point of WebView that Firefox et al can provide it as well as Chrome?

Yes, Firefox can act as a WebView implementation on any Android phone. Apps don't need to ship this.

No it can’t. https://wiki.mozilla.org/Mobile/GeckoView: “GeckoView serves a similar purpose to Android's built-in WebView, but it has its own APIs and is not a drop in replacement.”

The purpose of WebView has certainly never been about using a different engine; rather, it’s to have only one copy of a browser for apps to use as a widget, rather than each app bundling its own browser which uses vast amounts of space and raises serious security concerns.

The situation I’m responding to is the suggestion that you bundle your own browser rather than using one provided by Android System WebView or equivalent.

Not really, though vendors of Android can provide alternative web views (of course). Samsung used to (may still) which was a giant pain in the ass, since you'd see different behavior out of web views on Samsung versus almost any other Android device, but Samsung was too common to ignore, so if you did cross-platform apps that included web views you ended up with platform specific bugs for iOS, then for two flavors of web view on Android—and then multiply that by the many versions of Android you'd have to support, for Samsung and everyone else, versus maybe 2 versions for iOS for non-super-huge-userbase applications (where those sub-1% users not on the two most recent versions are worth spending money to support, because you have so many users that 0.5% or whatever is still a ton of people)

If anyone could provide web views they'd be practically useless, as they'd just be a constant source of bugs, and everyone would simply start embedding a browser engine instead.

Can. As opposed to both can't and must.

So you can bundle your own web view, if you want. You don't have to. You have the option.

I wonder if Google's QA is getting worse. I had a broken Android Google Maps update last week where street view had multiple severe bugs: it didn't even show the blue lines when enabling the street view layer, split screen didn't show the marker for the virtual observer's location, and tapping in the map in split screen just rotated the view instead of moving the viewer to the new location. Something this broken should never, ever make it out the door.

I attended a webinar from a famous e-commerce platform. They mentioned that they no longer have QA and make use of CI/CD pipelines (edit: canary deployment, feature flags, etc.) to move fast and break things.

I assumed that everyone else no longer has QA and just pushes emergency fixes in a JIT fashion. I also assumed that our company is traditional and backwards for still having QA.

Dedicated QA is amazing and it blows my mind that people think that CI is a 1:1 replacement. CI is great, but testing is for the bugs you anticipate. Good QA people find all the bugs you never even thought of.

Experienced this this morning on my Pixel 1. Woke up to several apps in a crash loop every few seconds and most others crashing on startup. After a reboot didn't solve anything I thought perhaps my (older) phone had suffered a RAM failure since I hadn't updated anything in several days. After finding an article about the issue I updated Chrome which corrected the issue.

This started happening for me last night. Just updated webview to 89.0.4389.105 and everything works now.

My phone rebooted a few times during the last week.

At first I thought it was just planned obsolescence killing a three year old device, but fortunately that doesn't seem to be the case.

I don't believe that a system reboot is a symptom of this issue, just app crashes. Also, the bug started yesterday afternoon. I doubt this has anything to do with your system reboots.

does anyone know if they released any technical details? isn't webview open source?

Reply on news.ycombinator.com