Hacker News Reader

Comment by revisionzero | Hacker News Reader

No idea how this patch passed QA Testing internally.

Though I've pointed out before that I don't think formal testing is a big part of Apple culture, I'm at a loss to come up with how this passed the most basic smoke testing. Or to flip it around, what random variable wasn't accounted for that allowed the devices in the test matrix to pass? Sure, it seems obvious that Apple just didn't test it. But c'mon, we know that can't be true. They must have loaded on some iPhone 6s. So what's the missing piece?

The update works fine if updated via iTunes. Only OTA updates are broken. Maybe they didn't test the OTA update process? It sounds absurd but it's the only viable option in my mind.

It's surprising to me that those would even differ in the first place. You'd think that you'd have a single deliverable which is delivered and signed, and the installer does the same thing regardless of where it gets the package. What makes OTA different from iTunes?

The iTunes update is the entire OS patched, so it's like a 2GB download. The OTA update is a delta update, which is only 75MB or so.

Ah, got it!

Pure 100% speculation: Maybe this only impacted some carriers and maybe Apple only tests on a subset (e.g. AT&T was tested, but T-Mobile broke, etc).

Would be interesting to hear what carrier it broke on for people.

I have no insight on internal culture, but Apple does have formal QA teams. I'd be surprised if they weren't testing.

People underestimate how hard it can be to catch problems that only manifest on a particular network setup, etc., even with a formal test team.

The obvious one would be carriers, especially since the error affects the phone part. I have a hard time imagining a "basic smoke test" when it comes to a phone that has to actually try to connect to an external network, different depending on locations and contracts.

I agree, carriers might be the likely candidate. As for a smoke test, I'm seeing reports from users on US carriers. If it were "phones break on some obscure carrier in China", okay, fine. But when it appears to be breaking on carriers that are available to the testing labs right there in Cupertino, meh, it's got to be at least one other thing.

As another commenter said, I'd love to see a post morteum strictly in the interests of "don't let this happen to you". Sadly, we're unlikely to ever see it.

AFAIK the issue only affected OTA updates. I suspect that the vast majority of testing occurred without using OTA. I wouldn't be surprised if they are a bit more aggressive about testing OTA updates too after this.

This never would've happened if Steve Jobs were alive.

Apple pushed several broken updates under Jobs. Remember the iTunes update that wiped hard drives?

I haven't heard of that before now, and now I am super curious. Is there an article that details what happened?

http://hints.macworld.com/article.php?story=2001110223300292

So the root cause of this is similar to that of the broken Bumbeebee upgrade script [1,2]: a typo (missing quotation marks or unintended space) caused far more data than intended (a whole volume or /usr) to be rm'd.

1: https://news.ycombinator.com/item?id=2661209 2: https://github.com/MrMEEE/bumblebee-Old-and-abbandoned/commi...

I think it was a joke.

Reply on news.ycombinator.com