Corrupt App Store binaries crashing on launch | Hacker News Reader

Corrupt App Store binaries crashing on launch

marco.org

147 pointsshawndumas49 comments

This is also happening to our app, Gaia GPS, and it's pretty much ruining my holiday.

It seemed to effect US users around noon PDT, and then a batch of international users around 5 pm PDT.

I blogged for us: http://www.gaiagps.com/news/article/Bad%20App%20Store%20bina...

I also sent the App Review team an email asking if they will expunge all the bad reviews once this is over, but I doubt that's going to happen.

Not part of this, but I have experienced in the past sporadic download corruptions with users. I've never figured out the exact cause, but a reinstall has always solved it. It's been roughly 10-15 users out of 25000 or so purchases. My best guess is that some part of the distribution pipeline does not fully validate the app package and re-download it if it fails.

I've seen download corruption effecting apps from the Mac App Store.

This is a tough one, but I think that should be affecting.

It was a typo and I really think you should avoid correcting people's grammar on this forum. It doesn't add anything to the discussion.

I noticed something even odder when I was updating Instapaper from 4.2.2 to 4.2.3 on my iPad 2, iOS 6 beta 2 yesterday: Instapaper is about 25 megabytes, and I'm on a desperately slow network. It should've ended in no less than 5 minutes, but it just took over 15 seconds.

I'm certain that I'm on iOS 6 ( :D ), and I'm certain that I updated Instapaper (4.2.2 crashed on lunch in iOS 6, that's why he sent out 4.2.3 and now I can open Instapaper), and that my network is slow and it shouldn't have taken less than 5 minutes. So, what's the story? To my knowledge, delta updates are still not available to iOS apps for some reason. And I couldn't find anything about them in devforums/iOS 6 docs/WWDC sessions.

So, could it be relevant? Maybe they were testing delta updates by mistake and some apps got burned?

Doesn't iOS 6 add delta updates for apps? Or was that still only for system updates?

To best of my knowledge, it's still only for OS itself. Searching for 'delta' in devforums.apple.com and developer library doesn't seem to return anything relevant, and I've watched or at least skimmed through most WWDC sessions and it wasn't mentioned anywhere.

But I'm not an active iOS developer at the moment and might be mistaken, or haven't looked enough.

I have two waiting-for-review updates that have been sitting in the queue for 9 days now. This seems much longer than my past experience of 2-4 days turn-around. I wonder if these delays are related to the corrupt updates issue.

We had a app finally get thought the review queue last week after a 12 day wait. Maybe there's a recent surge of apps hitting the store post-WWDC? No problems with corrupted binaries for us though.

Wow 12 days. Haven't had to wait that long for a few years now. Probably also a slew of updates due to developers updating keywords caused by the recent change to the App Store search algorithms.

http://news.ycombinator.com/item?id=4153610

Does someone at Apple manually check the whole app when you submit an update?

Yes, they have human reviewers launch the app and tap around a bit, in addition to automated code analysis.

that 'automated code analysis' also let a 'Flashlight' app that was, actually, a covert tethering app go live.

As an app developer myself, I never said that it was good review :-)

Yes, absolutely. In fact, based on the fact that my apps frequently sit in 'In Review' (versus 'Waiting for Review') for 12+ hours usually, it wouldn't shock me if they had multiple people review each submission.

How much they play around with the app seems to vary - I've had odd, hard-to-trigger bugs get my app rejected, even though they existed for several prior versions.

Each app submission/update costs Apple a fair amount of money to review.

It definitely seems like multiple people can be involved if there is uncertainty on whether to reject the app. From experience, I've found that if my app is sitting in "In Review" status for more than a certain number of hours, it's likely to get rejected.

The only time Apple gave out hard numbers on their review setup, a couple of years ago, they had two reviewers check each submission, with an average of 6.5 minutes per reviewer per submission spent checking it. Obviously, that average contains quite a bit of variation.

That checks out with my experience. Typically an app will go from 'in review' to 'processing for the app store' in about 15 mins.

Interesting. I don't think I've ever had one take < 1 hour, and most are 3+ hours from 'In Review' to 'Processing for App Store' or 'Rejected'. 12-24 hours is fairly common for me, and I just had one approved the other day that was 'In Review' for ~48 hours.

My app always takes many hours to review. There are a lot of buttons.

They also check more expensive apps more thoroughly, according to the iOS developer guidelines.

another source for submission times http://reviewtimes.shinydevelopment.com/

Updates to my main app usually took between 3-5 days. The last update took 28 days. (just released one week ago)

One of my apps went into "In Review" status at midnight (19 hours ago) and still hasn't been rejected or approved. The rejection or approval has always happened in less than 12 hours for me, so it seems like Apple might be holding all approvals until they've resolved this issue. At least, I hope that's the case... I really feel for the developers who got a slew of one-star reviews because of this.

I've had apps "In Review" for a couple days.

Months here.

Months too. Eventually got released though.

It is worth emailing them if it takes longer than a few days. There is a process where after a couple of days, it should go into extended review, and they will email you.

Any developers know how to change an app update that is "Waiting for Review" from "Automatically Release" to "Hold for Developer Release?"

Not possible without rejecting the binary unfortunately. If you submitted it a day ago or so, just reject it. It'll get dropped down in the queue but if the situation calls for it, might be necessary.

In the same spot. We have multiple apps with updates, partly to get our keywords back together after the "Chomp" store search update. I had been hoping we'd get reviewed in the next day or two, but now I'm happy to wait a few more days so we don't hit this issue.

Just reject the binary -- better to wait a week for the update than to be in this situation.

I think you are right. Was just browsing the app store and seeing a number of apps getting 1-star reviews because of recent updates.

My update has been "waiting for review" for a week. I'm going to just let it proceed. When the update goes live I'll test it and if it's corrupted I'll just remove it from sale. Hopefully this will prevent users from updating. Then I'll email support and wait until the issue is resolved.

Funnily enough, updates can still be received if an app is removed from sale or even deleted by the developer.

Don't play games with this one. If I were in your shoes, I'd pull my binary, too.

You do not want to be waiting on the App Store Review team to fix this. I'm one of the developers in Marco's post; it's been 24 hours and all I've gotten is a boilerplate response from the App Review team.

Ugh, the need to delete the corrupt version and then install the good version also gets rid of all user data. What a mess.

Of course, this could be fixed by slightly increasing the version number or some other way to let iOS know that the good version is to be installed as an update over the broken one.

Hopefully no one was relying on that income or those good reviews. More the reason why developers should make sure diversify their product strategy.

This is actually quite common and appears to resolve itself within a few hours after the update goes live on the app store.

I just updated the "CUE" app in the last few days and it crashes on launch every single time.

Something similar has happened to me. Apple needs to address this issue immediately.

I suspect the issue isn't with the store corrupting binaries but the application servers being under heavily load and dropping connections to the user. Begs the question why they aren't doing MD5 validation of the binaries before launching and notifying the user.

It is 4th July holiday after all. Lot more traffic.

They're distributed as .zip files (renamed .ipa), so a corrupted file or bad download wouldn't extract anyway. The real issue here is in their DRM: they have to re-encrypt the binary for each user, and the encryption seems to be incorrect in some cases. That's also consistent with the FairPlay log error in Marco's post.

If that were the case then it would affect all apps rather than a small handful of apps that updated yesterday.

Christmas day is the big one for the App Store. I doubt July 4 comes very close to that.

July 4 is actually bigger than Christmas for our apps. We have an outdoorsy app.

Not necessarily. All the apps are cached by Akamai so it could be that newly created app bundles get retrieved directly from the source.

Also July 4 is a pretty big holiday as far as usage patterns go.

I don't think that's possible.

Reply on news.ycombinator.com