Previously Downloaded OS X Installers No Longer Work | Hacker News Reader

Previously Downloaded OS X Installers No Longer Work

tidbits.com

194 pointsingve100 comments

I was bitten by this this week. Tried to reuse my installer to install OS X El Capitan on a MBP without redownloading it over my super-fast 2Mbps internet connection.

First, I try to install it by just copying the installer app - "can't be verified".

Then I make a bootable USB stick using DiskMaker X - "can't be verified". I run an integrity check on the installer - all good.

I then try running `/path/to/Install\ OS\ X\ El\ Capitan.app/Contents/Resources/createinstallmedia --volume /Volumes/USB-STICK --applicationpath /path/to/Install\ OS\ X\ El\ Capitan.app --nointeraction` - "can't be verified".

Finally I give up and the system is installed via the App Store. The download fails once, it finally completes, and I am left clueless as to why I couldn't reuse the old installer.

An "expired certificate" error message would have saved me many hours. It's unfortunate for the user that Apple puts so little emphasis on letting them be their own tech support.

Apple is just awful about providing meaningful, useful errors. The worst is AirPlay. When it fails to connect, which is about 33% of the time, there's nothing. It doesn't even pop up a generic alert, it just silently (or not so silently, if you're playing music) reverts to local playback.

Many other examples abound. Most errors at least provide a message, but one so generic as to be useless.

I go back to the days of System Bombs and Unimplemented Trap errors.

It was maddening.

Ahh the bomb, a great (Susan Kare) icon:

https://en.m.wikipedia.org/wiki/Bomb_(icon)

MacsBug was an essential utility back in those days.

I never got around to learning to use MacsBug.

I just stopped running applications that caused too many crashes.

I miss those days dearly. I LOVED being a Mac user during the 1990s.

That's pretty terrible. No software performs flawlessly, so good error messages are a key part of usability.

I think Apple is trending away from being usable and towards looking usable. Simple, friendly error messages look much better....

or TimeMachine

"sparsebundle already in use" is the FOAD of error messages. I do love how the (now stale) tools to fix this are maintained by a volunteer on an external site unrelated to apple.

I try not to be too grumpy about it, but I paid $3k for a laptop not to have to fucking deal with windows-style normal operation of the OS and related tools is busted and you're gonna sink hours into debugging it. And least when linux breaks it tends to leave error messages and details in syslog...

My latest pet peeve is `mds_stores` deciding it needs to spend forever using my iPod, so I can't unplug it without incurring a scolding. It's not hard for Mac OS to stay less terrible to configure than Linux, and less terrible to use than Windows, but that's a low bar.

Do you have your iPod configured in iTunes with "Enable disk use"? Sounds like Spotlight is trying to index it or something...

I probably do. Thanks to my partial understanding of how the software works, I can guess that `mds_` is Spotlight trying to index my music player, but I wouldn't want to try to explain it to my dad. Software that responds to "eject this" with "no, because f* you" because it is busy creating meta-data is user-hostile.

I switched from OSX Yosemite to Windows 8.1 a while back.

I have had substantially fewer problems with Windows 8.1.

hmm. windows 10's inability to not talk to microsoft skeeves me out though.

Just yesterday I performed a clean install of Windows 10 on my desktop work computer. After joining our domain (configured in the most standard way possible), the start menu stopped working and edge couldn't start anymore. All commands to fix this situation (through powershell) gave cryptic hexadecimal error codes. After a few hours it turns out the Windows Firewall service was disabled. How this causes the Start Menu to malfunction, I don't know...

Again this was a clean installation. Windows 10 is a joke.

> And least when linux breaks it tends to leave error messages and details in syslog...

Tends to. Not directly a Linux issue, but i was trying to figure out why one specific program was giving me corrupted MAC errors when connecting via sftp.

All searches indicated it was a network config issue, but no changes seemed to matter.

Eventually i checked the libs the program was compiled with and found the sftp one was "jurassic". One update later and no more errors.

that said, more often than not, a quick dmesg or tail /var/log/messages is all that i need to get something purring again.

Yeah, diagnostic messages can be pretty useless. Sometimes all you can do is

    tail -f /var/log/system.log

Ugh. At work, the computers use AirWatch. Oh. My. God. They fill the OSX logs with debug messages. Literally "I got 8 bytes from this with this offset" type messages for an application that spends most of its time running in the background. Here is an example:

  2016-03-03 10:59:16.863 PM AirWatch Agent[507]: void AgentReadCallback(NSData *__strong, NSFileHandle *__strong) [Line 1096] Agent received message of type 140
  2016-03-03 10:59:16.863 PM AirWatch Agent[507]: -[AWAgentController isCurrentUserManaged] [Line 221] Current User is managed
  2016-03-03 10:59:16.863 PM AirWatch Agent[507]: Server Starting new loop for data
  2016-03-03 10:59:26.825 PM AirWatch Agent[507]: void AgentReadCallback(NSData *__strong, NSFileHandle *__strong) [Line 1096] Agent received message of type 140
  2016-03-03 10:59:26.831 PM AirWatch Agent[507]: -[AWAgentController isCurrentUserManaged] [Line 221] Current User is managed
  2016-03-03 10:59:26.832 PM AirWatch Agent[507]: Server Starting new loop for data

Is this normal operating procedure for OSX apps? Fill the system.log with print statements in a loop that runs every ~10 seconds?

It is for those that leave their debug logging enabled. A lot of code I've seen uses CocoaLumberjack[0] or something similar, which makes it easy to turn that off for production builds.

[0]: https://github.com/CocoaLumberjack/CocoaLumberjack

Encountered the exact same issue. But all kind of weird bugs turned up in the older version of App Store, to the point where I had to re-format the drive. And the Apple forums are absolutely no help. took me 3 hours to do something that should have been trivial..

super-fast 2Mbps internet connection

Don't complain. Even with torrents and 100Mbps fiber, we max out at 1MB/sec here. It reportedly costs 20,000CNY/month (~USD$4500) for an uninhibited 512k connection out of mainland China, if you can get it. (You generally can't.)

Not that that isn't awful, but 2Mbps is 4 times slower than 1MB/s...

Sure. Actually we rarely get even that, except on domestic transfers. 1MB/sec is best-case, torrent-only scenario ~1-5AM. Makes you value mirror operators.

Around 300 kB/sec lets you get 1GB/hour, which means you can get a perfectly watchable 1080p movie in 2 hours. Yes it's nice now that I have 3 MB/sec down, but it wasn't hell before and I lived with that for 10+ years. Many people still have this kind of connection - typically billed as 5000 down / 800 up. Streaming, I have no idea.

Rest assured, I don't use the internet to download television.

Oh, that's interesting. Sorry for stereotyping you as a torrent user. What kind of large files are you downloading?

~every nonfiction book posted.

YOU don't complain, even though it's slow, at least it isn't metered :/

I get 10GB/mo and it's $15 for every GB I go over. I'm a "giant pirate" and would usually go through that in an afternoon when I had comcrap. Oh well.

$15/GB is a lot. My cell phone plan charges only $5.

There is something wrong with your connection; where are you? I'm in Suzhou, I have no problem maxing out my connection downloading things outside of the GFW. It starts slow but speeds up about a minute in usually. With VPN turned on I am capped at about 10Mbps. Direct lines are easy to get, we had one for a while but decided getting everyone needed a connection, a VPN account was enough. We were paying US$2k for a 10Mbps "dedicated line" to HK.

You're next to Shanghai. I'm in Yunnan. Different world.

I was hit by this issue too. Everything that came up online (and worked in the past) was to reset the date in the Terminal. Of course, that didn't help in the slightest.

But if you quickly use sudo .../createinstallmedia to make a bootable usb installer from your annoyingly re-downloaded App store installer, you should be safe from this, shouldn't you? That's what I did when I installed, but I haven't tried to use my usb installer, so I suppose it could have expired too. (edit) Sigh, yes, expired also. Re-downloading...

Changing the date in the system (and disabling network time) worked well. A quick search led me to this solution, avoiding the whole re-download to please the sentient dictatorial master.

I had a handful of apps I use all the time stop working entirely about a week ago. They were purchased from the App Store but are no longer in the App Store. I am furious. I am willing to bet the expired certificate is the culprit.

I've had to carefully maintain the .app's myself across a couple Mac's as Apple stopped letting you download things that are no longer in the store. Now they just don't work at all.

The older I get the more I think maybe Richard Stallman isn't crazy.

> I've had to carefully maintain the .app's myself across a couple Mac's as Apple stopped letting you download things that are no longer in the store.

Since when? I always download .dmg's and install them outside of the Apple Store. You have to do this roundabout security measure in Preferences to run them, but they're always runnable...

http://www.idownloadblog.com/2015/10/07/app-store-apps-disap... at least October, but I've had issues downloading them for years.

You have to do this roundabout security measure in Preferences to run them, but they're always runnable...

Note: you don't have to. You can Control-Click and then choose 'Open' to bypass the signature or unsigned app check. You can also remove the quarantaine attribute with:

    xattr -d com.apple.quarantine Your.app

Nice! Thanks!

He's not a particularly effective advocate, but he is an excellent prophet.

That's been a mark of prophets ever since Cassandra was cursed by Apollo.

Let's not forget it's not exactly Apple's fault that some developers remove their apps from the App Store. While it's annoying, and we can wish all we want that Apple would make apps permanently available once downloaded, it's really the developer of the app that has failed you, not Apple.

No, I'm pretty sure it's Apple's DRM that got us into this mess.

As far as I understand (but I have only distributed an OS X app outside the App store), the DRM is opt-in.

App store guidelines require that packages are sandboxed and signed. If you have a signed application, you can circumvent the signature check by disabling gatekeeper, removing the quarantine attribute, or control-clicking and choosing 'Open'.

The DRM mechanism is called 'receipt validation' and has to be enabled by the app developer:

https://developer.apple.com/library/mac/releasenotes/General...

I can sort of see why Apple provides this (to entice companies to publish in the app store), but a developed can decide to be customer-friendly and not check the receipt. So, I think it's fair to blame the developer, not Apple.

(Please correct me if I am wrong, as said, I never distributed an App Store app.)

Apple introduced a change that make a developer removing their software from the market be more damaging than it used to.

Say that I buy a piece of software and receive either an installer download or some medium containing the software. The company removes it from the market. I can still perform new installs of their software.

I buy something from an app store. The developer removes it from said store, making it inaccessible for future installs. This developer did the same thing that the other one did, but I'm worse off. How is it more the fault of the developer than Apple/Google/Amazon/Microsoft/Valve/EA/etc? Who changed the way that software distribution works?

Doesn't it cost devs $99/year to be in the app store? That 's an incentive to remove quit the store if you aren't making enough money.

Under windows, when I sign a binary, I also have the option (well - it's more like a strong recommendation) to time-stamp the signature.

That way, windows can check whether the signature was made during a time when the certificate was valid. This means that when your certificate expires, you won't be able to sign new binaries, but at least the existing ones continue to work.

This is something Apple should really consider implementing - even just for the sake of archival of old OS versions that people might still want to install for nostalgias sake.

Does Microsoft co-sign the binary or something? If not that feature seems like it would defeat the purpose of expiring certificates.

It means that the certificate was valid at the time of signing. Trying to sign an executable after the certificate has expired will not produce a valid code-signed executable. It totally makes sense. That it is possible to make the certificate valid by simply setting the system clock (this works both on Windows and according to the article, OSX) shows that the whole certificate expiration thing is basically useless.

I'm not sure I understand. It doesn't matter if the consumer of the certificate is able to set their clock to make the certificate/signature valid because most people don't do that, but if the producer of the signature can simply set their clock to produce a signature/timestamp combination that's considered valid by a consumer regardless of their clock, then what's the point of expiring the certificate?

But yeah, it looks like Microsoft countersigns, assuming this is what that original comment was referring to: https://msdn.microsoft.com/en-us/library/windows/desktop/bb9...

It gets timestamped by the certificate authority, using their authorized servers. For example, I have a code-signing certificate issued by Comodo, and when I sign my software, it get timestamped by: http://timestamp.comodoca.com

I can't simply set that clock.

Yes I should have been more clear. A user which wants to run a signed executable with expired certificate can trick the validation process on his machine by setting the system clock back into the valid period.

But for the code-signer this trick doesn't work, the sign-tool needs to talk to a time-stamp server, usually owned by a certificate authority. I know that the Microsoft signtool.exe and the Java jarsigner can use timestamps, apparently on OSX this works too (search for --timestamp cmdline arg: https://developer.apple.com/library/mac/documentation/Darwin...)

How does this help? In order to provide any security you would need to limit the time from signing, which would be very similar to just issuing the certificate for that much longer. There is no way to verify that the binary was actually signed at that time.

It uses trusted timestamping servers run by certificate authorities to sign the timestamp information.

Ok, I didn't realize that. But it still doesn't solve the problem of compromising an old certificate and creating a fake signature with your own time.

Expired and compromised are two different things. If compromised, it will be published in a CRL with a reason flag.

The reason why certificates expire is because they will become easy to crack as computers get faster. So this would effectively be removing the expiry date. Now you can crack any old certificate and sign things claiming that you did it before the certificate expired.

As someone above has already said.

To do this, you'd need to compromise or convince a trusted timestamping authority to sign your signing request with an old date.

A timestamp authority is trusted in the same way that a certificate signing authority is, so you'd have to compromise a timestamp authority as well as the private keys paired with the certificate. A self-timestamped file would have the same level of trust as a file signed with a self-signed certificate.

Apple's code signing actually has this and it's on by default. Don't ask me why it didn't help with their installers, though..

Not familiar with actual Macs, but I did install OS X plenty of times on PCs. Can't you just change time on your machine, install it and then change it back? I'm sure OS X can be installed offline.

The article mentions this as a solution, yes.

Apple has been getting sloppier and sloppier over the past 5 years or so. For instance, even if you have selected "automatically set my timezone based on my location", it won't update unless you actually open the date/time preferences panel every time you change zones.

Really? I flew west coast to east coast yesterday and the time on my machine updated with no manual intervention.

Worked fine for me going from NL > UK yesterday. Yosemite.

Works for me probably 8-10 times per month going between time zones. Now Flux... that never realises...

worked for me flying from uk to mainland europe, and back.

How is this relevant?

It is sloppy to have your installs expire (presumably by accident) with a not very helpful error message?

Funny, I've had crazy clock skew problems pretty consistently with Windows computers over the last several years, and always wondered "why can't this be a solved problem like on my mac"

This always used to be a problem for me but it's been working recently (Since Yosemite? El Cap? Not sure)

Lots of people saying "works for me" but I can corroborate your experience; just flew Tokyo to Florida and didn't notice until the second day that my calendar app thought it was the wrong time and my menu bar clock was wrong. Opening the Date & Time fixed it.

So like a lot of bugs, this is probably one that happens "to some of the people, some of the time".

(Latest OS X 10.11 on latest-model MacBook Pro.)

> There is one caveat to all this. Apple won’t allow a newer Mac to download versions of OS X that aren’t compatible with that Mac, so on a 27-inch iMac with Retina display, for instance, the App Store app refuses to let you download Mac OS X 10.7 Lion.

It's an edge case, but it's still annoying that I can't do this. What if I'm trying to get a non-functional older Mac running again? Stopping all users from doing this does mitigate incompatibility support issues, but it hobbles power users.

To allow this, I don't think they need to add a switch to preferences. I'd be happy with a defaults invocation.

There's a reason: Lion didn't have drivers for the hardware in question. OS X has all drivers, like for graphics etc. built in. That's why you're limited to versions after you Mac was made.

Sorry, I didn't explain myself well enough. What if I want to download a specific image on a different Mac to the one I want to install the OS on?

Oh sorry. Hmm that's troublesome. I wonder how far back this goes. Apparently just setting your clock back bypasses the check!

All of this is just more evidence driving me away from using any kind of app store. I download and install apks on my android phone, and executable/tar installers on my computers. It's also the reason why I'll never buy an iOS device, or even open the Windows 10 store.

The best app stores are the ones beginning with apt-get, yum, slapt-get etc. I'd very much like to see a real Linux/Unix phone with a battery that isn't a joke being played on consumers. I'm tired of looking after an smartphone that requires as much care and time as a small child.

> The best app stores are the ones beginning with apt-get, yum, slapt-get etc

At least apt is as pissy about expired GPG keys as MacOS is about expired code signing certificates.

The whole "old installer" and "bootable disk" has been a pain-point since 10.8 for me. I have done 4 or 5 clean installs for friends in the last 6 months, and I tried to make a 10.10 or 10.11 bootable USB... no go - each time, a new issue.

I have ended up every time using my older 10.8 (ML) install USB every time, then just firing off the free App Store upgrade to 10.11.

In newer Mac's (2012 onward IIRC) you can just boot into a net installer, and it goes and fetches the installer from the internet before re-installing.

Even older Macs (2010 models):

http://www.macrumors.com/2012/02/07/apple-brings-lion-intern...

What happens if no internet connection is available?

The main reason I don't do this is because I'm often installing the OS to a brand new (blank) drive that has absolutely no recovery on it.

It is part of the ROM. It can be done with a blank drive.

http://www.macrumors.com/2011/07/20/internet-recovery-lets-n...

So the drive doesn't even have recovery on it.

It kinda seems like Apple has set out to prove that having a release schedule that is date and feature driven at the same time is not a good idea.

This is not related to their release schedule, it has to do with the certificates used to validate software. You can view an explanation of application level code signing here [0] and the concepts also apply to signing OS images.

[0] https://developer.apple.com/library/mac/documentation/Securi...

I believe they're saying that it is related to Apple's release schedule because having a hard deadline every year and a fixed feature set for that hard deadline means that you run into problems with quality.

I would dearly love for Apple to take a year or two (or, hell, five, I'm happy with the current feature set) and just fix bugs. No new features, just bugs. Do another Snow Leopard release, or three.

I have a cron script that hard kills finder hourly because it eats a gig of ram and gets slower and slower until there is typing lag in it

My partner's itunes refuses to reliably see her iphone.

TimeMachine shits the bed on the regular.

SMB appears to be the future but TimeMachine still requires brittle afp.

On a 15-in late 2013 laptop with 10.10.current, about once out of every 20 times I open the lid the laptop doesn't wake up. Even when it does wake up there is a black screen for a noticeable amount of time. This never used to happen on 10.9.

Apps regularly lose the ability to create audio (spotify, chrome, vlc) if you let them run too long. The solution is to sudo killall audiod.

etc etc etc

http://bugreport.apple.com/

I don't know Tim Cook, but I don't think anything will get better under his watch. Certainly, people grow, but his background has been as a "make the trains run on time" kind of guy. So that's probably what he most cares about.

Here are some of his previous jobs[1]:

- 12 years in IBM's personal computer business, ultimately serving as the director of North American fulfillment

- vice president for corporate materials at Compaq

- closed factories and warehouses, replacing them with contract manufacturers

- advance investment in flash memory from 2005 onwards, guaranteeing stable supply

A logical extension of stuff like "fulfillment", "corporate materials", and "stable supply" would be: "release new software on a regular basis". No?

Still, what is the commercial alternative? Win 10? Not for me!

[1] https://en.wikipedia.org/wiki/Tim_Cook

The funny thing is you can kindof tell Apple sortof heard the complaints out there, and that some of the buzz around El Capitan was that this is what it was supposed to be.

From what El Capitan is... it seems reasonable to conclude that either they simply don't have the management/talent to pull this off anymore, or they only care enough to pretend to do it rather than actually focus on it.

Anecdotal of course, but El Cap really did solve all of the instability and weirdness I had with Yosemite. I went from rebooting once a week due to random slowdowns (typing lag), graphical glitches or freezes back to rebooting only when there was an OS update.

AirDrop still doesn't work though...

The article noted that one workaround for people in the middle of an install just to change the date. Is this a general workaround: change the date pre-February 2016, run installer, change date back?

This is not a problem we ever had with installers on CDs...

Seriously Apple, stop making it so goddamned difficult to use your software!!

I should do this when I get a chance. Good to know. Glad I have a external Hd to store it on

Apple is really killing me with this shit. They're not making anything more secure, they're just making bitrot worse and being a pain in the ass. People need to install and run programs, please.

Completely by-the-by, but I couldn't help but notice the small advertisement at the bottom for Smile PDFPen[0]. I'm using uBlock Origin, but this wasn't blocked by it, presumably because it's unique to this site, and is indistinguishable from the rest of the content. However, it's very unobtrusive, doesn't track me at all and has no influence on the actual content of the article, so I have no problems with it being here.

This makes me wonder if there could be a set of guidelines that could be formulated for acceptable advertising on websites...?

[0]https://imgur.com/pQIFQFb

You're still being tracked. The URL itself is encoded, so there's tracking there, and when you visit the link I have 20+ tracking tags load from Google, Yahoo, MS, Rubicon, etc.

Unless you blocked all of that, expect to see retargeting ads for PDFpen in the near future.

Adblock Plus has such guidelines for the "acceptable ads" it lets through by default.

Adam Engst, publisher of TidBITS here... Thanks for noticing that Smile is sponsoring TidBITS. As you note, the sponsorship is unique to our site, and uses entirely internal systems for display, so there are no trackers or the like. Smile chose to do their URL via a URL shortener, so I don't know what could be determined if you click it, but it's not loading any code merely by viewing it. The trackers the other commenter saw are due to the graphical ads and affiliate link generators we also have on the page.

Two random facts to note. First, as far as I know, TidBITS did the first sponsorship program for Internet content back in 1992. If that was in any way responsible for what Internet advertising has become, I apologize. :-) Since we believe in archival content, you can read that article at http://tidbits.com/article/2995

Second, our primary source of revenue is voluntary memberships. We also make about $5K per year from graphical ads, and in last year's membership drive, I challenged the 90% of our readers who are not members to join, promising that we'd drop the graphical ads and affiliate link generators if we could raise another $5K. Alas, the challenge fell short. http://tidbits.com/article/16234

Reply on news.ycombinator.com