Reviving a 16-year-old Mac App
blog.tumult.com
blog.tumult.com
I had to manually create, sign, and notarize a Mac app the other day and it was total madness. It took multiple tabs of Apple documentation (new documentation, I might add, created this year because everyone complained last year about how the process was impenetrable) and back-and-forth with some seasoned Mac devs before I had something that would launch successfully on a fully updated, GateKeepered system.
No personal experience with either, but I would wager that you would encounter somewhat similar roadblocks if you try submitting things to Chrome App Store or Microsoft Store.
Not any more. Apple is so much of a control freak lately that non-app-store apps on Catalina are still required to go through them for "notarization" to be allowed to run on an unmodified OS — and yes that requires the $99 account. For me personally, that's the reason I'm staying on Mojave. That and 32-bit apps.
It isn't just notarization, it is also that notarization requires app hardening which has very strict rules. Shipping an app with 3rd party binaries that supports older versions of macOS is especially tricky to get right.
Also using direct distribution you have to deal more with Gatekeeper.
One particularly fun issue is that if you distribute your app as a zip and a user downloads (to their standard ~/Downloads/ folder) and runs it, then Gatekeeper will use path randomization (aka app translocation), which effectively makes the app look like it is on a read-only volume. Older versions of the sparkle update framework would not show update prompts if on a read-only volume (as what's the point?), and therefore if a user continued to run an app from their downloads folder they would never get updates!
Apple made this change without informing any developers that their users could be left behind for a while. I imagine this security feature prevented users from getting many security fixes.
The way to disable the app translocation is to have a user manually drag the app to their Applications folder, which is why so much software is distributed in DMGs now with the Applications folder symlink.
Now I remember seeing these weird long paths pointing to weird places with the word translocation in them — never really dug into the why. I've also seen apps ask to move themselves to /Applications when launched from ~/Downloads.
Anyway, with all these Gatekeeper changes, it's almost as if Apple doesn't want non-app-store apps at all.
Of course that's their idea, The Mac Store has been a moderate failure, so now they are pushing people into it slowly more and more every release.
As Apple and Microsoft (with their Windows Store) have learned, if there's a real choice between an app store and a standard install, nobody will pick the app store. They cannot promote it naturally, they have to push it through by force.
People went as far as disabling all windows updates manually.
Also as a second point, you can totally have a software updater without an app store.
I don't have any particular problems with the rental model but in this case I had bought it on the basis of it not being that.
Apple themselves ships some software, like the non-Mac App Store versions of Xcode, as xips (signed zips), but for some reason decided 3rd party developers cannot use these.
Apples recommendations are in the "Shipping your Signed Code" section of this tech note:
https://developer.apple.com/library/archive/technotes/tn2206...
Apple tries to hide the option, but you can still run non-notarization software on a Mac. The first time you run the application, open in from the right click menu. It'll give you a warning, and if you select OK it'll run. After that you can just run it by double clicking like normal, and it won't warn you.
To answer the question of why I use a Mac: mostly because the hardware is good (I'm on a early 2015 MacBook Pro), the software consistent and high-quality (I refuse to run Electron) and the tooling fairly decent (I have UNIX, and MacPorts, and I can compile the rest). And it works extremely well with all my other hardware, which is something that Linux still doesn't really do well. And by this point I have enough macOS experience that while I counter-intuitively probably have more complaints about the OS than most other people, I also generally know how to fix it. While it's often compared to iOS in how annoying and locked-down it is, it's really nowhere close. Out of the box, it might be approaching that point, but macOS holds the trump card of being able to pretty much turn off everything you don't want. Want to debug a system process? Turn off SIP. Want to let your shell script manage your mail messages? Insert it into the TCC database. Apple thinks some operation is something only their code should do? Well they can pound sand, I can turn off AMFI. macOS lets you do this, and as annoying as it is to keep doing all of this it hasn't yet gotten to the point where it's enough of a drag to make me stop using it.
The answer to a slightly more on-topic question: why would I develop for macOS? The answer to that is really "I mostly don't". I have much more iOS development experience (although my most recent job in that area was actually "spotting exactly what part of your app is pretending that it's using native controls", not actual programming). I know enough of AppKit to be useful, but actually write very few apps. Most of the code is just regular old stuff that would work equally well on Linux and I just happened to compile for macOS. The rare exceptions are if I am writing something for myself (in which case I care little about any codesigning stuff) or I am contributing on an open source project (in which case Xcode takes care of this for me). I only hit these notarization/Gatekeeper hurdles recently because this particular thing needed to run on other people's computers and and had to be manually cobbled together for complicated reasons which I cannot describe at the moment.
The issues noted are specifically about the process of submitting apps to the macOS app store. A process that for iOS and macOs has been notoriously confusing and problematic with regards to the signing process.
You think you have it sorted... then boom! it wont pass signing validation and you cant submit to the app store.
It's much better than it used to be though. And it's one of those things that you do once, get it working, then forget how you did it and come back to it a year later and it's confusing all over again :-)
The dev tools themselves (XCODE) are actually pretty good.
If your a Mac user, you can copy and paste one line in the Terminal and you can run whatever simple unsigned apps you want.
But things have gotten worse for developers recently with Catalina. I should note that I don't write mac-specific code - I mainly write things that target Linux servers.
The Mac has long been the best unix workstation on the market. It is solid, generally stable (although that's been slipping), and certainly has by far the best window manager. It has nice consumer apps for when you need them and is a solid, if not always up to date, unix. The hardware is generally great.
Then the sandboxing, weird parallel fire permission systems/quarantining, new filesystem conventions/restrictions and so on made it incredibly difficult to treat, well, like a unix workstation.
We'll probably keep using Macs at work out of inertia, and it is a lot easier for me to use other resources there. And my personal Mac laptop is probably good for quite a while yet - I tend to get 7+ years out of them. But this is the end of the road for me and the Mac. I require that my tools do what I tell them, not the other way around.
I was on that same line of thinking until some time ago when my old macbook pro died, and I ended up again on Linux, on a thinkpad. I'm currently running Manjaro (plasma + i3), and tbh, the Mac's window manager is awful once you get used to the power of i3. Dev experience (for anything but Mac/iOS apps) is probably better on Linux, too...
If you haven't used Linux in a while (as was my case), I can highly recommend it.
Maybe on Mac. On Windows you just run signtool.exe and you're done.
> I require that my tools do what I tell them, not the other way around.
Sounds like you'll be glad once you can ditch that old Mac and get a proper Linux dev-machine ;)
Unfortunately I've been disappointed with Apple's stewardship of the Mac under Tim Cook. The move toward soldered RAM and SSDs throughout most of their lineup, the increased locking down of macOS (especially with Catalina's enforced notarization), the stagnation of the Mac Pro and then the doubling of price of the entry-level Mac Pro in 2019 from $2,999 to $5,999, and the butterfly keyboard fiasco (although finally completely over as of this week) have pushed me toward a greater appreciation of the Linux/BSD ecosystem, despite its shortcomings. I'm still using my Macs (albeit they're still running Mojave), but my long-term plan is to switch to Linux or FreeBSD, though I might keep a Mac around for Keynote use unless a compelling competitor for Linux emerges. The thought of replacing my MacBook Air with an X-series ThinkPad sounds really enticing, as well as one day replacing my Mac Pro with a Threadripper build.
(Sarcasm, of course... Although I don’t quite see what makes it so difficult a problem. Unless there are patents involved, maybe?)
I’ve already determined that when I pick up my next contract, I’ll be buying a Lemur Pro from System76.
Apple, you had me 17 years ago. Now you’re losing me.
This way I can run a win10 be for games too
Yes, the killing of 32-bit apps in Catalina and the code signing issues are very frustrating. It's still worth it though. People are finding ways around it. The biggest annoyance (for me) has been the breakage of 32-bit Wine. I bought a license for Crossover [2] though, so it should hold me over until the upstream Wine has 32-bit-on-64 support working.
[1] https://brew.sh
Since the late 1990's, I've alternated: I use a Mac for a while until I get fed up with it, and then I use Linux for a while until I get sick of that, and then I go back. (I've used Windows at work, and it's not any better.) I just need enough time to forget how much I hated the other one.
This is exactly how it is for me! Except I switch between all three. It's like StarCraft, each has its strengths and weaknesses.
Same way with iOS and Android. iOS is much nicer to use until I run into the inevitable, "Oh, i'm not allowed to do that", so I just carry two phones around...
While things have certainly taken a tumble in the last few years with the faulty keyboards and slipping software quality, I am still yet to find a better laptop than a MacBook Pro, when considered as a complete package.
It’s very nice for command-line development too as it’s a decent Unix under the hood.
Developing Mac apps can even be pleasant! Xcode has its good points and the APIs are mostly very nice.
The hassle of distributing Mac apps is the only big downside. It’s the same on iOS, but there you don’t have a command line and scripting available.
And before you say "security": it ain't that if someone else (Apple) has the private key.
I won't say "security" :) but I'm sure a lot of app developers do not want to be held responsible for jeopardizing users' machines or data.
While Whisk doesn't have to deal with SIP, it still has to get around sandboxing on the Mac App Store version to be able to read files that are associated with the .html page (like in an <img> tag). We pretty much need to throw up a scary dialog asking users to allow access via an Open Panel. On any "normal" (non mac app store) app this is implicitly granted, and I'm sure users don't give it a second though. I personally was worried about what the reaction to asking for permission would be and so wrote up a whole explanation on the sandboxing technology and reasons for our request [3]. Originally I did prompt the user to choose the root level of their file system, but App Store review would only approve of "project folders."
[1] https://www.madrau.com/support/support/srx_1011.html
[2] https://arstechnica.com/information-technology/2019/09/no-it...
[3] https://tumult.com/whisk/documentation/v2/app-security.html
You know how I log into a remote server securely without ever entering my password? I enter the command "ssh" and the server name. Why the hell isn’t notarization implemented, effectively, as "notarize appname.app" where all the details are resolved securely, just as magically, to your developer keys?
Yet if you upload something to the App Store, you understand Apple’s backwards mindset completely. The whole concept of a 6-step wizard that you have to CLICK through, EVERY time, for EVERY update, confirming stupid little things that should be in a config file, means that Apple has no idea how this stuff really ought to work. And that means that Apple must not use this themselves. They have to have some secret, simple, tolerable process internally which is why they never improve the tools that literally everyone else must put up with.
Further, the first version would write status to standard error. Clearly a mistake, Apple did fix this in a future version... which of course broke everyone's automation around it. My polling script now logs the tool's stdout and stderr just in case.
Try automating signing a windows driver. It's all GUI.
At least the mac code signing tools are command-line.
I can't report much good about the Apple signing side. In the end we put that into a VM because different xcode versions will produce vastly different signatures for no obvious reason, and the older ones were more widely compatible.
Shameless plug of my own: Last month I shared a post sharing a few thoughts on secure software redistribution, notarization, safe updates, and so on: https://henvic.dev/posts/cs-security/
This is apparently like a DEFCON 0 type operation to perform on a Mac requiring two guys in white lab coats to turn keys at the same time at opposite ends of the room, because there was no way to even ask for the permission to run the command if I remember right. It ended up being a case where I would be spending a ton of time to get more functionality than the command line script, so I quit. Plus it was one of those one plane ride projects.
Sure, they're horrible, but I look at it as Apple generously creating jobs for engineers. I had a full-time job as a build engineer specializing in navigating the Lovecraftian nightmare of iOS build automation. Sure, they could've made it easy, but instead they chose to help transfer large sums of money from my family-friendly multinational megacorp employer to me. I returned the favor by using some of that money to buy their products.
I keep seeing this kind of other-worldly self-defense only related to Apple. Positioning a brand as an element of the users' personal identity is truly an impressive thing...
> Positioning a brand as an element of the users' personal identity is truly an impressive thing
Apple definitely did an impressive job at that!
It stopped being fun somewhere along the way. This was partly because I never knew what new requirement Apple was going to add with each new annual release of macOS. Would my app work? Would it not work? How many days, weeks, or more of work would I need to do to keep my app working while not actually making it better for my users?
This mirrors my feeling. It might be easier to learn to program say TypeScript or Go today than it was to learn php or C in the late 90s, but actually creating a complete program, distributing it, having it run or accessed by users is much harder than when I learned to program back then.
This isn't what I'm talking about. With just one user, even for desktop software the complexities of distributing the software are negligible.
I've got some old PHP code that dates back to the 90s and is still running although I had to make some changes when my web host EOL'd PHP5 (I'd note that some of the files have a .php4 suffix).
One part of me doesn't like how much space is wasted by the new Mac UI trends shown in the second screenshot, mostly the left side panel. But I understand screens are different these days as well.
I assume you are referring to the inspector panel on your other left; the two main reasons it is wider are (1) that it shows HTML5 validation error text descriptions in the 3rd tab and (2) this is the same width we use in our other application Hype, so they now look like they belong together. I did have a moment of feeling that I was contributing to the "war on information density" with this change.
I'm glad that the brighter colors are more subdued now, annoying UI elements like drawers and palette windows have mostly been replaced by inspector panes, and brushed metal is long gone.
It would be good to be able to side load hobby apps, but that’s never happening.
The fee does also get you 2 tech support requests, which should connect you with their engineers if you need help (never needed this, can’t say how extensive it is).
Also, I did read somewhere that they waive the fee for some nonprofit/education institutions.
I mean, all this is about mass distribution to unsophisticated consumers, not hobbyists - without that $99 you can work on your hobby projects and you can share them with other hobbyists; the only place where that $99 is a barrier is the distribution channel to non-hobbyist users.
In essence, the message from Apple to hobbyist developers is to 'go big or go home' - if you're seriously developing something useful, then $99 is insignificant; and if the time effort you're putting in that app is not much, much more than $99 then it's reasonable to draw a line that they don't want such hobby projects on the app store. You can experiment with it and share it in small communities and when (if!) you think this project has become serious enough to go beyond hobbyists to the mass market, only then you need to pull out that symbolic $99 to demonstrate that you mean it.
You can still do that.
Why do you want to publish on the Mac App Store? If you think there's an advantage, then the costs are justified.
The only solution was to sign/notarize it, and I have to admit it took me more than a few days to figure it out, as finding exactly what works for distributing a plugin was not documented. And this is just to distribute a usable plugin on GitHub !
Whisk appeals to me because I _know_ it's going to be a polished, lightweight app, but with VS Code having quite usable previews I don't see the value of its current price point (to me).
(Personally, I find the extensions I’ve tried clunky, so I’m curious if there’s a better way.)
I haven't coded for the mac in more than a decade but what I remember from ObjC/Cocoa was that it was quite pleasant to use and very powerful. By powerful I mean that the ratio between LoC and smiles on my face was skewed towards smiling.
macOS is probably even more stable.
Swift, on the other hand... it's been a moving target for sure, and I'm glad I didn't just rewrite everything (like I did for one app).
This was more of a passion/craft project than one I expect will make a gigantic impact. A side motive was to help move out some code of our Hype app to an "app shell" where I can more quickly make other applications with the same trial/licensing/windowing/etc. components.
The main use-case where this tool has been useful, though, is when I'm working on my Mac app that uses WKWebView and Javascript pretty extensively. It's much faster to get little JS snippets right compared to the really slow build+run cycle in Xcode.