MacOS Catalina: Slow by Design?
sigpipe.macromates.com
sigpipe.macromates.com
- Using a new, random executable. Even echo $rand_int will work. Edit: What I mean here is generate your rand int beforehand and statically include it in your script.
- Using a fresh filename too. Just throw a rand int at the end there. e.g. /tmp/test4329.sh
I MITMd myself while recording the network traffic and, sure enough, there is a request to ocsp.apple.com with a hash in the URL path and a bunch of binary data in the response body. Unsure what it is yet but the URL suggests it is generating a cert for the binary and checking it. See: https://en.wikipedia.org/wiki/Online_Certificate_Status_Prot...
Here's the URL I saw:
http://ocsp.apple.com/ocsp-devid01/ME4wTKADAgEAMEUwQzBBMAkGB...
Edit2: Anyone know what this hash format is? It's not quite base64, nor is it multiple base64 strings separated with '+'s but it seems similar...
Edit3: Here is the exact filename and file I used: https://gist.github.com/UsmannK/abb4b239c98ee45bdfcc5b284bf0...
Edit4 (final one probably...): On subsequent attempts I'm only seeing a request to https://api.apple-cloudkit.com and not the OCSP one anymore. Curiously, there's no headers at all. It is just checking for connectivity.
RAND_FILE="/tmp/test-$RANDOM.sh";
time_helper() { /usr/bin/time $RAND_FILE 2>&1 | tail -1 | awk '{print $1}'; } # this just returns the real run time
echo $'#!/bin/sh\necho Hello' $RANDOM > $RAND_FILE && chmod a+x $RAND_FILE;
echo "Testing $RAND_FILE";
echo "execution time #1: $(time_helper) seconds";
echo "execution time #2: $(time_helper) seconds";
Introducing a network delay makes the effect much more obvious. Normally I see a delay of about 0.1 seconds, but after using the XCode network link conditioner (pf rules) to add 500ms latency to everything the delay shoots way up to ~2 seconds.example output:
Testing /tmp/test-24411.sh
execution time #1: 2.32 seconds
execution time #2: 0.00 seconds
with developer tools checked both executions report "0.0 seconds".After blocking that domain I can see that tccd and syspolicyd are logging some error messages to the console related to the failed connection. I don't recommend blocking because my guess is that'll put syspolicyd/tccd in some unexpected state and they'll repeatedly keep trying to make requests.
Try this for watching security related console log messages:
sudo log stream --debug --info --predicate "processImagePath contains 'tccd' OR processImagePath contains 'syspolicyd' OR processImagePath Contains[c] 'taskgated' OR processImagePath contains 'trustd' OR eventMessage Contains[c] 'malware' OR senderImagePath Contains[c] 'security' "
syspolicyd explicitly logs when it makes the network request. syspolicyd: cloudkit record fetch: https://api.apple-cloudkit.com/database/1/com.apple.gk.ticket-delivery/production/public/records/lookup, 2/2/23de35......
(you need to enable private logging to see that url)Problem is, that they did it unannounced. There must be really some weird stuff going on in those managers heads. How can they possibly think to go away with that?
From April 10, 2019: https://developer.apple.com/news/?id=04102019a
https://developer.apple.com/documentation/xcode/notarizing_m...
If you're compiling something yourself, the compiler won't put a quarantine bit on it and it will execute fine. Same with homebrew/friends.
Scripts don't need to be signed. There is something else going on here.
No, the entire thing is the problem. Windows 10 can still open applications that were compiled in 1994, and it doesn't make it less secure.
Every software place I've worked gives a special urgency to security stuff.
And even if features don't come out regularly, security updates do. This is more of that.
No, AV vendors did it for decades. In a more efficient way though.
AIUI doing what you said would permit the network request to proceed, and it would fail because nothing is listening on port 80 [1] We already know that the phone-home bails out when there's no network connection, so perhaps that code also bails out on connection failure?
Alternatively, is there some way to make DNS lookup itself fail for ocsp.apple.com?
Last resort, if we know how to fake the response, running a dummy server listening on localhost would be faster than allowing the request to go over the internet.
[1] Empirically, `curl http://0.0.0.0` yields a connection failure. I think I know that 0.0.0.0 is used in a listening context to mean "listen on all interfaces" but tbh I don't really know what it means in a sending context. Maybe someone can educate me?
But from Edit2: Your hash is some sort of base64
let str =
"ME4wTKADAgEAMEUwQzBBMAkGBSsOAwIaBQAEFDOB0e_baLCFIU0u76+MSmlkPCpsBBRXF+2iz9x8mKEQ4Py+hy0s8uMXVAIIfYbtkeEKZsI="Then we see weird random gaps in the alphabet used, not so weird, because not every character will be used in every string:
Prelude Data.List> map head $ group $ sort $ str
"+0246789=ABCDEFGIKLMOPQRSTUVXYZ_abefghiklmpstuwxyz"
If we fill these up then: Prelude Data.List> let xs = "+0123456789=ABCDEFGHIJKLMNOPQRSTUVWXYZ_abcdefghijklmnopqrstuvwxyz"
Prelude Data.List> length xs
65
So base64 with some non standard symbols. I don't know what standard base64 is supposed to look to be honest, so perhaps it is standard base64. The = is definitely padding.https://support.apple.com/en-ca/guide/console/cnslbf30b61a/m...
If you did get it to work could you paste the logs somewhere?
The way to avoid this behavior is to staple the notarization ticket to your bundle (or dmg/pkg), i.e. "/usr/bin/stapler staple <path>." Otherwise, Gatekeeper will fetch the ticket and staple it for the user on the first run.
(I'm the author of xcnotary [1], a tool to make notarization way less painful, including uploading to Apple/polling for completion/stapling/troubleshooting various code signing issues.)
The "Developer Tool" pane in System Prefs, Security, Privacy is the same power. Drag anything into that list you'd like to grant the same privilege (such as xcodebuild). This is inherited by child processes as well.
The point of this is to avoid malware packing bits of Xcode with itself and silently compiling itself on the target machine, thus bypassing system security policy.
Maybe in some cases, but the article says "even if you write a one line shell script and run it in a terminal, you will get a delay!"
Shell scripts don't come in bundles. I don't think this kind of stapling is possible for them? I don't think it'd be reasonable to expect users to do this anyway.
For example, I can run "terraform apply" and it could take up to 5 minutes to start, leaving my computer almost unusable until it runs. The weird thing is that this only happens sometimes. In some cases, I restart the laptop and it starts working a little bit faster, but the issue comes back after some time.
It's already been a few months since I try to run every command from a VM in a remote location, since I am tired of waiting for my commands to start.
I have a macbook air from 2013 which never had this issue.
Any easy fix that I could test? Disconnecting from the internet is not an option. Disabling SIP could be tried, but I think I already did and didn't seem to fix it, plus it is not a good idea for a company laptop.
Don't we have some sort of hosts file or firewall that we can use to block or fake the connectivity to apple servers?
Certain BSD-syscall-ABI operations like fopen(2) and readdir(2) are now not-so-fast by default, because the OS has to do a synchronous check of the individual process binary's capabilities before letting the syscall through. But POSIX utilities were written to assume that these operations were fast-ish, and therefore they do tons of them, rather than doing any sort of batching.
That means that any CLI process that "walks" the filesystem is going to generate huge amounts of security-subsystem request traffic; which seemingly bottlenecks the security subsystem (OS-wide!); and so slows down the caller process and any other concurrent processes/threads that need capabilities-grants of their own.
To find a fix, it's important to understand the problem in fine detail. So: the CLI process has a set of process-local capabilities (kernel tokens/handles); and whenever it tries to do something, it first tries to use these. If it turns out none of those existing capabilities let it perform the operation, then it has to request the kernel look at it, build a firewall-like "capabilities-rules program" from the collected information, and run it, to determine whether it should grant the process that capability. (This means that anything that already has capabilities granted from its code-signed capabilities manifest doesn't need to sit around waiting for this capabilities-ruleset program to be built and run. Unless the app's capabilities manifest didn't grant the specific capability it's trying to use.)
Unlike macOS app-bundles, regular (i.e. freshly-compiled) BSD-userland executable binaries don't have a capabilities manifest of their own, so they don't start with any process-local capabilities. (You can embed one into them, but the process has to be "capabilities-aware" to actually make use of it, so e.g. GNU coreutils from Homebrew isn't gonna be helped by this. Oh, and it won't kick in if the program isn't also code-signed, IIRC.)
But all processes inherit their capabilities from their runtime ancestors, so there's a simple fix, for the case of running CLI software interactively: grant your terminal emulator the capabilities you need through Preferences. In this case, the "Full Disk Access" capability. Then, since all your all CLI processes have your terminal emulator as a runtime ancestor-process, all your CLI processes will inherit that capability, and thus not need to spend time requesting it from the security subsystem.
Note that this doesn't apply to BSD-userland executable binaries which run as LaunchDaemons, since those aren't being spawned by your terminal emulator. Those either need to learn to use capabilities for real; or, at least, they need to get exec(2)ed by a shim binary that knows how.
-----
tl;dr: I had this problem (slowness in numerous CLI apps, most obvious as `brew upgrade` suddenly taking forever) after upgrading to 10.15 as well. Granting "Full Disk Access" to iTerm fixed it for me.
On a clean Catalina install this does not happen. Does “terraform version” have the same delay? If not, check your remote configuration - maybe run with TF_LOG=trace. Terraform Cloud will definitely highlight the inherent performance problems of using a VPN.
The comments on the article are annoying: it good that there's a mini way to reproduce, but please, use some further debugging like tcpdump (it still exists on osx, right?). Last time I summarized osx debugging was https://www.slideshare.net/brendangregg/analyzing-os-x-syste...
I'd also stress test it: generate scripts in a loop that include random numbers and execute them.
This would require breaking your code signature and as such requires extra entitlements in the hardened runtime.
The Mac remains a Mac: if you turn off SIP it also disables this behavior. You are free to choose less security for more convenience if that is your preference.
> Apple’s most recent OS where it appears that low-level system API such as exec and getxattr now do synchronous network activity before returning to the caller.
Can anyone confirm this? Because honestly this is just terrifying. I don't think even Windows authorises every process from a server. This doesn't sound good for both privacy and speed.
"Full Disk Access" to allow a program to access any place on your computer without a warning. A few programs requested this, so it looks like it's been around for a while.
The other one is "Developer Tools" and it looks pretty new. The only application requesting it is "Terminal". This "allows app to run software locally that do not meet the system's security policy". So, my reading of this is that in Terminal, you could run scripts that are unsigned and not be penalized speed-wise.
Don't think so? Apple now theoretically has a centralized database of every Mac user who's ever used youtube-dl. Or Tor. Or TrueCrypt.
Speed, definitely not, this is going to make things slowwwww
I could not for the life of me understand why go build would take upwards to 30 seconds to run and sometimes 100ms. I finally realized it was related to my internet connection being extremely spotty. I went online and searched if anybody had the same experience with `go build` but couldn't find anything.
I finally know what happened. This is a pretty intolerable "feature".
> Hello
> /tmp/test.sh 0.00s user 0.00s system 0% cpu 5.746 total
> Hello
> /tmp/test.sh 0.00s user 0.00s system 79% cpu 0.006 total
And even if I didn't connect to my VPN:
> Hello
> /tmp/test2.sh 0.00s user 0.00s system 0% cpu 1.936 total
> Hello
> /tmp/test2.sh 0.00s user 0.00s system 78% cpu 0.005 total
That's just ridiculous and unbearable.
Apple should provide a way to disable this notarization thing, and the user should still be able to enable SIP while disabling it.
additional information:
- macOS version: 10.15.4
- terminal: iTerm2 3.3.9
- didn't install any "security" software
airyote% echo $'#!/bin/sh\necho Hello' > /tmp/test.sh
airyote% chmod a+x /tmp/test.sh
airyote% time /tmp/test.sh && time /tmp/test.sh
Hello
/tmp/test.sh 0.00s user 0.00s system 74% cpu 0.009 total
Hello
/tmp/test.sh 0.00s user 0.00s system 75% cpu 0.007 total
Is it possible that Allan Odgaard, as good a programmer as he unquestionably is, has something configured suboptimally on his end? Because it just strikes me as super unlikely that Apple has modified all the Unix shells on macOS to send shell scripts off to be notarized. (From what I've read, while shell scripts can be signed, they can't be notarized, and Gatekeeper is not invoked when you run a shell script in Terminal -- although it is invoked if you launch a "quaurantined" shell script from Finder on the first run, but it treats the shell script as an "executable document." This is the way this has worked for years, as I can find references to it in books from 2014.)I have my complaints with macOS Catalina, and I know that Apple's "tighten all the screws" approach to security is anathema to a lot of developers (and if there was a big switch that I could click to disable it all, I probably would), but I'm using Macs running Catalina every day and I gotta admit, they just don't seem to be the dystopian, unlivable hellscape HN keeps telling me they are. At least off the top of my head, I can't think of anything I was doing on my Macs ten years ago that I can't do on my Macs today. ("Yes, but doing it today requires an extra step on the first run that it didn't used to" may be inconvenient, but that's not the same thing as an inability to perform a function -- and an awful lot of complaints about modern Macs seem to be "the security makes this less convenient." There's an argument to be had about whether Catalina's security model strikes the right balance, of course.)
First, disable SIP to allow yourself to modify the system. Then, disable AMFI, the component responsible for code signature checking, entitlement enforcement and all that very useful stuff, with a kernel argument:
nvram boot-args="amfi_get_out_of_my_way=0x1"
Then you should be done.if he switched the /bin/sh out to /bin/zsh or /bin/bash which ever his default shell was, he wouldn't have seen the first delay.
Also worth noting "echo" doesn't spawn a process but is a routine in the shell itself. If you replaced echo with something that does spawn a process "like scp" it would be interesting to see the results. And if that's doesn't introduce latency then I'd try it with some hello world programs with a UUIDv4 in the binary to ensure they haven't seen the hash before.
Catalina has a huge number of things that synchronously block application launch, and if any of them fail you get nothing but a hung app. A friend and I have a running discussion of the many ways where an application would just hang and we’d send samples and spindumps, to each other trying to figure out the right daemon or agent to kill to get the process to start responding again. It’s madness.
I first used the author’s timing method. First runs are consistently about 300 ms, subsequent runs consistently about 3 ms. Something is happening at first run.
Some in the comments are saying it’s “local stuff”, so I tested timing again with internet off. First runs go to about 30 ms, subsequent remain the same. So there is “local stuff”, but it doesn’t explain the delay.
Just to be entirely sure, I installed Little Snitch and got clear confirmation: running a script you just wrote results in syspolicyd connecting to api.apple-cloudkit.com. syspolicyd is the Gatekeeper daemon.
I don’t know what exactly is being sent. Maybe somebody else can do a proper packet analysis.
My only problem in moving to Linux software is that I prefer Apple's hardware. I'm on the 2019 16-inch MBP. Linux's compatibility with all the T2 and SSD hardware isn't there yet, but apparently it almost is.
If Linux on the T2 MBP becomes solid and stable in the next 1-2 years, after extensive testing I may move over permanently. I already use Linux on secondary computers, and I love and value its privacy. Same with my phone. I just love my privacy.
My needs are a high bar though. Productivity must be held back by nothing. I use macOS notes extensively and it syncs with my iPhone which is an extremely useful tool for me to note things down both in audio and. It needs to be reliable and - heh - 'just work'. I just discovered the cross-platform 'Standard Notes' app, with a bit more money paid out to Linux-compatible services like that, maybe it can all work. Casual photoshop can be taken care of via a VM.
Surprisingly, macOS Catalina is itself a disrupter to my productivity. It seems buggy as hell - glitchy, and weirdly slow for many extremely basic things - all since Catalina. I just don't get it. Is it caused by this article's observation? Something's definitely going on.
Maybe Apple will fix this in the next release? Like how they fixed the keyboard?
Either way, I still want to move to Linux on this fabulous (fixed) hardware that is the 16-inch MBP. (T2 issues aside.)
That's just the technical problems. I'm willing to give the UI a break, since it's probably as much me adjusting as it being bad.
This is my first Apple anything, and if this is what "just works" looks like, I don't want it. I could be more productive on an Android tablet at this point.
I worked on the team in charge of improving iOS (13) perf at Apple and IIRC there was no dedicated macOS “task force” like the one on iOS.
Luckily some iOS changes permeated into macOS thanks to some shared codebases.
Apple doesn't give a fuck about macOS since 2015.
It's not surprising. Macs are less than 10% of Apple's revenue.
https://www.macrumors.com/2020/04/30/apple-2q-2020-earnings/
This design seems to cement the trend at Apple to position their products as consumer appliances, not platforms useful for development.
The problem is, there's nothing else out there. Everything is going to shit in one way or another. Windows is now a disaster, Linux was always a disaster in terms of user experience and isn't improving.
Mac OS was the last bastion of somewhat good, thoughtful design, user experience and attention to detail and now they've gone to shit too.
> Is there any "security" software running on your Mac? I've seen this sort of thing caused by that, but not in general.
> I ran the two line test and it had no delay at all. The Mac doesn't check for notarization on shell scripts or any non-bundle executable. I just did it again with a new test2.sh and Wireshark capture and there is nothing.
> I do a lot of Keychain code and I've also never seen those delays. The reason I suspect they told you not to use that API is that it's in the "legacy" macOS keychain. They really want everyone to move to the modern keychain but lots of people, myself included, still need the older macOS specific features.
> I'm not saying you are crazy, but all of these things though are the trademark reek of kernel level security software that is intercepting and scanning every exec and file read on the system. We had an issue with Cisco AMP once that took Xcode builds from under 10 seconds to over 5 minutes until we were able to get it fixed.
Having to wait 5-10 seconds for a new terminal tab as Sophos churns (checking autoccomplete scripts, rbenv, etc) was infuriating. Oddly, there was fate sharing with Internet interception, so there was a good chance the browser was getting dragged down too, and vice versa.
Convincing corporate IT of how bad the problem was was maddening. Based on what this author says, 10.15 on rural internet sounds like hell.
% rm /tmp/test.sh ; echo $'#!/bin/sh\necho Hello' > /tmp/test.sh && chmod a+x /tmp/test.sh
% time bash /tmp/test.sh && time bash /tmp/test.sh
Hello
bash /tmp/test.sh 0.00s user 0.00s system 83% cpu 0.004 total
Hello
bash /tmp/test.sh 0.00s user 0.00s system 77% cpu 0.003 total
vs the one from the article:
% rm /tmp/test.sh ; echo $'#!/bin/sh\necho Hello' > /tmp/test.sh && chmod a+x /tmp/test.sh
% time /tmp/test.sh && time /tmp/test.sh
Hello
/tmp/test.sh 0.00s user 0.00s system 2% cpu 0.134 total
Hello
/tmp/test.sh 0.00s user 0.00s system 73% cpu 0.004 total
(edited for formating)
And I don't mean developers. They're all pretty educated people but it's taken me by surprise. They come to me in frustration over Mac, they don't want to return to Windows and they really, really, really want linux. I've been using linux since about 1997 so they come to me. I usually push back, thinking "do you really want a unix workstation?!" but they insist.
My strategy has been some x2xx lenovo (like x230 or so) for about $300 from ebay, 8/16gb of ram or so with an SSD, the extended battery pack, putting mint on it and then just handing it over. Everyone, much to my continued surprise, has loved it and are really happy with it.
It's happened 4 times now and I'm still shocked every time. They've told me they use youtube to figure things out.
They're fine with libreoffice, gimp does what they need, supposedly spotify works on it fine, they don't know what bash or the kernel is and it's all fine. Incredible.
I still use a Mac at home for entertainment (I'm typing this comment on one), and I have to say it works much better used that way. I don't have to worry anymore about random Mac OS upgrades breaking functionality that Apple doesn't care about because it's not part of their vanilla out-of-the-Apple-Store experience, but is vital to me as a developer such as 3rd party window management, dock improvements, keyboard tweaks, or not delaying every new execution by phoning home (LMAO).
This year marks the first year that I can just use linux without having to debug it.
Linux as an actually better experience, without gigantic embarrassing flubs like this, is looking better by the day.
It's also true that the trackpad isn't as good as Windows. (It used to be that Mac had the best, but Catalina managed somehow to screw up the trackpad and make it laggy. Catalina has not been good for me!)
From my perspective as a quote-unquote power user, it feels like Apple just constantly insists on shooting themselves in the foot with unnecessary and ill conceived innovations. Either way, I'm happy with my new setup and probably won't go back to macbooks anytime soon.
Coldboot Windows 10 from pushing the power button to reaching the login screen is 7s for me (i7-7700, m2 SSD, 32GB RAM).
I never ever had quicker startups on OSX.
It was quite slow compared to OS 9, but even most Linux installs have way better performance on equivalent hardware. Windows really is dog slow by comparison.
[0]: https://eclecticlight.co/2020/01/27/what-could-possibly-go-w...
[1]: https://eclecticlightdotcom.files.wordpress.com/2020/01/appf...
Fedora 32 Workstation is pretty good if you want to see the best of what Linux can offer. It may not be the lightest and fastest distribution but it is easy to install and everything works. You'll get to experience Gnome which is the most original Linux desktop environment and the best one in terms of user experience in my opinion.
If you want something more traditional with the start menu or dock or desktop icons, perhaps something like KDE Neon is better place to start. It might feel more familiar. Will be lighter/faster too.
Put each of them on a USB and run them live on your machine for few minutes each and see which one makes more sense to you.
Note: I really wanted to like WSL, but it just didn't work for me.
The feeling reminds me of the first Macbooks I used when switching away from Windows Vista.
By far the best linux I've tried when trying to get feature parity with macOS.
It is lightweight, since you choose everything that is installed, sort of opt-in.
It has all the latest software.
It has "rolling releases" which means there is never a giant lost-weekend distribution upgrade.
It has the AUR (arch user repository) for just about any software ever.
Debian or similar or ArchLinux if you have a desktop.
if you want: nice experience out of the box
I would recommend: Arch, Gentoo, Debian Net inst, Void
if you want a base system and install things you want on top of it
It's ubuntu without the bullshit monitization.
I've been running a Debian thinkpad for the last meaningful stretch of time, but from what I recall macOS quarantines any files created by the user via an extended attribute `com.apple.quarantine`. Quarantined files are not allowed to be executed by gatekeeper. It's not about a network check, they just can't be executed. If the user removes the quarantine attribute, then gatekeeper will shut up and the files will execute normally. Alternatively, if a file has a signed hash stapled to it i.e. if it has been notarized, then gatekeeper will also allow execution after verifying the signature. This doesn't require a network check either.
Interestingly, the way to bypass the quarantine behavior is to unarchive a folder. Archives themselves include the quarantine attribute, however, files extracted from the archive using a terminal program (a "developer tools" program) don't. And so macOS doesn't care. Also tools like `curl` don't apply the quarantine bit to downloaded files so curling a binary or shell script still works just fine.
How can something as damning as this ever reach end consumers without getting detected?
It's really frustrating to see Apple make all these poor decisions and they almost never are willing to admit their mistakes and go back. In the rare case when they do (e.g. butterfly keyboard, Mac Pro), it takes them years to turn around.
Teaching my daughter to program on a modern computer, we spend more time bootstrapping and in process, than we do in actual development.
Also, they are a threat to a free market for software, as they regulate their walled garden with arbitrary rules and skim off a lot of value.
I honestly don't understand why a large portion of developers have so much love for Apple. I'm personally a proud owner of a desktop PC with an ASUS motherboard. It serves me fine, and gives me full control over the software installed on it. I'm not a laptop-person but I believe there are many perfectly capable non-Apple laptops out there.
care to elaborate a bit? what did you understand?
i just can't get my head around this idea that most non-mobile OSes have become such hostile environments...
yes, the population at large only uses their phones and tablets and doesn't care much. but they would be left without any entertainment if it wasn't for those of us who still need decent non-mobile environments.
Apple's ecosystem is also an issue. iOS + macOS is still much better than anything on the market (no alternatives really).
I would say go for it, I'm glad to not be dealing with any of this nonsense, while paying a premium for it.
I use linux to watch movies, create music, play games and everything else. What exactly makes it a "horrible system outside of work" for you?
The Windows + Linux combo is way better for all productivity, gaming and development than the mess macOS has become since Jobs passed away.
Changing the filename to test2.sh on the Mac (which should trigger the delay, right?) gets 0.006s, 0.006s.
I don't think the shell scripts are doing what they claim (and wouldn't the second run be faster anyway because of caching?)
First off the new apps (music, podcasts, etc) are terrible. They killed off iTunes but replaced it with much worse. These apps don't behave like standard macOS apps, the UI is full of inconsistencies and is just so empty. This website has nice examples of the failures of modern Mac OS: https://annoying.technology
For some reason after updating the "new updates" badge was stuck on the system preferences icon (and even on the preference pane itself) despite no updates being available. I ended up having to delete a plist and reboot to fix it, apparently a common issue.
The Mail app will now randomly play the "new mail" sound. I can't confirm it for sure but I'm assuming it's treating read, existing mails when they are moved to the trash/archive or newly created drafts. They screwed up the mail app, a problem that has been solved for decades. WTF? The worst is that I see no major changes in there, so why touch the mail client in the first place if you're not even going to give me additional features in exchange?
Xcode was stuck upgrading in the App Store. It would start the process and never make any progress. Cancelling it had no effect. Rebooting cancelled it but the second attempt, while making progress, ended up failing with a generic error message with no actual information. Logs are useless because they're being spammed by all the background processes even during normal operation making it impossible to find anything. Finally the third attempt succeeded.
1Password now takes 5 more seconds to unlock my password database. Somehow this disgrace of an OS slowed down the password hashing process by an order of magnitude.
Switching screen resolutions or connecting to an external screen takes a good 10 seconds of flickering and frozen UI before everything starts working again. This is now actually worse than both Windows and Linux. I dread moving the laptop or touching the USB-C cable (also because USB-C is so brittle) when it's connected to an external monitor out of fear that it'll disconnect/reconnect and I end up in a 30-second cycle of flickering.
I upgraded a couple of days ago, so those are not early bugs. Apple had a year to fix all of this. The Xcode thing might be an isolated issue but there's no excuse for the general performance penalty or the stuck update badge which has many hits on search engines suggesting it's a widespread issue.
I'm literally halfway there as I type this, Xcode 'installing components'. Having to upgrade essentially everything just to get the right dev tools for the current iOS is madness, feels like buying a new house to fit the new coffeemaker...
It’s not quite random: it plays the sounds as it gets new email, but then it takes anywhere between a couple of seconds to a minute for the new email to be visible in the UI. Infuriating.
> Xcode was stuck upgrading in the App Store. It would start the process and never make any progress. Cancelling it had no effect. Rebooting cancelled it but the second attempt, while making progress, ended up failing with a generic error message with no actual information.
I just normally kill the store-related daemons when that happens.
Just like Twitter's UI, app developers think they know what content is best for you with a 'feed' or 'featured'... they've completely abandoned chronological ordered lists of content unless you click 2-3 buttons.
[1] Catalina has been painless for me, not sure why my experience was different than everyone else
My one remaining serious annoyance is that my external monitor color settings are screwed up and there appears to be no fix. Reds are purple and everything is just a little washed out, which is a shame for a 4k monitor that was beautiful with Mojave.
Strangely, right before the computer restarts, or if booted in safe mode the color starts to look perfect again, but I can't seem to replicate that in normal operation.
I have to disconnect and reconnect USB-C 3 times, turn off the second monitor, switch inputs, restart the €3000 machines twice or whatever. So annoying, how does this pass QA at all?
Also, don't setup and use multiple users at the same time. That's really messy as well.
I use 1password and it doesn't take 5 seconds to open. Did I accidently install linux or something? because since it's the OS causing your delay it would be causing me to have the same delay.
xcode installs just fine for my entire team. Just did the update myself, worked just fine.
I plug into a dock and undock constantly during the day, and while it could be quickinger, 10 seconds and flickering is NOT my experience.
and what the fk are you doing to your connections that you consider usb-c brittle?!?
>
> echo $'#!/bin/sh\necho Hello' > /tmp/test.sh && chmod a+x /tmp/test.sh
> time /tmp/test.sh && time /tmp/test.sh
Am I missing something here?
I just did this, and the timing between the first and second run was barely noticeable -- in fact, the first run was slightly quicker:
> echo $'#!/bin/sh\necho Hello' > /tmp/test.sh && chmod a+x /tmp/test.sh time /tmp/test.sh && time /tmp/test.sh
> Hello
> /tmp/test.sh 0.00s user 0.00s system 55% cpu 0.006 total
> Hello
> /tmp/test.sh 0.00s user 0.00s system 41% cpu 0.010 total
This is on macOS 10.15.4.
The tone of this article reminds me of a passage from the seminal Google+ Platforms Rant:
> Like anything else big and important in life, Accessibility has an evil twin who, jilted by the unbalanced affection displayed by their parents in their youth, has grown into an equally powerful Arch-Nemesis (yes, there's more than one nemesis to accessibility) named Security. And boy howdy are the two ever at odds. > But I'll argue that Accessibility is actually more important than Security because dialing Accessibility to zero means you have no product at all, whereas dialing Security to zero can still get you a reasonably successful product such as the Playstation Network.
- My external trackpad isn't able to connect, at all. Audio devices require that I kill coreaudiod before connecting, otherwise they just disconnect after a few seconds.
- I can wake the laptop with a bluetooth keyboard, but when it's awake the keyboard stops working. Flipping the switch on the backside of the keyboard lets it reconnect again.
- There are transitions that you cannot disable that makes your laptop feel super slow. In Mojave you could disable them, in Catalina you can't unless you want to run with SIP disabled.
- There's also a super fun bug with mobile hotspot failing to activate, and there's no way for you to just manually connect to your own hotspot, it has to go through this bluetooth activation, even though your mobile hotspot is visible and connectable on all other devices. You end up in situation where you connect to your friends hotspot and they connect to yours, since neither of you are able to connect to your own.
I've given up. The quality control in Apple is down the drain, and have been for quite some time. I'm fixing to downgrade to Mojave this weekend, hopefully that will make it more stable. But I'm not holding my breath. To add injury to insult I'm on my third broken keyboard now. Next time it breaks I might just use the consumer laws and make them refund the laptop so they'll have to take a big loss for creating such a flawed device.
I was considering one of the new 13” MBPs but that seems unlikely if injecting network latency into syscalls is the direction things are going.
If you’re not building Mac/iOS apps, find a Linux laptop you can tolerate for development and an iPad Pro for everything else.
All they'd need to do - and it's very simple - is count the number of requests of each given hash lookup.
Since they know the hash for each of their own executables, that gives a direct count of "most used" through to "least used" programs.
Not sure if they'd have the hash for third party executables though, to know what the given hash request corresponds to.
If they receive the hash for 3rd party executables when developers sign things, then Apple seems like it's able to generate usage stats for their entire OS and 3rd party app ecosystem.
I love macOS, but their software generally has issues with flakey internet connectivity and long latencies - down here in South Africa, ~400ms RTT is not uncommon.
As of Catalina, there's no sane way to install the Nix package manager without losing functionality because macOS now disallows creating new files in the root directory[1]. Nix stores its packages in the /nix directory and it's not possible to migrate without causing major disruptions for existing NixOS and other Linux users. This is too bad, since apart from Nix being a nice package manager, it also provides a sane binary package for Emacs. The Homebrew core/cask versions only provides a limited feature set[2][3].
[1]: https://github.com/NixOS/nix/issues/2925
[2]: https://github.com/Homebrew/homebrew-core/issues/31510
[3]: https://github.com/caldwell/build-emacs/search?q=support+is%...
sometimes old errors and mistakes come back and bite
I really wish Apple would give third parties the ability to create firmlinks (or at least give Nix one), or barring that, give us a sane way to mount encrypted volumes at the same time that the system volume is unlocked.
This comment has worked for me on two machines: https://github.com/NixOS/nix/issues/2925#issuecomment-539570...
It mentions that anyone with an apple developer ID can notarize a qualifying app and submit this notary to the Apple Notary Service. However, the proof of notarization—the notarization ticket—might not be stapled to the application.
In the case of no stapled ticket, Catalina contacts the notary service to see whether a ticket exists. If so, the app is good to go.
[0]: https://eclecticlight.co/2020/05/22/how-come-someone-notariz...
EDIT. More informative link here[1]. It specifically outlines what happens on first run of an app. (and there's a great diagram if you scroll down)
[1]: https://eclecticlight.co/2020/01/27/what-could-possibly-go-w...
But alas the 1000s of engineers gotta be put to work somehow.
I was actually getting a mac mini now that I'm working from home (I thought I'd get better integration with some of the company's wfh infrastructure while still having a unixy environment, so a win/win situation), but I cancelled the purchase after reading this. I get that you can jump some hoops and set some apple specific flags to things so that it works better, but the reason I wanted a mac was to make things easier and not having to look into obscure APIs and features to get simple things working. I was really looking forward to that, but I don't feel that sort of investment will be justified with issues like this in their OS :/
It’s indeed a huge mess, from a privacy standpoint too, not just a performance one. It’s sad also to lose things like AirPlay or iMessage as collateral damage in the process. :/
I just can’t tolerate a machine that hits the network hundreds of times a day when doing normal computing tasks that do not involve the network. They even tolerate this sort of spyware in App Store apps, too.
Is it too much to ask for a polished workstation OS that lets me boot and edit a local text file of notes and save and quit without notifying 4 different parties that I did so?
running just firefox and terminal, ps -ef|wc -l is 198
and many of them have no reason to be on my system.
I think it just skips the checks if internet isn't available. But doesn't that kind of defeats the point of notarization?
EDIT: how to staple: https://developer.apple.com/documentation/xcode/notarizing_m...
It waits 5 seconds while trying to connect, and then it gives up and caches the program as un-notarized, allowing it to run faster on later executions.
Notice that notarization seems to be disabled if the network is disabled from within the OS. To observe the 5 second delay you need to cut the connection outside (e.g., on your router), while the mac still thinks it is connected. I observed it by running catalina inside a virtualbox, and disabling its network.
echo $'#!/bin/sh\necho Hello' > /tmp/test.sh && \
chmod a+x /tmp/test.sh && \
time /tmp/test.sh && \
time /tmp/test.sh && \
echo 'echo Hello2' >> /tmp/test.sh && \
time /tmp/test.sh f=$(mktemp) && \
echo $'#!/bin/sh\necho Hello' > $f && \
chmod a+x $f && \
time $f && \
time $f && \
echo 'echo Hello2' >> $f && \
time $f
On my system: Hello
real 0m0.131s
user 0m0.001s
sys 0m0.002s
Hello
real 0m0.004s
user 0m0.001s
sys 0m0.002s
Hello
Hello2
real 0m0.004s
user 0m0.001s
sys 0m0.002sI have issues with a lot of APIs, but SecKeychain has got to be one of the worst. I don't think it's gotten any love in many, many years. Unlike literally every other Apple API that a Macintosh application might reasonably use, you call its functions (even from Swift) by passing strings as (length:UInt32, data:UnsafePointer<Int8>?) pairs, and getting results out by passing (length:UnsafeMutablePointer<UInt32>?, data:UnsafeMutablePointer<UnsafeMutableRawPointer?>?) pairs, and checking OSStatus return values. Every aspect of it is painful.
In Apple's "Documentation Archive" there's three "Sample Code" downloads related to Keychain. The newest one is for TouchID, and the oldest is for PowerPC. This is an area of the OS that doesn't get much attention.
> This issue has been reported to Apple and assigned FB7679198. Apple has responded that applications should not use this function, though the documentation for SecKeychainFindGenericPassword does not state that it is deprecated
I see that it's now grouped in a section of the docs called "Legacy Password Storage", but not actually "deprecated". Strange. That means you won't get any indication of its non-current status from Xcode, or even reading the release notes.
I like that there's a newer (and presumably less awful) interface. I don't look forward to having to rewrite/retest that corner of my application. Seeing all the CFString/CFDictionary casting and OSStatus checking with the new functions, it still doesn't look all that great.
Setting Terminal as a Developer Tool in Security&Privacy fixed it
I'm currently trying to figure out how to emulate windows from a *nix distribution using qemu. I plan to use this as a "home lab" (k8s cluster or just plain fucking around), but still retain the ability to play an occasional AAA game.
This just makes me wanna start using it for more things besides dev work :(
It'll be interesting to see how much power we developers will let Apple take from us before we jump the garden wall.
I am pretty sure the laptop market has been shrinking generally (as more people have a phone but no laptop). And most developers I know have macs. They probably don't want to make the OS significantly worse for developers...
Nope.
Definitely agree on this one here - I've noticed a big speed improvement when disabling SIP debugging with "csrutil enable --without debug" while in recovery mode.
I should note that the main reason I disable SIP isn't for speed, but to install the yabai window manager to make Aqua far more useful as a developer. I wrote a recent blog post on this, actually (https://triosdevelopers.com/jason.eckert/blog/Entries/2020/5...).
WTAF. If this is really true, this is a reason for me to leave the platform for good. This is just in-acceptable in so many ways.
Wow, this is extremely infuriating! I just ran the "hello world" test script with the network connection disabled and it took 5 seconds to run!
$ echo $'#!/bin/sh\necho Hello' > /tmp/test.sh && chmod a+x /tmp/test.sh
$ time /tmp/test.sh && time /tmp/test.sh
Hello
/tmp/test.sh 0.00s user 0.00s system 0% cpu 4.991 total
Hello
/tmp/test.sh 0.00s user 0.00s system 77% cpu 0.005 totalThere are a bunch of people who can't reproduce the slowness at all, but nearly all downvoted or you have to wade through 100's of comments to get to them.
The majority of comments are just dumping on Macs, nothing whatsoever to do with the content of the article, and seem to be blindly assuming it's true.
And I can't seem to find any substantive discussion of whether this is actually real or not, or just some weird bug on the author's machine.
I don't see any evidence that Catalina is "slow by design", just a single anecdote from the author. I was definitely hoping for some more substantive critique/discussion...
The down votes are because it seems pretty clear that the people who don't experience have long lived instances of their os and likely have grandfathered or disabled security settings. There are a lot of people saying ita pretty easy to replicate with a new os.
And it is, I just did it. Did you?
It's possible that they have certain security features disabled.
> The majority of comments are just dumping on Macs, nothing whatsoever to do with the content of the article, and seem to be blindly assuming it's true.
Welcome to Hacker News…this is common on any discussion on any topic, especially one that many people can understand in some way.
I recently took the trouble to completely wipe the disk and reinstall macos mojave and it's still happening so it's not due to cruft installed over time in OSX. I dunno. I'll deal with it until it gives up the ghost and probably move to a windows machine with the work they're putting into WSL2
The problem was: Parental Control. Apparently, every request was checked and thus slowed the whole thing down. Needless to say, a couple days at least were wasted in this.
I do wonder if the author has something similar going on, either with a directly attached disk or a network share.
Archive: https://web.archive.org/web/20200522164507/https://sigpipe.m...
There's a lot of bullshit on Windows too but nothing near OSX levels of wannabe big brother shit.
Can't think of a better long term short right now in the market than Apple (and sister cult Tesla but the electric story is at least in the early days so they may do ok)
They're not implemented in a braindead way that's being discussed here but they're at the same level big brotherness-wise, if not worse.
I run the commands and get:
Hello
/tmp/test.sh 0.00s user 0.00s system 8% cpu 0.045 total
Hello
/tmp/test.sh 0.00s user 0.00s system 75% cpu 0.005 total
If I'm reading this correctly, the first run takes less than a twentieth of a second, and the second a two-hundredth? I've never experienced anything like "have the entire machine freeze for 1-2 seconds every 10th minute". And I have the slowest internet package I can buy.The only delay that's ever noticeable is when running a program I've installed for the first time, which yes usually seems to take a few seconds, before often telling me the application couldn't be verified or something, do I want to run it anyways. Which makes sense if you're running a checksum on a 400 MB application binary. But after that first time, starting an app is always instant.
Can anyone else elucidate what the author is talking about? They're presenting it as a universal, but maybe there's something else going on with their machine? Clearly something's wrong on their end, but possibly it's just some kind of bug. I'd avoid jumping to conclusions that executables taking a second to launch is "by design".
EDIT: switching from zsh to sh gives more granular results:
Hello
real 0m0.009s
user 0m0.002s
sys 0m0.003s
Hello
real 0m0.005s
user 0m0.001s
sys 0m0.003sSo it's real.
However, scripts are NOT notarised, so what is it doing?
EDIT:
So after digging the scripts are being "checked" for malware, as part of XProtect.
This is interesting, it seems to be hashing scripts and testing to see if its known malware.
Anyway, easy to disable, but weird stuff.
Still no idea what or why the panics would happen, or why the reinstall solved it.
Catalina has been a very bumpy road for me so far.
Who at apple thought it was a good idea to hop on the internet when invoking an application without any warning? This is loony.
Whatever problem you're having, it's a problem specific to your machine.
Why does it need the internet all the time?
Are they checking if the app is dangerous? Are they logging all my activity?
I wonder if that defeats the phone home that this article is highlighting.
I clearly won't switch to their system anytime soon...
I've heard people ask me "why bother with Linux when MacOS is Unix?". Well technically it is from its heritage, but it gets less unixy by the day.
Yes, they have polished the GUI, which makes it easy to navigate by mouse. But, when you need to work in speed mode, then you reach for the keyboard shortcuts.
The problem, is that there are plenty, too much sometimes, and they are often inconsistent between applications.
And yes, the Mac has a keyboard shortcut assignment tool, but it often doesn’t work correctly.
I must give credit to Microsoft here. They at least seemed to have perfected most of the common keyboard shortcuts.
Some good features about Windows shortcuts.
1. Alt-Spacebar to open the windows control menu, to move, minimize, maximize, or close the window.
2. Alt combinations are used to control the active Window application itself.
3. Alt-F4 to close the window. But, I would have preferred Alt-Escape instead, to close the window.
4. Control key for shortcuts inside the application. Like, Ctrl-C for copy. O for open. P for print. Etc.
5. Then the Windows key, to control Operating System level shortcuts. Like Win-M to minimize all windows. Win-L to lock the computer. Win-R to launch a command.
Some feature I would like are to use, Win-Spacebar to open a command search, similar to Win-R, but with the ability to list all possible commands. Similar to activating the command palette on VSCode.
And Ctrl-Spacebar, to activate keyboard commands for the active window. Kinda like Emacs, where I can run macros on it, like highlighting the words that I want, and execute something on it, like changing to uppercase, or converting to comma separated, or whatever else is needed.
Apple is the Father, Apple is the Mother.
After Apple has re-invented or re-written the MSFT playbook of the 90s, nothing surprises me anymore.
Yet I cling to these machines, that take away the freedom to do with my hardware as I please. It's odd.
The first computers I ran OS X on were a Pismo Powerbook and one of the first iMacs. Both with upgraded hard drives and maxed out RAM. They were almost unusable, and we'd put classic OS back on them, a new release of OS X would come out, and repeat.
I later got a chance to use a shiny new G5. I couldn't believe how slow it felt. Same goes for the PowerBook G4. The first Intel MacBook Pro didn't feel any faster.
Somewhere around the i5, Mac OS started to feel 'okay'. But I'd always still feel blown away at how fast a similar machine felt running Windows or Linux.
But I've stuck with it ever since 2010. I remember talking about my 16", saying "It's really fast...for a Mac."
Yes these features could be better implemented, but I'm happy they're there. It's very important to be able to opt out of them, but I like that they're the default.
Notarization needs a cleanup pass and the rest of it seems like it needs an optimization pass.
P.S. The rationale for notarization is to not distribute and thus advertise the filters and detection mechanisms Apple uses to detect malware. If these things were distributed then malware authors could analyze and evade them. Security through obscurity does make a certain amount of sense here as the Church-Turing thesis means there are an infinite number of ways to implement any given thing including malware and there is no single filter or analytical step that can detect all possible malware permutations.
That's true (or else there are 0 ways), but it's not what the Church–Turing thesis says.