Comment by saagarjha | Hacker News Reader

Comment by saagarjha | Hacker News Reader

This blog post has some good discussion about what makes a good locking primitive. On macOS, os_unfair_lock is generally a good choice for many applications. That said,

> At this point, you might wonder if os_unfair_lock – possibly coupled with the undocumented flags – would be a good fit for your codebase. My answer is likely yes but you’ll have to be careful when using it.

I would hesitate to give this advice. The overwhelming majority of applications on macOS do not have performance profiles like Firefox does. The system allocator (rather than jemalloc) is the right choice for all but very few use cases. There are a vanishingly small number of usecases where using the undocumented flags is appropriate.

The rules are different when you’re a popular browser with an existing relationship with Apple and dozens of smart engineers who can (at least in theory) understand the consequences of using this API, and leverage an existing relationship with Apple to ensure that it doesn’t become a liability in the future as the platform evolves. For pretty much everyone else, this should be nothing more than a “wow, neat” blog post.

> leverage an existing relationship with Apple to ensure that it doesn’t become a liability in the future as the platform evolves

I sincerely doubt that Firefox can leverage anything with Apple, or that Firefox engineers have much of a relationship at all with Apple. Who does, really.

I'm not sure it matters though. Many smaller Mac developers have a long history of using private API, and it's largely fine. There's no truly "safe path" with Apple, because Apple does whatever it wants whenever it wants, and I've seen them break "supported" API plenty of times, and leave it in a broken state. But the risks of any given brokenness are small, and you can't predict what will break from year to year, so what can you do except go with what works, and hope for the best.

It's certainly fair to say that the majority of applications on macOS do not have the performance profiles of Firefox, but it's also the case that the majority of applications weren't even using lower level primitives like OSSpinLock in the first place, so replacing it was not a concern.

With all due respect, you’re a small indie developer, and the situation is different when you make a browser used by millions of people. Apple does do whatever it wants, but sometimes it (wisely) realizes that what it wants is people to not be upset that people’s favorite app is broken in the next release of macOS, so occasionally steps are taken to ensure that doesn’t happen. This ranges from committing to SPI for longer than it is necessary for Apple’s own use, to putting in bundle ID or linked-on-or-after checks, or in rare cases reaching out to developers directly.

The target for this comment is developers like you, but those who are less familiar with the platform than you are but of similar import. Those are the ones who will read this and say “wow Firefox says this is how to make my app fast so I should do it!!1!” and what is going to happen is either they’re going to use it wrong, because they aren’t really experts on how locking works, or they’ll copy the code of of Firefox and their app will break in a future OS release while Mozilla will update their code in July.

> Those are the ones who will read this and say “wow Firefox says this is how to make my app fast so I should do it!!1!” and what is going to happen is either they’re going to use it wrong, because they aren’t really experts on how locking works, or they’ll copy the code of of Firefox

You can't save people from themselves, but in any case this seems to overlook what I said at the end: "the majority of applications weren't even using lower level primitives like OSSpinLock in the first place, so replacing it was not a concern."

I wouldn't worry about native Mac developers, who I suspect are mostly a dying breed of grizzled old veterans nowadays. To illustrate, I noticed recently that the last post on https://www.reddit.com/r/macprogramming/ was over 2 years ago, April 2020.

Counter argument - https://www.reddit.com/r/swift/ has posts within the last few hours.

Granted Swift is technically a cross platform language but it feels safe to assume that the vast majority of users are developing for macOS and iOS.

> the vast majority of users are developing for macOS and iOS.

The vast majority of Swift users are developing for iOS.

The article is about macOS and private API. Thus it's only relevant to a small subset of developers who write native Mac software distributed outside the App Store.

Note that https://www.reddit.com/r/iOSProgramming/ is itself also extremely active with posts within the last hour, so it's strange to cite r/Swift as evidence of Mac activity.

The vast, vast majority of the posts on there are for iOS development. macOS barely gets mentioned.

I just got a message from a contributor to a popular macOS project I work on about this blog post and how it can help improve our performance, so I think you're mistaken.

> so I think you're mistaken.

What exactly do you think I'm mistaken about? The conclusion of your sentence doesn't follow logically from the first clause.

The readers of your comment here know nothing about this macOS project or the contributor who sent you a message. In any case, there's a vast gulf between someone sending you a message and someone actually writing and releasing bad code.

Happily, rather than using non-API OS internals, Firefox could recognize that their behaviour does not match normal applications and so they might have to actually implement spin locks themselves rather than relying on a API that is designed to support more general apps. That's what the other browser engines do, including webkit and that's an actual system library on macOS and iOS.

Yes but also do note that WebKit is probably among the most NIH codebase on the platform.

Those reasons are largely that the things that are perf critical for a browser engine don't match anything that normal apps do. That's the same reason gecko also isn't using the system allocator, as we see here.

You really don't want the rest of the OS making the performance trade offs people expect from browser engines these days.

Blink uses os_unfair_lock on macOS: https://source.chromium.org/chromium/chromium/src/+/main:bas.... As does WebKit: https://github.com/WebKit/WebKit/blob/4c09a48ea418ff15f65d6c....

Yes. Blink and WebKit, and I suspect a bunch of other - correct - software uses os_unfair_lock. Because os_unfair_lock is public, documented, and stable API.

The function this article is talking about is not API, it is part of the OS internals, presumably used to implement the standard API functions.

I fully agree but also want to emphasize that the people who work on this at WebKit also enjoy making their own stuff.

The solution here, rather than using OS-internals is for Firefox to implement spin locks themselves, which is the level that they should be thinking if they're making their own allocator. This is what webkit, chromium, etc - webkit even has a blog post: https://webkit.org/blog/6161/locking-in-webkit/

I get the impression the author was a little hesitant to do any spinning in userspace.

> As an intermediate solution, I initially fell back to OSSpinLock on older systems. Later I managed to get rid of it for good by relying on os_unfair_lock plus manual spinning in user-space.

You're confusing general purpose locks with special-purpose ones used in the memory allocator. Firefox has its own GP locks, you can find them here: https://searchfox.org/mozilla-central/search?q=&path=xpcom%2...*

However those are not suitable for use in the memory allocator, and neither are WebKit's. WebKit uses os_unfair_lock within its memory allocator:

https://github.com/WebKit/WebKit/blob/520379e30f3b2b6d4de995...

And so does Chromium:

https://source.chromium.org/chromium/chromium/src/+/main:bas...

Correct, and that isn't what this article is talking about.

This article is about an internal function in the OS that they've decided to is what they'd like to use, and that the only reason they aren't using it is because it doesn't exist in older versions of macOS, because you know, it is an implementation detail of the OS.

os_unfair_lock are a documented part of the OS API: https://developer.apple.com/documentation/os/1646466-os_unfa...

Just a note thay you're replying to the author, so I'm sure they know what it's talking about :)

Reply on news.ycombinator.com