Hacker News Reader

Comment by rossmohax | Hacker News Reader

In Apple defense, sloppy fsync behaviour is clearly documented: https://developer.apple.com/library/archive/documentation/Sy...

View on HN

That's not defence. It fails the principle of least-surprise. If everyone's experience is that fsync is flushing then why would somebody think to look up the docs for Mac in case they do it differently?

They do it according to POSIX spec. Linux is the oddball here.

The POSIX spec is deliberately ambiguous about this.

So? Did Linux do it like that before?

https://lwn.net/Articles/270891/

It does it like that now, which is what I'd expect if I'm writing software.

I’d argue more people develop on non-Linux devices such as Windows and MacOS on linux itself.

F_FULLFSYNC is nonstandard. As far as I know there is no standard-complicant way to get data on to stable storage on macOS. That's a bit of a problem. It makes a lot more sense to make the standard-compliant way actually sane.

I have said a few times already - F_BARRIERFSYNC. This is likely equivalent to what linux is doing.

edit: sorry - not 'standards compliant' (whatever that is - does linux declare support for SIO?), but probably what you are looking for.

It isn't. I already replied to you above. A barrier does not guarantee data durability and we already know Linux fsync() == macOS F_FULLFSYNC because they have the same (lack of) performance on the same hardware.

Thanks marcan. Apologies for wasting your time.

I just looked into this, since what you say and what Apple’s documentation says are two different things.

Here is Apple’s documentation:

https://devstreaming-cdn.apple.com/videos/wwdc/2019/419ef9ip...

F_BARRIERFSYNC: fsync() with a barrier

F_FULLFSYNC: Drive flush its cache to disk

This sounds like the Linux fsync() and Linux syncfs() respectively. What you say is that F_FULLFSYNC is the same as Linux fsync() and your performance numbers back that up. Unfortunately, you would only see a difference between Linux fsync() and Linux syncfs() if you have files being asynchronously written at the same time as the files that are subject to fsync()/syncfs(). fsync() would only touch the chosen files while syncfs() would touch both. If you did not have heavy background file writes and F_FULLSYNC really is equivalent to syncfs(), you would not be able to tell the difference in your tests.

That said, let’s look at how this actually works on Mac OS. Unfortunately, the apfs driver does not appear to be open source, but the HFS+ driver is. Here are the relevant pieces of code in HFS+:

https://github.com/apple-oss-distributions/hfs/blob/hfs-556....

https://github.com/apple-oss-distributions/hfs/blob/5e3008b6...

First, let me start with saying this merits a faceplam. The fsync() operation is operating at the level of the mount point, not the individual file. F_FULLSYNC and F_BARRIERFSYNC are different, but they both might as well be variants of the Linux syncfs().

For good measure, let us look at how this is done on the MacOS ZFS driver:

https://github.com/openzfsonosx/zfs/blob/master/module/zfs/z...

The file is properly synced independently of the mountpoint, such that other files being modified in the file are not immediately required to be written out to disk. That said, both F_FULLSYNC and F_BARRIERFSYNC on MacOS are mapped by the ZFS driver to the same function that implements fsync() on Linux:

https://github.com/openzfs/zfs/blob/master/module/os/linux/z...

For good measure, let us look at how syncfs() is implemented by ZFS on Linux:

https://github.com/openzfs/zfs/blob/master/module/os/linux/z...

It operates on the superblock, which is what MacOS’ HFS+ driver does.

From this, I can conclude:

Linux syncfs() == macOS F_FULLFSYNC on HFS+

Linux fsync() == macOS fsync()/F_FULLFSYNC/F_BARRIERFSYNC on ZFS

Also, MacOS F_BARRIERSYNC is a weakened Linux syncfs() and Apple’s documentation is very misleading (although maybe not technically wrong). POSIX does allow fsync to be implemented via syncfs (sync in POSIX, but I am saying syncfs from Linux to be less confusing). However, not issuing and waiting for the completion of an IO barrier on fsync is broken behavior like you claim.

I am not sure how MacOS APFS behaves. I imagine that additional testing that takes into account the nuances in semantics would be able to clarify that. If it behaves like HFS+, it is broken.

Edit: Upon further examination and comparing notes with the MacOS ZFS driver team lead, it seems that HFS+ is syncing more than requested when F_FULLFSYNC is used, but less than the entire filesystem. You are fine treating it as a Linux fsync. It is close enough.

F_FULLFSYNC is the equivalent to this:

https://man7.org/linux/man-pages/man2/sync.2.html

Linux only stopped doing the clearly wrong thing in 2008 or so iirc.

It is still dumb that there's a definition of fsync() that does not sync :-/

> why would somebody think to look up the docs

It seems reckless to me to not do this when you're interacting with the filesystem using low-level APIs (i.e not via Swift/Obj-C).

> That's not defence. It fails the principle of least-surprise.

Welcome to C APIs in general, and POSIX in particular.

>That's not defence. It fails the principle of least-surprise.

Only if the standard where anything else is a "surprise" is 2022 Linux.

Many (all?) other unices and macOS itself since forever work like that. Including Linux itself in the past [1]

[1] https://lwn.net/Articles/270891/

Drive caches also used to not exist in the past. At that point, behavior was the same as it is on Linux today. It then regressed when drive caches became a thing.

Maybe it not being added to OSes when drive caches came into the picture was arguably a bug, and Linux has been the first OS to fix it properly. macOS instead introduced new, non-buggy behavior, and left the buggy one behind :-)

> Drive caches also used to not exist in the past. At that point, behavior was the same as it is on Linux today. It then regressed when drive caches became a thing.

You mean in the 1980s? Linux wasn’t used before this wasn’t a concern for sysadmins and DBAs. This concern has been raised for years - back in the PowerPC era the numbers were lower but you had the same arguments about whether Apple had made the right trade-offs, or Linux or Solaris, etc.

Given the extreme rarity of filesystem corruption being a problem these days, one might conclude that the engineers who made the assumption that batteries covered laptop users and anyone who cares about this will be using clustering / UPS were correct.

The minute storage manufacturers introduced drive caches is the minute this bug became the responsibility of storage manufacturers. IMO it’s not the kernel’s responsibility.

Now Apple is the primary storage manufacturer for Mac.

Any references for Unices traditionally skipping FUA or synchronize cache to the storage stack? Sounds surprising to me. Here's Solaris for example: https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSDiskWrit...

Also re Linux here's eg PostgreSQL 9.0 documentation saying ext4/zfs + scsi used the "SYNCHRONIZE CACHE" command with fsync even back then, and a equivalent SATA command being used by the storage stack with SATA-6 and later drives: https://www.postgresql.org/docs/9.0/wal-reliability.html

On ZFS on Solaris / Illumos you get a choice of whether fsync() acts as a write barrier or actually waits for writes to complete.

I’d argue maybe .5% of people are working on something where this is even close to being a concern. Those people probably know what they need to use.

Apple doesn’t need to defend anything.

I am sick of this callous and capricious disrespect for users and their data, rampant throughtout this wanky industry.

Do lawyers use Apple computers? Do they work on important documents relating to life and death?

Some people have literally been executed because developers couldn't do their job properly. People have been sent to jail for decades because developers fucked up in the british postmaster scandal.

Average people life in a dangerous world- work with documents about their financial wellbeing. They live in opressive countries where being gay is punishable by death. They drive 2 ton death machines. And now that we have put computers in places where life and limb depends on them, we are responsible for doing the job properly, that's why we get paid.

Reply on news.ycombinator.com