Hacker News Reader

Hacker News

Comment by amelius | Hacker News Reader

Full thread

amelius

So is this any different on Linux? Why is OSX in the title?

Are syscalls more expensive on OSX?

View on HN

It's interesting that if you google "how do i create a large file on osx" you get guidance for this command[1].

But yes, syscalls are more expensive on OSX.

[1]: http://stackoverflow.com/questions/26796729/quickly-create-a...

>"But yes, syscalls are more expensive on OSX."

Can you elaborate on this?

The XNU implements system calls in the same way a BSD system does. If you are talking about the Mach aspect of OS X, the BSD part can call down to the the Mach directly without using Mach messages. XNU is not a traditional microkernel even though Mach is in there.

I don't think anyone seriously expects OSX to be as performant as Linux, since it doesn't feel faster, and a lot of the problems are obviously[1] the results of the microkernel side. That being said, a little googling shows a direct head-to-head[2] comparison.

[1]: http://sekhon.berkeley.edu/macosx/

[2]: http://www.academia.edu/2685902/An_Analysis_of_Mac_OS_X_Leop...

I have spent a great deal of time studying the code path taken on system calls in xnu/osfmk[3] versus Linux[4] in building my Linux emulator for OSX[5].

[3]: https://opensource.apple.com/source/xnu/xnu-1504.3.12/osfmk/...

[4]: https://github.com/torvalds/linux/blob/master/arch/x86/entry...

[5]: https://github.com/geocar/ml/

I think many OSX supporters do the platform a disservice by defending it without spending any time studying it or the competition. OSX has some serious performance-based weaknesses, that are perhaps a real worthwhile trade for a lot of things "just working" -- especially when compared to Windows or Linux -- but are still a weakness.

I certainly wouldn't classify myself as an "OS X" supporter by any means. I don't think I've ever actually defended it :)

I was more responding to the blanket statement that system calls are more expensive on OS X.

You first citation is ancient by the way, thats 11 years ago now.

You second link states in the conclusion section:

"First, Mac OSX uses a hybrid monolithic and microkernel architecture inwhich system calls must be wrapped into an RPC messageto the Mach microkernel. "

But in XNU/Darwin there are three different mechanism for systems calls - traditional BSD style traps, Mach traps, and Mach RPC.

I didn't really understand how the other 3 links related or I guess specifically what I was looking for in those.

> So is this any different on Linux? Why is OSX in the title?

mkfile is not available on Linux. The equivalent utility is xfs_mkfile or fallocate.

No, mkfile creates a file with written extents (no sparse or uninitialized areas), while fallocate will (with ext4/xfs/...) only map some extents which are marked uninitialized. So while mkfile 10G would have to write ~10 GB of data, fallocate 10G is nearly instanteneous, because it most likely just allocated a single extent.

xfs_mkfile could do what mkfile does, the description isn't conclusive enough.

You can use good old `dd` for this:

    dd if=/dev/zero of=test bs=1k seek=2m count=1

Or python:

    python -c 'f=open("test", "w"); f.seek(2e9); f.write("\x00")'

Both these calls create a ~2GB file with holes in it. They return almost instantly on OSX.

EDIT: oh wait, this isn't what you want. you want a file _without_ holes, my mistake.

Yeah without hole would be

    dd if=/dev/zero of=test bs=1k count=2m

OSX doesn't support sparse files...

It does, just not at the HFS+ layer. Sparse bundles are the functional equivalent at the Vitual file system layer and are used by Time Machine and are useable by developers or via the CLI. Sparse images are entire virtual volumes and are used for FileVault.

https://en.m.wikipedia.org/wiki/Sparse_image

>just not at the HFS+ layer

macOS supports sparse bundle with HFS+

As I was saying, that's implemented above HFS+

To clarify: FileVault on 10.6 and earlier; current FileVault is FDE.

I guess it is the same. Try doing a dd if=/dev/sdX of=/dev/null or dd if=/dev/zero of=/dev/sdX (be warned, the latter wipes the disk!).

Then, experiment with various buffer sizes (bs=1k, 10k, 100k, 1M) - I personally use 1M with dd.

Be warned on OS X you have to use /dev/rdiskX instead of /dev/diskX, as the latter is a buffered version that usually is slower.

You know, you can dd to a file to avoid destroying a drive. And you can provide a `count` to write fixed-size output files, dd will write bs×count bytes (technically it writes ibs×count, and bs sets both ibs and obs).

    dd if=/dev/zero of=foo bs=1k count=1m

would write 1GB in 1k blocks, and

    dd if=/dev/zero of=foo bs=1m count=1k

would write 1GB in 1m blocks.

> You know, you can dd to a file to avoid destroying a drive.

Then you're at the whim of more or less obscure caching layers in the Linux kernel. At least there's a filesystem cache plus something in the block-device layer that reads ahead in order to speed up fread calls with low buffer sizes.

Also, due to filesystem fragmentation the file will be distributed across locations on the disk - that doesn't matter much for SSDs, but on "spinning rust" the head seeks distort the performance. A dd on a raw disk, however, will not cause any seeks except those for background processes' file operations.

You can add `direct` to dd if you're concerned about the caches. And if you want to avoid seeks, just use `fallocate`. It'll do the Right Thing(tm) on CoW and non-CoW filesystems - either just creating a hole or reserving extent space, without having to write all the blocks in either case.

I think the main difference is that `mkfile' doesn't exist on Linux? As far as I can gather, it's a Sun OS utility, that BSD has and therefore OS X.

I've read that some Linux distributions have `mkfile', but it's just a script wrapper around `dd'.

Good point, the reason is simple: I didn't check on Linux (and have updated the post to reflect this). I am guessing it would be similar, but don't have any data.

It is.

Reply on news.ycombinator.com