Addressing The Outmoded Swapping And Paging Strategy in OSX | Hacker News Reader

Addressing The Outmoded Swapping And Paging Strategy in OSX

dropsafe.crypticide.com

37 pointslallysingh22 comments

There is also the "throw money at it" fix for this – an SSD. I've been enjoying one, of the Intel X25-M G2 variety, it solves VM paging/thrashing slowness very nicely and is hella fast to boot.

I have the same drive. I currently have 6GB of VM, but I've had it up as far as 14GB with no issues.

I've actually had Photoshop run out of the 20 GB that are free, by allocating all of it to manipulate a huge image. It takes about two seconds for it to allocate all of that space.

no pun intended, right? :P

The reason that firefox shutdown is so slow is because it's paging everything back in to run the destructors for the objects that got paged out, not because the OS is trying to give the swapfile back.

This article is an example of why Apple was smart for going Unix. It attracts really smart geeks that can help shed light on performance issues in a way normal consumers just can't.

Unfortunately this is not the smart geek that they are looking for. He's worse than a normal consumer: he's a power user. Note how he's not done any benchmarks at all, he's just insisting that this must be the cause because the handholding Linux distros he's familiar with all use swap partitions, so OS X must just be retarded or something.

Paging is fucked in OS X, but this isn't why.

He focuses entirely on its use of swap files, which is not a problem at all, especially since the middle of the 10.5 betas. Before that, the system would only allocate new swap files and never do compaction or reclamation, so if your filesystem was normally within 10G of being full, it could page its way into filling it, and grind into a performance hell orders of magnitude worse than what he's complaining about (for years I had to reboot every 100 days or so to reclaim swap). Insisting on using fixed swap partitions made sense twenty years ago, but it sure as hell doesn't today as long as you do GC.

Paging on OS X is much higher latency than any other modern unix, but the fault lies with how VM works in the guts of Mach, not some userland configuration trivialities. Chapter 8 of Amit Singh's Mac OS X Internals has details of the implementation but doesn't go too far into why it sucks.

blasdel, would you reading Mac OS X Internals if Apple wasn't Unix?

Smart Geeks intelligently exploring and debating things like this is a win for any OS.

> Insisting on using fixed swap partitions made sense twenty years ago, but it sure as hell doesn't today as long as you do GC.

The main advantage of using fixed swap partitions is that you don't have to go through the file system to access swap. This is why swap files used to be slower on Linux with the 2.4 series of kernels -- with 2.6 and swap files, the kernel builds a map to the LBA at startup for direct physical access, so there's no performance difference any more. Shame on Linux distros that still use swap partitions in 2010.

Windows does the same, which is why Windows doesn't need (and has never supported) swap partitions, either.

However, this sort of map implies that while increasing swap size is easy, reducing the size of swap files is really difficult while the system is online, which is why Windows (and I believe Linux) don't attempt to solve this problem.

You can totally reduce the size of swap files on Linux, it's done the same way it is on OS X. It even has a nice syscall: swapoff(2) and a little utility that calls it swapoff(8) that comes with the standard util-linux package.

You create another new swapfile, turn it on, and then turn off the old one. The VM will copy all the old pages still in use in the old file and distribute them among the highest-priority enabled swaps with free space by round-robin. If you create the new swapfile sparsely, you don't even chew much disk space.

I discovered something hilarious in Linux's OOM-killer a while back. It has a bunch of heuristics for picking what process to kill next to save the rest: the very first one is "are any processes blocked in the swapoff syscall?"

These sorts of mistakes seem to be common: forgetting that Moore's law means that memory settings need to be adjusted at least every other year. For some reason, one of the Smalltalk distributions still uses default GC settings from 1994! That's only about two orders of magnitude off! Everyone just knows to go and tweak those.

Clearly the solution is to disallow multitasking.

But seriously, I find it strange that so many people in the comments section are complaining about how much swap Firefox uses. Don't they have machines with more than 128M of RAM!? (My Firefox is using 131M of RAM right now.)

My firefox memory usage has regularly gone over 1 gigabyte of RAM. Currently, it's sitting at 531 megabytes of resident pages, with 39 of them shared. It's also at 1.4 gigabytes of address space used.

Ah. I use conkeror (an xulrunner app) and Adblock Plus, and that's it. If you use plain firefox with a lot of extensions, I guess it uses more memory.

Adblock Plus, Flashblock, Zotero, and I have Firebug installed but not enabled.

Would this also make a difference on a Macbook Pro? It seems that it should... When I look in my /var/vm I also have a collection of incremental swapfiles in there.

I also get lots of slow-downs when closing apps - exacerbated by the fact that I swapped my MBP's hard drive for a 5'400rpm, bigger one, which means disk IO is even slower. Anything that reduces disk IO would be welcome.

If you force-quit / SIGKILL the app, it won't bother to page as much of its bullshit back into memory as it's terminating it. I'm kind of baffled by the continued presence of graceful shutdown procedures in an age when everyone does recovery on startup anyway.

The problem isn't that the 5400rpm drive is 'slower', but that the average latency is higher (though only by about 1.5x) -- it likely has higher streaming bandwidth simply by virtue of greater areal density.

By the way, Apple does just that in Snow Leopard. The processes now have a "Sudden Termination" flag, and if a program has no unsaved data and is ready to quit, it sets this flag on itself. When shutting down such a process, the system sends it SIGKILL instead of a usual SIGTERM.

revisited:

http://dropsafe.crypticide.com/article/3915

very well done.

i wonder what the effect would be if you simply increase the paging size, as he describes, without creating the separate partition. this solution might be more appropriate for those of us who simply want to avoid the beachball-of-death, and aren't to hung up on speed.

Where did the article go? I can't resolve the host.

I didn't have any problems reloading right now.

http://downforeveryoneorjustme.com/http://dropsafe.crypticid...

It's just you. Maybe try again in a few?

Reply on news.ycombinator.com