Comment by otterley | Hacker News Reader

Comment by otterley | Hacker News Reader

Parent Full thread

It's important to get the terminology correct when discussing VM behavior. Page-outs are not identical to swap-outs. And unfortunately, due to the age and architecture of the Mach kernel, there's no way I'm aware of on OS X to measure the rate of the latter.

Page-outs refer to a file-backed page being committed to disk; these are perfectly normal and expected (program writes to a file, kernel eventually commits to disk). Every OS guarantees that file-backed pages be committed within some reasonably short period of time to reduce the risk of lost data in the event of a power outage (the sync interval).

A high page-out rate doesn't necessarily imply memory pressure; it may simply mean some program running on the system is writing to files frequently. However, memory pressure may temporarily increase the page-out rate if the sync interval hasn't yet kicked in.

Swap-outs, on the other hand, relate to anonymous memory pages (i.e. the heap). Swap-outs, in contrast to page-outs, are generally bad and indicate severe memory pressure.

That's not what pageouts as reported by OS X refer to. Try the following experiment:

  1) Open two Terminal windows.
  2) Run top in one of them.
  3) Note the pageout number.
  4) In the other, type echo foo > bar.txt
  5) Refer again to pageout number.

You will see that it did not increase, even though the system just wrote to a file. This will be the case even if you wait a while.

On OS X, the reported pageouts are dirty memory pages being tossed, not writes of files to the filesytem.

Your definition also does not match the historical distinction between swapping and paging and the distinction you draw is idiosyncratic. Originally, swapping referred to swapping out all of a process's memory, while paging is done in chunks. No system has true "swap" in this original sense any more, so the terms "swap" and "page" are now basically interchangeable.

Everything I said is true for at least for Linux, where files are memory-mapped into the process space, and swapped page counts relate to anonymous pages (look at /proc/vmstat for the gory details; note that pgpgin/out and pswpin/out increment under different conditions).

OS X may admittedly be different. It's unfortunate that it doesn't expose more counters to help show what's really going on.

As for "reported pageouts are dirty memory pages being tossed" - where are they being tossed to, if not the filesystem?

They are usually being tossed to the pagefile (except in the case of read-write non-anonymous file mappings, which are not super common on OS X and which are functionally equivalent to paging to the page file anyway).

On OS X files aren't memory-mapped by default.

Where did all of that come from? It's fantasy as far as I can tell.

I have never heard "Page-outs refer to a file-backed page being committed to disk."

I think you are confusing it with a file cache or buffer cache commit interval.

A page out is when a page of memory is written to disk ("paged out") so you can use more live memory than you have physical RAM. Fun fact: the kernel is smart enough to not page out the page in code.

> I think you are confusing it with a file cache or buffer cache commit interval.

That's a distinction without a difference - the file/buffer cache stores memory pages. They are simply file-backed pages.

Dredging up my knowledge from OS class, there are three sections of memory for a (Von Neumann architecture) program: the data section (also called the program section), the stack, and the heap.

The data section is the actual compiled bits of the program: the machine instructions themselves. The stack and heap are memory used by the program at run-time.

Completely separate from that, on a Linux system, there is the VFS buffer-cache system. When you write to a "file", you are actually writing to VFS buffer cache memory, and the OS then flushes that dirty page of VFS cache to disk, at which point the write is committed.

Your description of page-outs sounds as though you are describing the operation of the VFS buffer cache flushing its dirty pages to disk.

In my college education (1998 to 2003), we used the terms swap-out and page-out interchangeably.

However, I have heard some people make the distinction of using page-outs/ins to specifically refer to just the data section of an app being written to / read from disk, with term swap-outs/ins referring to the same operation on the stack and heap.

But I've never encountered the term "page-outs" being used to refer to dirty file-backed storage pages being flushed to disk. Is this an old-school hacker thing? Or did I misunderstand and create a straw man?

"to reduce the risk of lost data in the event of a power outage"??

Doesn't sound right to me; if the power goes, you're rebooting into a fresh memory image, not recovering to the memory state in the current pagefile.

Eagerly synching back to disk is surely more about making sure you have a minimal amount of dirty pages in RAM at any time, so that in the event that you need to use a lot of RAM for something else you don't have to wait until the backing store updates before the page space in RAM is free... no?

Edit: Oh, I see - you're saying that 'page out' refers to writing a page of data back to disk through a memory-mapped file - it's basically just a disk write, and has nothing to do with memory pages or pagefiles. Hmm... is that terminology distinction universal?

> Every OS guarantees that file-backed pages be committed within some reasonably short period of time

Just curious: is that "short period of time" usually in order of seconds or minutes? (I know different OSes have different defaults; I just want to know a ballpark figure!)

ReiserFS on Linux was somewhat unusual in that it had a 5 second timer used to flush all of the dirty VFS blocks to disk. If I recall correctly, EXT2 (the previous popular filesystem) just flushed everything as soon as possible.

The first time I installed a box with ReiserFS, I recall being alarmed by my box "ticking" every 5 seconds.

It's there to keep disk from trashing the head around, so I believe it's not more than a few spins of the platters.

:) Actually, I wanted to ask whether it's in order of milliseconds or seconds! Don't know how the "minute" found its way into my comment. A minute is reaaaly long.

Reply on news.ycombinator.com