I've been a Mac user since the beginning, and by far my biggest frustration is the perpetual running-out-of-RAM, even when I close basically everything. I have 4GB of RAM, and frequently catch kernel_task using at least half of it.
In the meantime, I'm also hedging my bets, and I've gotten very comfortable with Windows 7 for productivity (ok, it's really for gaming) and Ubuntu Linux for web/LAN serving.
Users of desktop systems clearly don't like this behavior, in fact they'll do crazy things like purging disk cache via cron every minute to try to stop this from happening.
Process A (let's call it Safari) allocated 600MB of memory. Out of this 600MB, it hasn't used 400MB for quite a while (because, for example, it contains data for tabs you haven't looked at for hours). Now I'm not sure how Darwin does this but I know for a fact that Windows NT kernels will try to write the contents of in-memory pages to the disk at the first good opportunity; this way they save time when the pages in question will really get paged out to the disk. I assume that there's a similar mechanism in Darwin. So it's very likely that the 400MB in question is already on the disk. Now the user starts process B (let's call it Final Cut Pro) that reads ands writes to the disk very heavily, and typically the same things. It's not an unreasonable thing to do on the kernel's part to just drop Safari's 400MB from the physical memory and use it for disk caching Final Cut Pro. Throw in a few mmaps to the picture and suddenly it's not obvious at all which pages should be in the memory and which pages should be on the disk for the best user experience.
The problem with this line of reasoning is that a large amount of cache will often not give you much more benefit than a small amount. Indeed, that's the nature of caching: you get most of the benefit from the first bit of cache, but the level of added benefit drops dramatically with more cache.
What if using 400MB of cache for FCP only gave 5% of a net performance advantage over using 40MB of cache? Would it still be worth it to take away that extra 360MB from Safari?
And there's the issue of human psychology: people deal much more easily with a little slowdown spread evenly than with a full-on stop for a short amount of time (even if the full-on stop scenario gives you greater average performance). I'd prefer Aperture run 5% more slowly than it might otherwise, if that meant I never saw a beachball when running Safari.
It would seem there's a simple solution -- another number on the system monitor displaying how much memory is available for use if needed.
So if my memory is "full" with a bunch of just-in-case stuff, I'll gladly swap it out for real data that a real running process is using it. But if it's "full" of data in use for running processes, then I want to think twice about opening a new application. And I want my memory manager to tell me the difference between those two "full" cases.
total used free shared buffers cached
Mem: 2042520 1816496 226024 0 294344 486908
-/+ buffers/cache: 1035244 1007276
Swap: 4194300 8172 4186128