Hacker News Reader

Comment by w0utert | Hacker News Reader

Not this again, we already went through this a few weeks ago.

Back then, I thought the conclusion was that there is nothing broken about OS X memory management, and that with every 'fix' you come up with, you will just introduce another degenerate corner case. The same holds for any OS, trade-offs are made that may have some negative effect in some cases, to the benefit of the general cases.

I don't recognize any of his symptoms anyway, and my OS X computers get pretty RAM-heavy use, with almost always a linux VM open, XCode, Safari with ~10 tabs, iTunes with a few thousand songs, etc.

Edit: Just to be sure I read through some of the links he provides that are supposed to explain what is going on and why the fix would be of any help, but nowhere do I see any hard facts that demonstrate what is going on. Only that he 'saw in vm_stat that OS X was swapping out used memory for unused memory'. I'd like to see some actual evidence supporting this statement.

View on HN

The thing is, Lion runs like crap without an SSD or insane amounts of RAM for many people. It doesn't help if it works for some, or even the majority. If anything, we need more people digging in OS X internals. I am sorry to use a tired analogy here, but Vista did work great for many too.

I have two Macs, one brand new, one migrated from SL, and 10.7 Safari was almost unusable on both until I installed SSDs. If that isn't a negative effect in every possible use case, then I don't know one. I actually guessed it was just that Lion inofficially dropped support for HDDs (by removing all caches or so).

I upgraded to an SSD while on Leopard, and was amazed at the speed. Snow Leopard continued to impress. Lion slowed things down terribly (even with the SSD!), but I have good news: ever since Mountain Lion DP2, it's been fast again.

Yes, I am a fan of ML so far too.

Given that Apple has fixed none of my reported bugs in 10.7, but I can't reproduce many of them in 10.8, I wonder if it even makes sense to analyze 10.7 anymore - seems it's a done deal for Apple.

This is very good news indeed. I'm okay that they don't backport bugfixes as long as ML comes out in a reasonable timeframe.

ML will be a paid upgrade right? So people who paid for Lion are worth nothing?

I've been trying to get people I know at Apple to start referring to it internally as Mountain Goat.

It's working.

Are you implying that Mountain Lion is slow?

I have the same experience: my MBP with 8GB RAM and SSD is pretty fast running Lion. But my two-year old Mac Mini at work with 4GB and platter disk runs terribly slow ever since I did a clean Lion install.

> The thing is, Lion runs like crap without an SSD or insane amounts of RAM for many people.

YES. I have a MacBook Pro Core i7 from a little while back with an old style spinning rust drive and a 11" MacBook Air Core 2 Duo.

For purely CPU bound things, sure, the Core i7 kicks the pants out of the Core 2. Same for videogames. For day to day use, though, switching between Eclipse, Xcode, Chrome, etc. the Air provides a much more uniform experience. At its best it's far slower than the Pro at its best, but at its worst it's much faster and more responsive. I rarely see beachballs on the Air. I used to see them all the time on the Pro (the Pro has been sitting on a shelf for the past eight months as I switched to working exclusively on my Air, partially for this reason).

So my experience is that something may not be broken, but something definitely isn't set up optimally for users with poor disk performance and high memory/CPU performance.

Why don't you spend $140 and add an SSD to the pro? You can keep the HDD for an extra $80 with a cd drive bay HDD caddy.

Doesn't the latter void warranty?

Maybe technically, but unless you strip every screw on your way into the machine, they won't know you ever had the cd drive disconnected. Just put it back when you take it in for service.

It doesn't appear to, however if you ever have to get your machine serviced you have to painstakingly undo and redo the installation, as Apple doesn't guarantee the machine that comes back from the factory will have your extra drive in it.

If I were to do it over again I'd just get a larger SSD and leave the optical drive alone.

Not in my experience. I did exactly that (install SSD in cd drive bay, and use orig hd for a second drive), and when I had a fan issue it was not a problem.

Only if you try and claim the warranty for something you broke c.f. https://en.wikipedia.org/wiki/Magnuson-Moss_Warranty_Act

Yes, but you can un-void it by plugging your CD drive back in. Note that replacing the hard drive voids the warranty as well.

Only voids the warranty on the hard drive, in all actuality they don't care... They will simply call up and ask about it and you tell them you replaced it due to corporate policy and all is well.

I got this one for $16 (no enclosure for your old CD-ROM like the $80 one): http://www.amazon.com/gp/product/B0058AH2US/ref=oh_details_o...

Left some big files on the old HD, and symlinked them. The disk stays idle in the CD bay until I need it, then spins up.

As I said, it was only part of the reason I shelved it.

It also turns out I rather like working on an 11" screen. Keeps me focussed.

This

I haven't tried ML yet. My MBP is brand new (bought in January 2012), factory configuration (4GB RAM, Lion)

If you use it 'lightly' (that is, only Safari open) it's a breeze. But of course, it's never only that if you want to do any work.

Frankly, 4 GB should be enough! My past machine (with 3GB - and Linux) would rarely swap (in fact I could keep swap off and use a Windows 7 VM) But you can go only so far with an aging CPU

Some of the slowness can be attributed to Safari/Firefox, sure

But it really seems to have something wrong. Maybe they really neglected people with spinning disks.

(Yes, I considered buying a MacBook Air but 128Gb was not enough for me and the other options were above my budget)

I have 12GB of RAM and Lion runs like crap on my iMac 2.93GHz i7.

It's like the system discards pages of programs just because the app has been inactive for an hour or so. So when I come back and start the same app the f*cking rotating HD I have sounds like a birds nest for too long periods.

Edit: Disabled the pager and the system now seems much more quiet regarding to disk seek noise when I start apps. Feels like a new machine! :-D

> It's like the system discards pages of programs just because the app has been inactive for an hour or so.

From what I've read, Windows memory manager does the same thing - after a while, it swaps out unused pages, even if plenty of free memory is available.

I wonder what's the logic behind this - did the engineers assume that the speedup coming from more free memory being available for disk cache is worth the hassle of waiting for the swapped out page (when it's actually needed)?

The logic is for things like daemons or infrequently used processes that wake up maybe once a day. They don't need to be in ram all the time needlessly. So swapping out their memory that hasn't been used for N hours isn't a bad thing.

And yes, the logic is sound, its better to use a bit of swap for an infrequent daemon and let 4-5 megs of memory be at the ready if needed than leave it in place all the time. The "speedup" is not a speedup for your use, its to allow for better memory management. Which is what the VM subsystem is there for. Second guessing it all the time just makes its job harder.

Swap use when there is free memory isn't a bad thing. This fetish people have with their OS using swap at times seems to border on the ridiculous side. My iMac at home has 16g of memory and 400g of swap used right now (8g active, lots of file cache that'll get purged). Most of the swapped files belong to things like my ruby+pry repl, a clojure repl I haven't touched for 2 days, and other random things I don't use often enough to warrant they stay in active ram. Why SHOULDN'T that memory be reclaimed and at the ready for a new program or some other request? Its just going to page it out then and likely take longer to do. The only time its "wrong" is when I start using those processes again, which takes all of 1-2 seconds.

Its a hard problem, and both OSX/Windows choose the best possible solution you can heuristically.

> The logic is for things like daemons or infrequently used processes that wake up maybe once a day. They don't need to be in ram all the time needlessly. So swapping out their memory that hasn't been used for N hours isn't a bad thing.

It is, actually. I find such things unacceptable, be they on desktop or server use cases. I put as much or far more RAM in my systems than they will need, and I expect nothing to be swapped until it's actually full. Many other people do as well, which is why the Linux kernel devs finally started fixing the stupidity several years ago. Time for OS X to catch up.

So why not lazy page? Swap to disk, but don't unlink the memory until it's needed by another process. If the paged app comes back to like first, it's still in memory and the paged copy on disk can be dropped.

I was under the impression(from my FreeBSD years) that pages that are not used for a while is swapped out and marked as inactive. The pages are then fast to throw away for other use, or fast to brink back to active state.

In Lion I get the impression they are just swapped out and thrown away / reused for something else despite there are no real pressure on the VM.

> From what I've read, Windows memory manager does the same thing - after a while, it swaps out unused pages, even if plenty of free memory is available.

Windows XP does that. It was a common source of grief. I remember it being mentioned as early as 2004. Since Windows Vista the memory manager doesn't have that problem.

I'm running on a 160GB HDD (nearly full), 2GB ram, on a 4 year old MacBook. I always have iTunes, Xcode, Safari/Chrome running with multiple tabs, and a mail client (Sparrow). The system originally ran Leopard and I upgraded to SL and then Lion. I wouldn't say it's slow. It's definitely getting slower but that's expected since it's such an old system. Could it be that you're just expecting too much from your computer?

"It's definitely getting slower but that's expected since it's such an old system."

I see this sentiment a lot, but I disagree with it. What are "iTunes, Xcode, Safari/Chrome, and a mail client" doing now that they weren't doing four years ago? Is it enough to justify their latest versions feeling less responsive than their versions from four years ago?

I think that as the developers are developing them on better hardware they pay less attention to performance. For example you wouldn't expect the lastest build of iTunes, or OS X to run on hardware for 10 years ago. As system specs improve developers seem to pay less attention to performance & file size. It seems crazy to me that modern web browsers are over 50mb in size. But as hard drive size isn't really a constraint anymore the developers don't optimize that aspect as much.

I would, actually. All iTunes is is an MP3 player, remember, and WinAmp did just fine on 90s machines.

Safari/Chrome: websites are more complex, more image-heavy, and more JavaScript-heavy

iTunes: layers of software for dealing with Wi-Fi sync, Ping social network, iTunes Match, and so on

Mail Client: totally agree with you there...

But I still pretty much agree with you overall...

I don't. Is that iTunes library really the same size it was when the machine was bought? Does it play songs with the same encoding size? Is the version of XCode still the same with the same feature set? Does the browser play HD video from the web as often now as it did years ago?

Things change. My old 2006 iMac core 2 duo feels a bit clunky sometimes these days, but it runs a lot of stuff fine and is actually just as good a machine as it ever was.

>I see this sentiment a lot, but I disagree with it. What are "iTunes, Xcode, Safari/Chrome, and a mail client" doing now that they weren't doing four years ago?

Lot's of things. XCode was rewritten and does live AST syntax completion, background compiles, etc.

Safari/Chrome have several more features --did Chrome even exist 4 years ago?

"nearly full"

Expectations are certainly a big part of the perceptual speed equation. But with OS X, don't underestimate the benefits of keeping your disk less than 90% full. With all the caches, iPhone and iPad backups (over 40 GB in my case), Xcode, sleepimages and swapfiles, installers (Adobe!), SyncServices, etc., a 160GB SSD fills up in no time. When things get slow, getting back below 90% works wonders.

Insane amounts? Like 16G? That I can buy for like $80?

16GB is an insane amount, regardless of the price.

Make that an extra $750 if you buy it built in to a new MacPro...

Also, because all this swapping the life of the SSD on Mac OS Lion are shorter than running on Linux or Windows.

My issue with Lion was that it broke WiFi on my 2009 iMac. Yet the 2011 iMac is just fine with it. There has been a near constant running thread on the Apple support since the day Lion dropped. All sorts of WiFi just going out, sometimes even with the icon being nice enough to gray out.

Currently at 145 pages and growing https://discussions.apple.com/thread/3191630?start=2160&...

Yeah, the 2009 iMac started working the day I put Snow Leopard back on it. I then sold it and warned the owner that upgrading to Lion was at his own risk. I replaced my router with an Airport Extreme; useful excuse to buy a new toy; at one point to see if that resolved it. I even moved the iMac NEXT to the router one day.

My late 2006 MBP had all sorts of wifi issues with 10.5 the first year or so it was out. Like routine kernel panic-bad level stuff.

AFAIK every odd-numbered version release of OS X breaks the networking on a significant number of machines.

Two things I have found to help the most to improve everyday usage on OSX Lion without an SSD: 1. Disable Spotlight on the boot volume. This keeps the mds process from reindexing things so frequently. 2. Use Time Machine Scheduler (free utility) to backup at 5am or something so it's not interfering with daytime tasks.

Not this again, we already went through this a few weeks ago.

In-fricken-deed...

We have been through this problem again and again and again, in different OSes, at different times and with different things triggering the various problems.

It usually ends with a "neck-beard" saying with enough authority "look, really, they are doing it right even if it seems totally illogical to you and any brokenness is just your configuration, little man". Which is to say "you might not like senseless disk-thrashing but would you rather have your machine randomly freeze when it got out of memory?" And scratching a little more, it comes down to admitting that memory-allocation is a hard problem in its full generality and they don't teach you that in application-programmer-school, and further that the solutions to it that any of these OSes have are tuned-black-magic-split-the-difference-haphazard affairs.

Consider. Either the machine keeps all your information in memory it or keeps on-disk and in-memory and either way, the machine hasn't a clue what information is important to you, my friend. It's just data to it. It's not like the computer is intelligent or anything. Why do you think they call it "random-access-memory"? The problem of dividing up chunks of memory for application programs to use is as hard as dividing up that hard disk for large and small files to live in, EXCEPT that application programs expect to be handed a chunk of contiguous memory when they call malloc. Hard problem even with the powerful tools that have evolved for solving it over the years. So when a given memory management scheme works, it isn't really "fixed", it just has been tuned for the corner-cases that are shouting loudest on the help lines.

And yes indeed, it is "funny" how just getting the "simple stuff" to work is a hard problem. IE you can find lots of simple examples where the standard solution seems fail terribly.

Angry how your 100 GB memory machine isn't faster? Look under the hood and you'll find Scotty from Star Trek shouting "Captain, I'm allocating your memory as fast as I can Sir..."

If you're not oversubscribed, it shouldn't thrash. Isn't that the complaint? Then its a legitimate issue.

I don't recognize any of his symptoms anyway, and my OS X computers get pretty RAM-heavy use, with almost always a linux VM open, XCode, Safari with ~10 tabs, iTunes with a few thousand songs, etc.

I think you missed a key ingredient to this problem, which is heavy disk reads caused by either spotlight or time machine.

Reading a massive amount of data (that you'll probably not use again anytime soon) has the unfortunate side effect of polluting the disk cache with junk. Now if OSX is anything like Linux in this regard, it is loathe to toss out old disk cache (in response to all the incoming junk it's being asked to cache) and will instead start swapping to free up more memory for disk cache.

Linux has /proc/sys/vm/swappiness to control how aggressive it will be in swapping stuff out to preserve precious buffer cache, but I don't think OSX has any such mechanism.

Isn't this what POSIX_FADV_DONTNEED is for?

http://linux.die.net/man/2/posix_fadvise

>I don't recognize any of his symptoms anyway, and my OS X computers get pretty RAM-heavy use, with almost always a linux VM open, XCode, Safari with ~10 tabs, iTunes with a few thousand songs, etc.

Oh boy I wish I could say the same. Admittedly I don't shut down on a daily basis but this didn't used to be a problem in SL. FWIW I've an old tank of a tower that has a video card on its last leg.. shutting down invariably leads to ~30 minutes of downtime while the card heats up and reconnects whatever needs reconnecting for both monitors to work.

Since Lion, I've noticed frequent hangs and beach-balls when doing even menial tasks. Transmit, terminal, texmate, a few tabs in chrome. If time machine starts backing up I can forget about a smooth Preview open or switching to a largish open textmate file without beach-ball'ing. If I want to use a Win7 Parallels VM--I can't do anything else. Even now as I type this I have a ubuntu vm running at the login screent and it causes the machine to shake off the cobwebs between almost everything.

It's certainly not a bad machine--there's 8gb ram, and tons of diskspace--good processor. In fact, I would go multiple months without a reboot and heavy use when on SL without hardly any problems at all.

Then there is the new i5 MBPro. Cool trick you can do: hook up an external monitor via thunderbolt and watch as the [left side] dock becomes a mangled mess with icons miss-positioned and wrongly triggering apps--it's like playing a game of whack-a-mole trying to open terminal to kill -KILL dock :)

The i5 has also been less than stellar compared to the older MBPro I sold to buy it in terms of performance.

For what it's worth, I'm running Snow Leopard (10.6.8) and I see the same sort of terrible performance the article describes.

Though I did improve matters drastically by telling mds not to index my (Linux) MP3 server and my Time Machine drive. That made it go from nearly unusable to just frequently annoying.

The other crazy thing I've been seeing is it routinely takes Chrome minutes to shutdown -- in fact, pretty much every time I try to reboot my MBP without shutting down Chrome first, the shutdown process times out trying to exit Chrome.

Ah yes. The dreaded Chrome renderer never enderer.

I too have experienced the Chrome issue enough that I don't even try to close it normally anymore. Force quit is the only way I exit Chrome. Thankfully, the restore tabs functionality works well.

I'm glad to see I'm not the only one. This drives me crazy every time.

Firefox 12 does this too, on both my MB Pro and iMac. I think it has to flush so many caches that disk I/O just takes forever. This is purely conjecture, though.

Not sure if it will help, but have you tried running Firefox's hidden profile manager and created a new profile? Firefox was horrible to run on OS X for me, it would drag the whole system down, but since changing to a new profile it has run like a dream and not reverted back.

Creating a new FF profile made a significant difference for me, too, when I did it recently (for the first time in a couple years). I suspect that will be true on any OS, if you do a fair amount of installing and removing extensions, user scripts, etc.

"Vacuuming" your Firefox profile's .sqlite files also helps.

I had to shut down on a daily basis because of this problem with my system having only 4 GB of ram. There is definitely something broken in there.

http://i.imgur.com/ohBEF.png

I don't see anything wrong in this picture. You still have free memory. Even if it didn't show free memory you would probably still be OK, because it would just be reserved for use

In that case it should be marked as inactive memory so it can be reused, that memory that was taken up by the end of the day would never be reclaimed so I was paging a lot.

>In that case it should be marked as inactive memory so it can be reused

That would be the job of the memory manager to decide when to do that. Memory could be kept in the non-free state for longer that it actually is needed, but still be marked internally to be available when needed.

OTOH, if you have paging, as you say, then something is wrong, true.

But I don't think that the screenshot shows something wrong.

And not to change the subject, but why does my instance of Dashboard need 340M of RAM? I have maybe 5-6 widgets. Is this thing spawning WebKit for each one?

>we already went through this a few weeks ago. . . . Back then, I thought the conclusion was that there is nothing broken about OS X memory management

I read through the comment of a few weeks ago, and I did not see anything conclusive or even anything that would outweigh my subjective impressions that something about Lion on my (stock) 2011 Mac mini is causing unnecessary lack of responsiveness.

You might be uncomfortable with subjective impressions and many pieces of weak evidence, but given the popularity of OS X on this site, not all of us want to wait for what you refer to as "hard facts" before engaging in a discussion of the issue.

Do you have a link? I tried to search hn but, alas, in vain.

Yes, I had it bookmarked. Actually, I had bookmarked a particular subtree of the comments tree, but repeated following of the "parent" link should get you the whole tree. The particular subtree is at http://news.ycombinator.com/item?id=3585181

I see the same issues the guy in the article has seen. Basically, I have a total of 8GB of ram in this here mac mini, and when my desktop session gets heavy things start going south.

For example: Let's take a large Chrome session (~150 tabs spread over several windows), an IDE open somewhere, Spotify, Steam and some background apps, and a small Windows VM. Generally, Activity Monitor would show that Chrome in this instance would be eating 2-3GB of RAM, the VM would be eating 1GB + change of unpageable ('wired') memory, the random utils & spotify & steam & ide du jour & crap would eat another 1.5GB or so, and, long story short, there's very little 'free' memory left (think <100MB) on this 8GB system but a good >1GB of 'Inactive' memory.

Everyone agrees that Inactive memory should be freed when more memory is required by the system and we're out of Free. My own limited testing shows instead that opening more tabs during normal use to the point where the Free memory is consumed instead results in massive delays across the system as memory is paged out to disk, and the Inactive memory doesn't seem to noticeably change in size.

I don't really know what it's doing, but it will consistently make the system very unpleasant to use for a good 30 seconds until the hard disk stops clicking. (It's unpleasant enough that I try to limit my browser session sizes now, and only run the VM when I need to.)

What you are describing sounds like plain old memory starvation, not really a problem with the memory management.

You don't have to fill up every last bit of RAM before the OS starts swapping, as there could be pinned memory pages or processes that want to do larger allocations that are only held for a short time. If you have less than a few hundred MB of unused RAM with all the stuff you mentioned going on, it only takes some kind of scheduled OS background job to push the OS over the line where it decides it needs to swap in/out.

That said, from the comments some people posted here, it does appear that at least in some situations there seems to be something going on in some versions of OS X Lion. If Snow Leopard and the Mountain Lion preview are unaffected with the exact same usage pattern, maybe there actually is some kind of bug in the OS X memory management. But I'd still like to see some kind of evidence, facts or statistics, as I have never experienced anything like it myself, not even on my MacBook when it still had only 2 GB of RAM.

>I see the same issues the guy in the article has seen. Basically, I have a total of 8GB of ram in this here mac mini, and when my desktop session gets heavy things start going south. For example: Let's take a large Chrome session (~150 tabs spread over several windows), an IDE open somewhere, Spotify, Steam and some background apps, and a small Windows VM.

150 tabs? A VM? An IDE?

That's just A LOT. Of course things will go south, what did you expect a magic machine that can run everything and whistle away hapilly with 0% load?

I bought a brand new Macbook Pro, 15" with upgraded CPU, etc. with 4GB of RAM last year (2011 model). I didn't want to upgrade to a SSD, I wanted to do that later. After I upgraded to Lion (through the Mac Store) the machine is as speedy as my old Macbook Pro I bought in 2006 and is running Snow Leopard. It's horrible and it's so annoying and I'm pretty much really angry. If Apple will not fix this I will NEVER buy a machine from them again. The only reason I'm buying this overpriced hardware is because I like working on OSX and I don't want to struggle setting up a "Hackingtosh". You probably won't define this as a bug because you're not experiencing it yourself. Maybe it's not an implementation bug but it surely is a bug from the user's perspective.

edit: Just for the record. I'm not doing some kind of heavy processing work. Most of the time I've got one Chrome window open with some tabs, email client, macvim and iTerm2. It's not like I'm doing some heavy work. I'm not even running a VM.

Use something like fs_usage to see which of your programs is churning the disk. This is not the normal state of Lion; there are two system services (Spotlight and Time Machine) which can generate tons of background I/O through normal operation - I usually disable TM outright and make sure Spotlight ignores my development folder.

I'll check that out. Thank you for your suggestion. One thing I find strange though, this started happening after I upgraded to Lion. It wasn't like that when I was using Snow Leopard and both Time Machine and Spotlight exist in Snow Leopard.

It's hard to tell for sure, but it seems like this problem may only be present if you're running certain kinds of applications. For example - if it's related to garbage collection, anything that uses ARC instead of GC isn't going to be affected.

I thought it was pretty clear that this isn't a "fix", but there's definitely something wrong here.

MacBook Pro, 8GB, leave it running for weeks: No performance problems at all.

People say "Get and SSD", well, I've had 2 SSDs and 4 SSD failures (one drive failed three times, the other once and its replacement is still going.)

So, I'm all spinning rust here. 1.5 Terabytes of rust in my Macbook Pro and the only time I have a beach ball is trying to launch Team Fortress (but I blame valve for that).

I have massive, MASSIVE Final Cut and Aperture libraries. I leave the machine up for weeks. I leave Time Machine running all the time- there isn't even a slowdown when time machine is backing up.

My hard drives are encrypted with full disk encryption which means not only am I running spinning rust but its encrypted rust which means every read has to be decrypted.

No slowdowns or beach balls. Sure the occasional poorly written program will have a beach ball, and rendering video takes awhile, but that's to be expected.

Yet people constantly say that Lion sucks? Really? And they have these more beefy machines with more RAM?

Something doesn't add up here.

3 year old (non 'pro') macbook here, 5GB & 7200rpm drive, avg uptime is 30+days or whatever the time between service packs is. No issues at all.

Came with Leopard, upgraded to Snow Leopard (not a fresh install) and them app store upgrade to Lion

machine is snappy as, unlike my coworker who has a brand new quad core i7 w/ 256GB SSD that runs Lion like a dog. no idea why, but my humble old macbook is faster in every way than his shiny new mac mini

Reply on news.ycombinator.com