Interesting. I am using OS X 10.5.8 with only 2G memory and when programming with Python, PHP, gcc and Apache this is actually OK. Not brilliant, but not slow either. The machine tends to be on all day. I
have noticed that Linux does seem to page more than it used to in recent years, this varies with the kernel. There
is more Linux disk activity these days than in previous versions, regardless of what swappiness I tell it to use.
I use nano or vim as my editor mainly. The setup I use on Linux is much the same. I don't notice much of a difference in terms of performance between OS X or Linux for the text-based and command-line related stuff that I do. I could sit down and be happy in either OS X or Linux and it wouldn't matter to me which one I'm using. I also use mutt for my email and cmus for playing music in both environments so I am seldom out of the command line for anything.
Although I do find apt-get much more efficient and I can get packages faster and with less fuss with Linux.
I do also use brew on OS X.
As far as I'm concerned, for what I do, there isn't a lot of difference ... I hardly ever use XCode, though.