I'm running 12.04 with I7 Ivy Bridge HD4000 too and have no performance issues. Maybe it's op's hard drive? I'm running the system on an SSD and it's a night and day difference to the rotating equivalents.
I find it is just quality of life. If I use 4gigs, I can only have 2 real memory intensive applications open at the same time. If I had 8+, I could easily have anything I want open in normal usage and not break a sweat.
It sounds like you might be using a 32 bit version, which will not utilize more than 4GB of RAM. 64 bit versions have the ability to address memory beyond the 4GB limit of 32 bit systems.
With 32 bit versions of Linux the kernel is mapped into the upper 1GB of address space, and then whatever is left over from the remaining 4GB is exposed for general purpose use, and made visible to the user. The means that by default, even if you have 4GB of RAM, you'll never actually see anything more than 3GB exposed for your use, because the kernel has already reserved 1GB of RAM for itself. (these ratios change proportionately, when less than 4GB of RAM is present)
It's also possible that your system is hitting the 3GB mark and then paging virtual memory to your swap partition.
It honestly feels like there is some kind of problem with the system swapping (or failing to swap some memory) or some memory leak that I can't quite track down. Maybe something got fixed somewhere in the last two years...
It's probably just that Lubuntu is my only reference point - it's so lightweight plain Ubuntu feels laggy in comparison. Doesn't make me any less happy with the increase in responsiveness though :)
You mean, not like this? http://i298.photobucket.com/albums/mm249/hrenistic/ubuntu_13...
Edit: 10GB RAM and SSD!