Huge Pages Are a Good Idea

evanjones.ca

54 points·signa11··7 comments

I wish Linux didn't make managing huge pages such a pain in the ass. You have to "allocate" huge pages ahead of time by editing a file in /proc/sys before you can map the huge pages.

Why can't free memory automatically be converted to huge pages when I call mmap(HUGE_TLB)?

I wish Windows was different in that regard. On Windows, large pages are borderline unusable.

They need special security privilege SeLockMemoryPrivilege, since vista introduced UAC most processes don't have that privilege.

Windows is unwilling to page large pages to disk, easy to fail with out of memory errors. I don't think there's a good reason for that: it's much faster to load 2MB from disk on modern computer, then it was to load 4kb from disk on i386.

Does THP not work for you?

Judging from the problems my boss has had with them lately they are anything but "transparent".

Hah, I think they were pretty buggy when they were first enabled by default in RHEL/etc.. We used to turn it off in our prod fleets after running into some serious memory / perf issues with them. I think they're in a better state these days.

You can add "hugepages=2000" to the boot command line.

"Transparent huge pages" is (sort of) the feature you ask for. It is very restricted in what it will do because of early mistakes pegging CPUs scanning for opportunities.

I'm always sad to see articles like this when large pages are so impractical to use for games on Windows, from MSDN [1]

"The memory is always read/write and nonpageable (always resident in physical memory)."

To say nothing of the fact that you have to do a privilege change to even access the API.

[1] https://learn.microsoft.com/en-us/windows/win32/memory/large...

Reply on news.ycombinator.com