Use OpenGL to get more RAM | Hacker News Reader

Use OpenGL to get more RAM

github.com

149 pointsdevbug55 comments

I worked with a programmer who cut his teeth on ancient iron in the 1950s. He had a computer that had 4K of ram and two 512 byte I/O buffers. He would set the loader for the program into the I/O buffer and then use that to load and initialize the rest of the program. The I/O buffers would go away on the first read/write. Every byte counted. I used the same techniques when I wrote MASM in the 1990s to create a 16k I/O buffer in the CGA graphics card. The graphics card memory was slower than regular memory, but the 16K made the program fly on reads and writes. I remember a utility that used the Hercules Video card memory for EMS (paged) memory. I think it came with the card.

While the readme does state this (could be) a joke, this unfortunately isn't possible with PCI-e today:

Motherboards only support directly accessing a (up to) 256 MB segment of VRAM directly from the CPU. This is the BAR1 space/aperture space.

So attempts to create allocations larger than that that are resident in VRAM but also accessible from the CPU will land in system memory, in order to ensure they are accessible from the CPU. Graphics drivers will either have the GPU read the data from system memory, or will do hidden copies to the GPU when they detect the resource is bound, etc.

The author sort of suspects this could be happening:

"There is no guarantee that the persistently mapped buffer technique actually references video memory. The worst case it's shadow memory and this actually wastes memory."

you could still use vram for paging

Why do motherboards have this limitation? There's nothing inherent in PCIe that restricts devices to such a small range.

Is this a 32-bit BAR / low 4GB space issue?

I'm not sure, BARs can be 64 bit and there is 6 of them.

This sounds like a little bit of confusion. Its the PCIe aperture your referring to, and its size/location is dependent on the machine/bios in use as much as the requested BAR sizes. A lot of recent xeon and workstation level machines actually again support 64-bit PCI aperture windows. See the discussion about which machines are compatible with the xeon phi which requires 8GB (or more) of MMIO space.

https://www.pugetsystems.com/labs/articles/Will-your-motherb...

Bottom line, most desktop machines only use a legacy 32-bit PCIe window below 4GB. That is why its limited to a fairly small amount of address space (256-1G generally). The ugly problem is that without purchasing a product its generally quite difficult to determine if it supports a 64-bit aperture correctly. Take the Asus x99A board I have, no mention of support one way or the other, but with an i7 (not just a xeon) it actually works with the xeon phi.

(BTW: Nvidia did a better job with this, their tesla cards have a GPU compatibility mode where they restrict the BAR to 256MB for machines that don't support 8GB BARs).

Around 2002 I had bought a machine with a duron cpu, which required a different kind of memory. I still had lots of memory for my old machine, something like 768MB cobbled together out of 64-256MB DIMMs.

So I created a ramdisk on the old machine, exposed it as a nbd (network block device), mounted it on over the network (one hop over a 100 Mbit switch, didn't have Gbit kit) on the new machine and created a swapfile on it. I couldn't believe how well it worked. I never actually had any flakiness issues and I used it like that for probably a year.

A new implementation but not a new concept:

https://wiki.archlinux.org/index.php/swap_on_video_ram

http://ft23.pmenier.net/docext/mtd/TIP_Use_memory_on_video_c...

This may be the very first occurrence of the idea on Linux:

http://web.archive.org/web/20081227185259/http://hedera.linu...

Hey, I remember using it back in 2005, when extra 28 MB made a big difference. :-)

It was surprising slow, though still faster (and quieter!) than HDD.

It used to be standard to use your VRAM as RAM where possible. If you were not coding a game but something like a business tool, you would just stuff parts of the application in VRAM so you could swap it out from there instead of from HD or, before that, floppy. This was without OS support mostly though. For work I haven't needed this technique since the end of 90s I think.

"Used to be standard" is highly debatable. The only people I ever heard of doing that were the crazy sort, enthusiasts pushing the limits of their rig, not average people.

That depends how far back you go. You are talking PC probably; most 'desktop' computers before that had so little memory this was common place and even mostly impossible to do 'useful' business applications without it.

Citation needed because honestly, I've never heard of this and I think you're confusing it with something else. I've been using computers since the introduction of the original IBM PC and have had to spend a lot of time dealing with memory issues over the years.

"Extended memory" drivers were a common thing back then but that was to circumvent the 640KB/1MB limit of DOS and they had nothing to do with video card memory. In that era few people even knew how much memory was on their video card because it was rarely a concern, most of the time it was just barely enough to hold a single frame buffer, typical of most VGA and "Super-VGA" cards.

Extended memory is that which is beyond the boundaries of the original IBM PC design: https://en.wikipedia.org/wiki/Extended_memory

It's not video card memory in any way. If I recall correctly the video card memory was often mapped into the 640KB-1024KB memory zone, but using that for system memory was futile since you'd be unable to see anything on screen.

I mentioned before PC in my comment to your comment right? You are still going on about PCs; I was not referring to PCs but rather MSX, CPC, Amiga etc. They had distinct RAM & VRAM (in the case of the MSX the cool machines had 128kb-256kb RAM & 128kb VRAM and you could not map one to the other => it was really different memory) and when you used text input screens you were wasting 112kb of memory if you didn't do this. The memory was slower to access from the CPU but it was much much faster than disks.

Not confused, just not talking about PCs in general (although I have heard cases of this happening there but that might be just indeed demo groups); I still do this on those old computers today for hobby purposes (the last work I did was somewhere end 90s on a legacy system which counted on video memory and which had to be ported so I had to remove & test that 'feature') ; there just isn't another way to get memory if you don't want to cheat (and solder in 1024kb).

Edit: I don't know if it was used in practice because there were so many different VGA cards, but [1] shows that it was common to have 256kb on a card which, depending on the era, was significant. And like said above, for business applications in text based environments (screen 0) you can use a large portion of video memory for storage while you still see something. But I can see the issue there; with other computers we didn't have that issue as you could trust a certain amount to be there for sure.

[1] http://www.osdever.net/FreeVGA/vga/vgamem.htm#detect

Hogging tabs in long-running Firefox instances has taught me that Windows (even 64 bit versions) still cannot address more than 3 GB of memory to 32 bit applications. It used to be that was because 1 GB was reserved for the GPU, among other things. I suspect (but do not know for sure) that this is still the case.

Originally, the top 2GB of the 32-bit address-space was reserved for the kernel, and the bottom 2GB available for each running program. This worked fairly well until programs started legitimately using 2GB of RAM, so Microsoft came up with a new scheme where only the top 1GB of address-space was reserved for the kernel, and the bottom 3GB was available for running programs.

Unfortunately, they couldn't turn it on universally; there were a lot of programs that "knew" that malloc() would never return a pointer with the top bit set, so they used it as a flag for their own purposes. Such programs would break horribly in 3GB mode when pointers could have the top bit set. Also, restricting kernel-space meant the kernel had less room to do stuff; if you were running an app that was heavy on kernel-resources but light on RAM usage, 3GB mode would be a downgrade[1].

So applications have to opt-in to 3GB mode by setting a special flag in the executable, and 32-bit versions of Windows had to be put into 3GB mode at startup to support it at all.

Of course, the real solution to "3GB mode at startup" is just to run a 64-bit OS, and the real solution to "2GB is not enough address-space" is to run 64-bit applications.

[1]: https://blogs.technet.microsoft.com/askperf/2007/03/23/memor...

> Unfortunately, they couldn't turn it on universally; there were a lot of programs that "knew" that malloc() would never return a pointer with the top bit set, so they used it as a flag for their own purposes. Such programs would break horribly in 3GB mode when pointers could have the top bit set.

Fascinating story. Never knew this. This dedication to backwards-compatibility is something Microsoft was known for. Reminds me of how they took great care when writing Windows to ensure perfect compatibility with old, horrible DOS programs, going so far as re-creating DOS bugs that these broken programs relied on. I suppose they were just accepting reality: if a user upgraded to Windows and some third party program broke, the user would blame Microsoft, not the incompetent application developer.

Both Windows and Linux are distinguished for their pathological conviction to never break userspace. Linux has its warts because of this, but since it's 10 years younger than DOS and since, as a Unix kernel, it was always very clear what the kernel should and should not do, it doesn't have quite the trail of hacks seen in Windows.

Windows has broken legacy programs though. In Windows 7 a lot of old programs stopped working in the consumer editions of the OS. I don't know if they've continued making their "professional" editions that provide comparability. I suppose that's a reasonable strategy to encourage developers to update their software without killing user productivity, which could workout for the best in the long run.

Linux the kernel yes. but Linux user space is notoriously against dealing with backwards compatibility.

As for Windows, best i can tell the problem was Win16 on 64-bit Windows. My father ran into that regarding some old solitaire programs he had been keeping around.

Nit: There's no such thing as the "Linux userspace". The GNU/Linux userspace is a collection of many different pieces of software (including the GNU tooling and all of the FreeDesktop stuff).

Yes, but outside of the kernel, and maybe the GNU tools, there seems to be very little willingness to entertain the need for backwards compatibility.

My issue was with the term "Linux userspace". I agree with the sentiment that many components of a modern GNU/Linux system don't seem to like the idea of backwards compatibility.

Or you could have separate virtual address spaces for kernel / userland.

You could, but:

"Just for comparison, Linux defaults to 3 GB for user mode and 1 GB for kernel mode. There are experimental patches to implement a separate virtual address space for the kernel, thus granting 4 GB for user mode AND 4 GB for kernel mode. Unfortunately, there is a 10-20% performance hit because every system call requires an address space switch (and TLB cache flush)."

Take it with a grain of salt, that quote is a decade old.

I wonder if this is something that is limited to a 32-bit kernel. Solaris gives each process the full 4 GB in userspace, and I think that a 10-20% performance degradation compared to Linux would be noticeable.

It is slower. Here's an old benchmark but presumably Linux and Solaris had the same shared vs. separate address space difference back then, and the results show it:

https://pdfs.semanticscholar.org/4489/18f01d17628cb94ac69f27...

They don't call it "Slowlaris" for nothing...

Its reserved for the kernel, not the GPU specifically. Normally the top half of the 32 bit address space was reserved for kernel use, and the bottom half for application use, but there is a flag in the PE header that tells windows to only reserve 1 GB for the kernel (and 3 GB for the application)

You can adjust the boot options on 32 bit windows to reserve 1GB for the kernel, or any number between 1GB and 2GB. It's irrelevant for 64 bit windows.

The flag in the PE header says that the application can handle addresses over 0x80000000. It don't change reservations.

On 64 bit windows, they get 2GB by default but can opt in to 4GB. Nothing is capped at 3GB.

32bit executables can use 4GB address space on windows if they have the LARGEADDRESSAWARE flag set, which has been the case for firefox.exe since 2010[1].

It may appear that they use less than 4GB if you look at the working set or private bytes figures instead of virtual size because they reserve more address space than they allocate. Or it might crash before reaching that limit due to large allocations failing.

But 64bit builds have been available for years on their ftp server, they just weren't supported. But since last december they are[2], so there really shouldn't be any need to run 32bit builds on a 64bit system.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=556382#c34 [2] https://blog.mozilla.org/futurereleases/2015/12/15/firefox-6...

> there really shouldn't be any need to run 32bit builds on a 64bit system

Maybe the 64-bit version doesn't support the Adobe Flash plugin?

The another problem with 64-bit versions is that all the pointers are twice as big. So if you have only 8 GB of physical ram, you won't necessarily be able to handle much more than with a 32-bit application. But with 16 GB or more the 64-bit version must be better.

> Maybe the 64-bit version doesn't support the Adobe Flash plugin?

64bit flash works for me.

> The another problem with 64-bit versions is that all the pointers are twice as big.

You're assuming pointers dominate the footprint. if you're seeing crashes due to address space exhaustion that's more likely due to large strings/blobs of data and not a huge amount of pointers.

Or to put it differently, do you really think OOM crashes are preferable to some background applications or memory-mapped files being paged out?

> You're assuming pointers dominate the footprint.

No, I just wrote:

>> if you have only 8 GB of physical ram, you won't necessarily be able to handle much more than with a 32-bit application

That means, I absolutely know applications that won't be better on a 8 GB RAM computer as 64-bit applications, and they were written using standard C++ libraries. Typically, such applications weren't optimized for cases when they use a lot of memory. If somebody competent spends enough time and energy, even such application can be made to use memory better but it's not something that "just happens."

And sometimes 32-bits application is a good trade-off. See for example

https://blogs.msdn.microsoft.com/ricom/2009/06/10/visual-stu...

"A 64 bit address space for the process isn’t going to help you with page faults except in maybe indirect ways, and it will definitely hurt you in direct ways because your data is bigger."

What I'm trying to suggest is that the topic is more nuanced as "64 doublegood 32."

I don't think all pointers are twice as big. How does Java do compressed OOPS?

Java has a couple of different options for dealing with objects but is able to manage the full set of pointers and know that (for example) code doesn't try to access memory+1 for looping etc.

Under 4G worth of size pointers are simply zero extended, so that the higher order bits are just zero and the lower order bits stores in 32 bits.

Above 4G but below 32G it switches to "compressed oops" mode. This right-shifts the address by 3 bits and ensures/mandates that all objects are aligned to 8 bits (2^3). When converting back again to a memory address it multiplies by 8. This effectively multiplies the top address space by a factor of 8 and so you can store an object in the same 32 bits but can be wider.

It's possible to shift the alignment (-XX:ObjectAlignmentInBytes=16) which will then right shift the address by 4 instead giving you 64Gb memory with 32 bit addressing.

Although it's possible to go higher the advantages tend to scale out; effectively you end up padding the object space more and more so simple objects take up larger and larger slots.

The bottom line is: by controlling where pointers are used, Java can use a different addressing scheme by unpacking and repacking object pointers on the fly. And for some operations (equality testing) it doesn't even need to unpack the pointers in the first place.

I knew Mozilla was working on it, and at some point abandoned the process, but I wasn't aware they had picked it up again. Thanks for the pointer!

VRAM is also where many "online" development suites—the BASICs and Pascals of the time—expected you to write exception traces. Rather than trying to "break into" a debugger (i.e. cram a debugger into the same address space as your software), you'd simply have your trap-handler persist the stack-trace to VRAM, switch your tape drive over to the devtools disk, and reboot. The "monitor" would load, notice that there's a stack-trace in VRAM, parse it, read the pages it mentions from your program (now on the slave tape) and display them.

Finally an answer to the age old question "where can I download more RAM?"

http://downloadmoreram.com/

Only, that it doesn't give you any extra memory at all. In case you're using a dedicated GPU, the peripheral bus inbetween (PCIe) is a serious bottleneck. In addition to that the OpenGL object model does not have the concept of automatic "object eviction"; on embedded architectures (Android) you may loose the OpenGL context and have to recreate it. But you'll never loose a single OpenGL object.

The bottom line is, that the OpenGL driver will create a backing store in system memory for each and every buffer object. So if you allocate 4GiB of OpenGL buffer objects, it will allocate 4GiB of system memory; if it's a shared memory GPU that's it, if it's a dedicated GPU the GPU RAM is actually more of a cache to the backingstore.

glMapBuffer will usually just give you access to this backingstore so that writes can be coalesced into a single transfer when unmapping. Also you don't want a full round trip read-modify-write. In case of coherent mappings the coalesced transfer is triggered by a GPU side data read operation on that buffer.

TL;DR: RAM on graphics cards is a cache on top of system memory (as far as OpenGL is concerned).

On desktop NV and desktop AMD I've observed (at least on Linux) via /proc/self/maps, that the pointer returned by glMapBuffer is a shared memory mapping owned by libGL. Further inspection shows coalesced reads and writes happening by the kernel through a DMA transfer operation, only when GL_MAP_COHERENT_BIT is used. The mapping is entierly virtual and is not touching physical memory until a R/W occurs. The driver's shared-memory explicitly flushes contents of the read or write request to device memory in the same way CUDA/OpenCL does. This can be observed by the ioctls the driver generates after page-faults occur in the kernel.

Once it's offloaded to the GPU, it could compress it on an LRU basis...

When out of FAST RAM, use CHIP RAM :)

Is there a reason operating systems don't/can't do this automatically? It sounds like a faster swap space than disk.

This only works when you have a dedicated graphics card (IGPs use system RAM) with dedicated VRAM (some dedicated notebook chips also use system RAM), which is also not needed for anything else (which is what you usually buy a graphics card / GPGPU acceleration card for).

That's a so negligible minority of devices (underutilized gaming rigs and nothing else) that OSes just don't bother.

If you consider that when you're on the desktop and not running a game, all those gigabytes of VRAM are basically going unused. So it should apply to any system with discrete GPU memory, which is surely quite a few systems? I guess those systems correlate with high-end systems with lots of system RAM anyway though...

> So it should apply to any system with discrete GPU memory, which is surely quite a few systems?

No smartphones, no tablets, only a small minority of notebooks (not a single macbook, e.g.), none of the business desktop PCs (which are by far the majority, volume-wise), no servers nor workstations (except those that have them installed for GPGPU purposes).

The exception are gaming laptops/PCs (and those will need the VRAM for actual gaming) and servers/workstations with dedicated GPGPU cards (which, again, would need the VRAM for GPGPU purposes).

That only leaves a few cases where you have beefy hardware and don't use it, which isn't significant enough for OS vendors to bother with. Thirdparty VRAM-as-RAM (or RAM disk) kernel modules / FUSE layers pop up from time to time (this isn't nearly the first), but they never gain any traction.

> I guess those systems correlate with high-end systems with lots of system RAM anyway though...

Usually VRAM is less than a quarter of the system RAM, or even less. And usually those systems have so much system RAM that they have enough spare to make RAM disks an a valid option.

Building on creshal's comment, ensuring direct access to it outside of gaming applications could be offered as an extra feature or selling point of those applications. First, it was just a graphics coprocessor. Second, they turned it into a more general coprocessor. Next, it's also extra RAM or cache for whatever we need enabled with flip of a switch or targeted by included library worst case.

This seems useful for AMD APUs where system RAM is reserved by the iGPU.

sounds to me like a perverse realisation of amd's apu ideal, treating VRAM like system RAM, although in this case they are still disparate memory spaces.

Isn't VRAM already used as system RAM in the AMD-based PS4?

PS4 has 8 GB of unified memory yes. The Xbox 360 had 512 GB of unified memory too.

Indeed. You could also use the PS3's VRAM as swap space when using OtherOS (Linux) before Sony removed OtherOS.

GPU malware is going to become really interesting once memory-coherent GPUs (like PS4's APU) become more available. That combined with something like packet mmap [1] may allow for data exfiltration from a GPU compute kernel without ongoing userspace assistance (MMIO packet send/recv => no need to have the CPU userspace side dispatch syscalls on behalf of the GPU).

[1]: https://www.kernel.org/doc/Documentation/networking/packet_m...

IMO Firefox need more CPU than RAM

Always need more ram!

Reply on news.ycombinator.com