Hacker News Reader

Comment by dandinu | Hacker News Reader

That's pretty accurate. I'm always amazed how much we move forward with technology, just to later realize we already had it 15 years ago.

regarding your question:

I have a 32bit XP version as well, and I actually started with that one.

The problem I was facing was that it's naturally limited to 4GB RAM, out of which only 3.1GB are usable (I wanted to run some beefier models and 64bit does not have the RAM limit).

Also, the 32bit OS kept freezing at random times, which was a very authentic Windows XP experience, now that I think about it. :)

View on HN

> out of which only 3.1GB are usable

That would be a real issue. I vaguely recall methods to work around this - various mappings, some Intel extension for high memory addressing, etc: https://learn.microsoft.com/en-us/windows/win32/memory/addre...

Maybe unrealistic :( I doubt this is drop-in code.

So the deal with AWE (Address Windowing Extensions) is that it lets 32-bit apps access memory above 4GB by essentially doing manual page mapping. You allocate physical pages, then map/unmap them into your 32-bit address space as needed. It's like having a tiny window you keep sliding around over a bigger picture.

The problemis that llama.cpp would need to be substantially rewritten to use it. We're talking:

  cAllocateUserPhysicalPages()
  MapUserPhysicalPages()
  // do your tensor ops on this chunk
  UnmapUserPhysicalPages()
  // slide the window, repeat

You'd basically be implementing your own memory manager that swaps chunks of the model weights in and out of your addressable space. It's not impossible, but it's a pretty gnarly undertaking for what amounts to "running AI on a museum piece."

Reply on news.ycombinator.com