Hacker News Reader

Comment by traverseda | Hacker News Reader

>Disable swap before building the index. Linux will aggressively try to cache the index being constructed to the point of swapping out parts of the JVM heap, which is obviously counterproductive. In my test, building with swap enabled was almost twice as slow as with it off.

This is an indication to me that something has gone very wrong in your code base.

View on HN

> This is an indication to me that something has gone very wrong in your code base.

I'm not sure on what planet all of these people here live that they have success with Linux swap. It's been broken for me forever and the first thing I do is disable it everywhere.

Linux swap has been fixed on Chromebooks for years thanks to MGLRU. It's upstream now and you can try it with a recent enough kernel with

  echo y >/sys/kernel/mm/lru_gen/enabled

On a planet where the heap size of the JVM is properly set.

I don't use Java often, but when I do, I have to look up again how the heap settings work.

It’s just -xXxXxX360NoScopexXxXxHeapSize=69G

There should be at least one : in there.

(IIRC these options are used like -XX:foo=bar)

(Edit - no, but some do use : instead of = I guess)

100% agree

[article author]

TBH this was sloppy on my part. I tested multiple runs of the index build and early on kswapd was super busy. I assumed Linux was just caching recently read parts of the source dataset, but it's also possible it was something external to the index build since it's my daily driver machine. After I turned off swap I had no issues and didn't look into it harder.

The usual thing would be to fadvise(POSIX_FADV_DONTNEED) the relevant file handle you don't want cached.

Edit: see for instance https://insights.oetiker.ch/linux/fadvise.html

As a workaround, you can mlock your process which should prevent the application pages from being evicted by swap.

FWIW this is what Cassandra does on startup, but it didn't seem worth it to go to the trouble of dealing with Unsafe for a demo project.

Can't mlock be wrapped out in a safe API?

Probably, but I'm not aware of any Java library that provides that functionality.

If you don't mind using a preview feature you should be able to use the foreign function API to call mlockall without any unsafe code.

Otherwise, JNA is probably the easiest way, and how Cassandra does it.

https://docs.oracle.com/en/java/javase/21/core/calling-c-lib...

https://github.com/java-native-access/jna

Linux swap has some fuzzy logic that I never fully understood. There have been times I've disabled it because it wasn't doing what I wanted.

I almost always reduce swappiness on a new install, the default of (60?) never served me well.

Yea this was strange, I've only ever seen swaps when my main memory was near being full. Maybe they're storing all embeddings in memory?

I routinely see Linux page memory out to swap while having 10+GB free. I can only guess that it really really really likes to cache recently used data from disk.

I've seen the same. It will sometimes prefer to swap rather than evict the disk cache.

I don't know how Linux does this in particular, but intuitively swapping can make sense if part of your allocated RAM isn't being accessed often and the disk is. The kernel isn't going to know for sure of course, and seems in my case it guessed wrong.

At a very abstract level there isn't much difference between dropping a disk cache and writing anonymous memory to swap. Obviously the actual step of writing to swap is more expensive but once the data is out of memory they will both require a read to get the data back. So if you imagine that you are occasionally reading a file and an anonymous page you can thrash on either if your system doesn't have enough memory. After the initial write to swap it is effectively identical to keep reading either back in.

It is much better to swap out anonymous memory that isn't being used than to flush file data that is being used. Or another way of looking at it if you run out of memory you will thrash with or without swap enabled. The difference is that with swap the kernel can evict the most rarely accessed memory whether file-backed or anonymous. Without swap the kernel is forced to only consider file-backed memory, even if it is being used more frequently than the anonymous memory.

All the major web companies disable swap so no work has been done to optimize it.

Yeah, in most situations I'd rather kill processes with excessive mem usage (possibly due to memleak) than have the machine grind to a halt by swapping. Sometimes my headless lab machines would become practically inaccessible over SSH if I didn't swapoff before running something that accidentally chews up mem.

I'll let my personal laptop swap, though. Especially if my wife is also logged in and has tons of idle stuff open.

Could just turn down swappiness?!

Memory managed languages won’t do long term garbage collection till memory is nearly full.

I suspect they just need to pass in -xmx options to the jvm to avoid this.

Not sure what long term garbage collection means. Because the JVM will GC all objects at any point if they are unused.

If you're referring to full GC you can configure how often that happens and by default it doesn't just wait until memory is nearly full.

Even if this wasn't factually incorrect, default max heap size on the JVM is 25% of system RAM.

I actually wish this were true because of more predictable gc pauses

With G1GC there is a setting called MaxGCPauseMillis which can give you predictability.

Reply on news.ycombinator.com