Hacker News Reader

Search (elastic/solr), Cassandra, presto, and most other java backend storage and data processing engines need 20+Gb (at least) to be mildly useful. It's not uncommong to go with 64gb+ heaps there too.

Microservices/API won't help with lowering these.

yes, exactly this. and by having scalable GC algos so you don't need to do "off heap" tricks in java, things are much easier.

I'm sure people do use huge heaps, but most web applications I know of (microservice or not) store the bulk of their data in a (separate) database. So, unless your application is a database, which might want a huge heap so it can cache a lot in-memory, I don't think huge heaps are really necessary. IMHO, at least.

Since it's not really possible to do reliable giant user space caches in the POSIX model of things you'd usually delegate to the page cache.