Hacker News Reader

Comment by ibraheemdev | Hacker News Reader

> There is no way for your operating system to know exactly how much stack space a thread will need so it allocates an amount on the order of around a megabyte. You only have around a bakers dozen gigabytes of RAM, so you can only make give or take 10,000 threads.

That's not true at all. The megabyte(s) of stack space is virtual, spawning a thread only requires a few kilobytes of memory initially, and is a lot cheaper than many realize.

View on HN

Here is the part where I admit that I am an idiot and don't actually know.

All I do know is that in Java I cannot create this many threads and in the talks given about Project Loom this is given as the explanation as to why.

https://www.youtube.com/watch?v=EO9oMiL1fFo @ ~4:00

Is it maybe not the actual spawning but some "switch" that flips once you start using the thread?

I'll update the post or leave a comment with a correction once I understand exactly what you mean

With Fibers you can implement user-space switching between virtual threads. For example, if a thread is going into a top-level select, switch it to a small stack putting the original large stack into a pool. On wake resume switch back to a larger stack. That’s basically what Project Loom is doing I suspect.

No, I understand how Loom limits the amount of stuff on the stack. Its just I thought threads always had those large stacks.

Based on what the top person said, I'm thinking that they don't get a large stacks until they cross some threshold of use, but I want to make sure.

Stacks are allocated statically upfront. However, you can change the size the kernel allocates, you can point them to use a custom memory space instead of the default one and you can change the stack you're running on. Also, like all memory pages, stack space isn't actually allocated until you modify it so if your stack depth doesn't get crazy large/you're not storing large objects on the stack, it's possible your 16 MB stack is only using 1 MB or so of real RAM.

The explanation in the video is good, here is a transcription for convenience (emphasis added):

> OpenJDK implements threads as thin wrappers around the OS thread [...] OS threads are expensive, by which I mean that we cannot have too many of them. Maybe a few thousand active ones, maybe ten thousand active ones, but that's about it. And the reason for that is that the OS doesn't know the various languages, and the most costly component of a thread is its stack, and the operating system doesn't know how the various languages it might support makes use of the stack. So it must address the worst case and allocate a very large stack. It's usually on a megabyte scale. And even though we use virtual memory and not all of it is committed to physical memory, once we touch a page it becomes committed – it can't be uncommitted again because the OS doesn't know how the stack is used. And in any event, all the operations are done at a page granularity which is 4k, it's also not that small.

There may also be arbitrary OS-specific limits. For example on Linux there is system-wide limit on the number of threads, controlled by /proc/sys/kernel/threads-max (docs: https://www.kernel.org/doc/html/latest/admin-guide/sysctl/ke..., see also: https://stackoverflow.com/a/8849114). On my system it's ~60,000 (with 8G RAM).

> once we touch a page it becomes committed – it can't be uncommitted again because the OS doesn't know how the stack is used.

I wonder if this could be solved by the code itself. Every so often tell the OS that you don't need all of the stack pages that are currently unused.

Of course getting the "every so often" right to avoid a huge slowdown in edge cases isn't easy.

for java, i wonder if you could make the stack-size smaller to increase the number of threads possible (as long as you know your application can handle this smaller sized stack - aka no recursion etc).

In Java, it defaults to 1MB on platforms other than Windows.

I've seen people OOM themselves by setting the default thread stack size massively high - they had likely gotten confused with the args (they'd used -Xss when I think they meant -Xms) and given their threads 1GiB of stack each.

I can't speak to how Java manages stack size between itself and the OS though, I'm sure there's deep magic there.

It's also not true when ... you tell it how much you need. Windows lets you specify a stack size [1].

[1] https://docs.microsoft.com/en-us/windows/win32/api/processth...

Fun fact, you can declare a stack size at compile time (something like `cl ... -stack:0x5000000,0x5000000 ...`) which let you create insane stack size so that you can fit your all your software in the stack. Useful when you don't want to had an allocator when making tiny tiny executable files :D

The main cost of spawning a thread is simply the time it takes to spawn.

What are you trying to say? This sounds like a tautology to me.

I think they meant that the main cost of a new thread that you'd like to create is the time it takes to spawn (i.e., the time it takes to set up the new thread and get it ready to run) and NOT the memory that gets allocated for it, nor the operating system overhead of keeping it around afterwards, etc, etc.

(I'm not agreeing or disagreeing; I'm only trying to helpfully clarify)

This is not a tautology.

They are saying that most of the cost is startup cost, not running overhead.

But does Loom reclaim allocated stack space once the thread's stack shrinks?

I believe so, yes.

> But does Loom reclaim allocated stack space once the thread's stack shrinks?

Could you elaborate why you asked this? Is it because it's not done already? I'm genuinely curious.

Reply on news.ycombinator.com