Minimize Heap Allocations in Node.js | Hacker News Reader

Minimize Heap Allocations in Node.js

blog.appsignal.com

57 pointsunripe_syntax29 comments

V8 dev here.

Yes, please avoid leaking memory -- but the intro to this article is incorrect in almost all major points:

> In JavaScript, primitives are assigned/passed by value and not by reference. If you assign a primitive type's value to another variable, the value is copied.

No, primitives are immutable but still passed by reference (that reference is passed by value, which is why assigning to the reference won't change them -- the same is true for objects). Can you imagine if we'd copied strings every time they were passed into a function?

> This also determines how they are stored in memory — primitive values are stored on the stack while objects are stored on the heap, with a reference to their location stored on the stack.

Also no, all primitives except small integers ("Smis") are stored in the heap.

> One important thing to note about variables stored in the stack is that their sizes must be known at compile time. This allows a program to allocate the right amount of memory in the stack. Because the JavaScript engine allocates a fixed amount of memory for primitive values, there is a limit to how large they can be.

Again, no, with strings as the obvious counter example. There's a limit to how large values in general can be, but it's around indexable size and pointer width, not a fixed amount of memory for primitives.

> Memory stored in the heap will live on unless it is explicitly deleted or deallocated.

Is the author confusing the malloc heap with the (garbage collected) JS heap? Sure, V8 has to manage the pages it allocates, but JS authors certainly don't

> To allocate memory in the heap, a program must first scan the heap to find a contiguous memory block large enough to hold the data.

As a sibling comment points out, there are optimisations here with linear allocation buffers that permit bump allocation. Also, don't forget the semi space for young objects, since the GC is generational.

> No, primitives are immutable but still passed by reference

For immutable values the pass-by-reference vs pass-by-value distinction isn't really important. Unless the language provides some way to get the identity of the object there isn't a way to tell a difference.

> Can you imagine if we'd copied strings every time they were passed into a function?

IIUC most Javascript interpreters optimize copying strings anyways, so this wouldn't be a big deal. For example even though `s.slice(1)` is a "copy" most engines (I believe V8 included) will use the same backing string for both. So the difference would be a pointer on the stack vs a pointer+length on the stack and copying an extra word is not a huge difference.

But maybe this is just being nitpicky, since the article is focused on performance the lower level details do mater not the semantics. However for strings I don't think the difference between their model and reality is very significant.

You're talking about semantics and you're totally right, there's plenty of cases in the implementation where things are not copied when they should be (string slices) or copied when they shouldn't be (boxed double writes into mutable double boxes). As you say though, the semantics are kind of irrelevant when you're talking about actual memory behaviour of allocations.

There's actually lots of really interesting consequences of the current implementation, and the various tricks it does around what's observable and what's fast, it's just not what's described here.

> For example even though `s.slice(1)` is a "copy" most engines (I believe V8 included) will use the same backing string for both.

That’s not a copy, it skips the first character!

It's semantically a copy of most of the string.

Nothing against the author, but damn is this a good example of why you can't just trust every piece of technical writing you see.

Maybe strings are not considered primitives by the author. They do give a list of "Data types stored in the stack" which I understood to be the "primitives" in what follows.

From the article, "Data types stored in the stack include: [...] string"

Ouch. Yeah that's wrong.

only SMIs are passed by value (because they are actually encoded in the reference data type)

It might be a bit outdated but here's more https://github.com/thlorenz/v8-perf

Also the https://v8.dev/ blog is awesome.

The errors have already been pointed out by multiple comments.

Opening the article, it is immediately obvious that the author sourced his material from random Js blogs. He understands very little about the v8 engine and just fills the gap with those material which are obviously wrong.

On the hand, he can be happy that his article made it here where he would be corrected for free.

Hope the author hangs out here...

> To allocate memory in the heap, a program must first scan the heap to find a contiguous memory block large enough to hold the data.

Does V8 not have a linear allocation buffer? I’m sure it’s a bump allocation, not a scan for free space.

I skimmed this [1] but I believe you are correct.

It's a generational moving GC which means bump allocation typically.

[1] https://v8.dev/blog/trash-talk

Looking at the code it seems to use a linear allocation buffer: https://github.com/oilpan-gc/cppgc/blob/main/src/heap/cppgc/...

You're looking at Oilpan, which is the C++ GC for Blink, not the one for V8.

But, the V8 one is very similar in many respects (the same team works on both), including linear allocation buffers.

They are one and the same I as I understand it now:

https://v8.dev/blog/oilpan-library https://github.com/v8/v8/blob/main/include/cppgc/README.md

Quote for the frist article:

> We launched the library [Oilpan] in V8 v9.4 and enabled it in Blink starting in Chromium M94.

It's a bit confusing, I agree -- Oilpan is part of the V8 sources for shipping-the-org-chart reasons, but it isn't actually the GC used by V8 itself for JavaScript objects and the JS heap, it's the GC used by Blink for DOM objects etc.

The biggest difference between the two is that JS objects are fully managed by the GC and V8 has full control over their layout and access (e.g. they are accessed via explicit pointer + field offset in the runtime), while Oilpan works on C++ objects; there's a billion consequences of this, in particular that JS objects are movable (because we can safely update pointers) and Oilpan objects aren't (because C++ can do pretty much whatever it wants).

At one point, it was supposed to be the universal GC for chrome. In fact, that was cited as the major holdup for getting the DartVM into chrome, they wanted a unified GC between dart, v8, and the DOM.

Thanks for clarifying!

Am I missing something, or does the article not go into its title of minimizing heap allocations at all?

It talks about not having memory leaks, but no strategies for minimizing the use of the heap in the first place. As I understand it, this is an important technique to eke out the most performance. I know I've seen an article on minimizing allocations on Go, and doesn't Java have "value types" for that sort of thing?

I would be interested in the JS equivalent, and am disappointed the article didn't actually get into it. It's imagine it's stuff like using byte arrays and such?

> In JavaScript, primitives are assigned/passed by value and not by reference. If you assign a primitive type's value to another variable, the value is copied - not assigned a reference to the first variable (which is what happens for objects).

Bit off topic but is there actually any observable difference between primitives being value based vs being references to constants? Here's the example they give:

  let num1 = 5;
  let num2 = num1;
  num1 = 6;
  console.log({ num1, num2 }); // { num1: 6, num2: 5 }

But it's exactly the same for objects (e.g. arrays in this case):

  let num1 = [5];
  let num2 = num1;
  num1 = [6];
  console.log({ num1, num2 }); // { num1: [6], num2: [5] }

Definitely easier to think about them being value based and of course they are implemented like that but I don't think it's the only possibly interpretation.

There is no difference between pass-by-reference and pass-by-value for immutable objects (unless there is a way to check object identity). So if you call the JavaScript primitive types Number, String, undefined, boolean, reference then all of them are immutable so there is no difference. Of course most people don't think of JS as passing a "reference primitive by value", they think about "passing an object by reference". Since all of the primitives are immutable it is probably then easier to think of JS as a pass-by-reference language from the semantic point of view. In fact if you really want to simplify you can imagine that every variable is just a reference to a value since it is equivalent.

You have to modify num1 to see the reference change. e.g: `num1[0] = 42`

Your code is currently only reassigning a new value to the num1 variable.

No because I was talking about constants so you can't modify them. (Obviously you can literally modify them in the code I posted but it was just an illustration.)

This is not a good article, it confuses immutable with fixed size, claims primitives are stored on the stack but includes strings in that set, and then claims that V8 used to have a problem with cycles because it used refcounting????

Also I found the writing itself to be long winded and hard to read even without the many technical errors.

If you want to reduce memory consumption by your node.js server, minify it (e.g. via esbuild in node.js mode) and don't use inline source maps (if you want to keep source maps, use .map files).

node.js keeps all source code as string in memory. So unminified code can bloat your runtime by a lot if you require a couple of external libraries. If you bundle the libraries as well, you don't even need to copy node_modules anymore, which can also save time in deploying a change.

This small change can cut your memory requirement in half.

Of course I wish node.js just had an option to free the memory of the source code garbage, but as of version 18.9.1 there is none.

> If you assign a primitive type's value to another variable, the value is copied - not assigned a reference to the first variable (which is what happens for objects).

Assigning an object also just copies the value - i.e. the reference to the heap value. So the new variable does not end up pointing to the first variable. Rather, both variables end up pointing to the same heap value.

I like this, it looks like a good reference on memory management...but I see Leszek saying, "no". OK, maybe the authors don't know the theory, but they learned some useful tricks to reduce. Will still read.

Related, when I need lots of memory in node, I use: `--max-old-space-size=` programmatically set to some % of total available memory on the system.

To be clear, maybe I was a bit negative, but I was just talking about the technical intro. The rest of the article on not accidentally leaking references through globals and timers, and what tools to use to catch them, is pretty solid.

Reply on news.ycombinator.com