Hacker News Reader

Comment by 3np | Hacker News Reader

Hm, that does not solve the issue in TFA and I'm not sure it consistently works like you intend:

    $ node -e "process.stdout.write('@'.repeat(128 * 1024)); process.stdout.write(''); process.exit(0);" | wc -c
    65536

You want to use `Stream.finished`:

    $ node -e "const { Stream } = require('stream'); s = process.stdout; s.write('@'.repeat(128 * 1024)); Stream.finished(s, () => { process.exit(0); })" | wc -c
    131072

https://nodejs.org/api/stream.html#streamfinishedstream-opti...

If this helps you, consider bothering your infra team to look at opening access back up for tor IPs (;

View on HN

> node -e "process.stdout.write('@'.repeat(128 * 1024)); process.stdout.write(''); process.exit(0);" | wc -c

That's missing the part where it waits for the last write to flush before exiting.

You might think that would be it?

    $ node -e "process.stdout.write('@'.repeat(128 * 1024)); process.stdout.write(''); setTimeout(()=>{ process.exit(0); }, 0);" | wc -c
    131072

Sure. But not so fast! You're still just racing and have no guarantees. Increase the pressure and it snaps back:

    $ node -e "process.stdout.write('@'.repeat(1280 * 1024)); process.stdout.write(''); setTimeout(()=>{ process.exit(0); }, 0);" | wc -c
    65536

Meanwhile:

    $ node -e "const { Stream } = require('stream'); s = process.stdout; s.write('@'.repeat(1280 * 1024)); Stream.finished(s, () => { process.exit(0); })" | wc -c
    1310720

You're not passing anything into the 2nd parameter of stdout.write()

I think this is what @jitl means:

  node -e "process.stdout.write('@'.repeat(128 * 1024)); process.stdout.write('',()=>process.exit(0)); " | wc -c

It writes an empty string and use its callback to detect if it has been flushed, which means previous writes are also flushed.

Your one liner doesn’t do the same thing as the code I posted. You missed passing a callback to stream.write and waiting for the callback before exiting the process.

Once we await that callback to resolve, then the buffered data from before that point is known to be flushed. I don’t want to wait for a “finished” event because I want my program to terminate once my main() function and a flush after main() is complete. If I wait for finish, a background “thread” can extend the process lifetime by continuing to log.

This is also why we don’t set process.exitCode and wait for the event loop to drain and Node to auto-exit. That might be the “right way”, but if someone leaves a lingering setTimeout(…, 10_MINUTES) you’ll have a bad time.

It’s also more clear to reason about. Our rule: if you want the process to wait for your async work to complete, then `await` it.

> Once we await that callback to resolve, then the buffered data from before that point is known to be flushed.

This is not true IME. Just more likely.

See my comment here: https://news.ycombinator.com/item?id=41829905

> If I wait for finish, a background “thread” can extend the process lifetime by continuing to log.

Can you not address this by calling .end() on the stream?

https://nodejs.org/api/stream.html#writableendchunk-encoding...

> Our rule: if you want the process to wait for your async work to complete, then `await` it.

In general I very much agree. In this specific case, though, the awaits are just buying you a bit more time to race the buffer which is why they appear to help. The "flush" will not wait for completion before exiting the process.

Maybe also worth keeping in mind that even if this would be working, stuff like this has historically changed breakingly a few times between major Node releases. If you're relying on unspecified behavior you're obviously in darker waters.

> See my comment here: https://news.ycombinator.com/item?id=41829905

Your comment isn't equivalent to the original code.

Your one liner is doing this: `process.stdout.write('')`

Jitl's example is doing this: `new Promise(resolve => stream.write("", resolve))`

He's passing in a promise resolver as the callback to stream.write (this is basically a `util.promisify` version of the writable.write chunk callback). If the data is written in order (which it should be), then I don't see how the `stream.write` promise could resolve before prior data is flushed. The documentation says: "The writable.write() method writes some data to the stream, and calls the supplied callback once the data has been fully handled". [1]

[1] https://nodejs.org/api/stream.html#writablewritechunk-encodi...

It's so easy to miss a tiny but important detail when working with or discussing Node streams and come out the other side bamboozled by bugs. This whole thread exemplifies why I tell people to stay as far away from streams as possible. Though in the case of stdout/stderr we can't avoid it.

> It's so easy to miss a tiny but important detail when working with or discussing Node streams and come out the other side bamboozled by bugs. This whole thread exemplifies why I tell people to stay as far away from streams as possible

100%.

Case-in-point: After coming back I realize that indeed I did overlook the callback/promise making the difference for the write call. Sorry for spreading lies!

Reply on news.ycombinator.com