Hacker News Reader

Comment by tantalor | Hacker News Reader

Reminds me much of channels in Go: http://golang.org/doc/effective_go.html#channels

Are Node streams pure, generic JavaScript? Do they rely much on Node internals or outside libs?

I ask because I experimented with "porting" Go channels to generic JavaScript in http://www.johntantalo.com/blog/go-flavored-javascript/

View on HN

From what I remember when I looked at Node's Stream source, I believe they're pure JS. They depend on the EventEmitter and they basically emit a "data" event with a chunk of data as many times as needed.

My main question is that since it's an instance of EventEmitter, and EventEmitter uses a synchronous loop to "emit" events, wouldn't stream-intensive code be slowed down by lots of requests due to the synchronous nature of EventEmitter?

> Reminds me much of channels in Go: http://golang.org/doc/effective_go.html#channels

Or producers and consumers in Twisted: http://twistedmatrix.com/documents/current/core/howto/produc...

This API appears to match the Stream API nearly perfectly.

Max, you should add a reference to Twisted Producers and Consumers API in the article as an example of a similar interface.

they are pure js, though streams in node emit Buffer objects which can be mostly emulated using html5 Int32Arrays

Had to look that up,

> A Buffer is similar to an array of integers but corresponds to a raw memory allocation outside the V8 heap. http://nodejs.org/docs/v0.3.1/api/buffers.html

Are streams limited to binary buffers, or may they also emit JavaScript objects or primitives? Binary may be great for low-level I/O streams, but I'd expect more nuanced events for high-level streams.

You can emit anything but it just so happens that 99% of the streams used in node core are Buffers for performance reasons (Buffers are allocated outside of V8's heap). An example of emitting objects is in here https://github.com/maxogden/dominode/blob/master/example.htm...

Node.js Buffers are actually built on top of V8's Uint8Array. Using 32-bit integers to represent 8-bit values seems quite wasteful.

Source please? The docs say a Buffer is "outside the V8 heap".

Yes, the chunk of memory is managed by node, but each Buffer instance is linked with a slice of the heap through `v8::Object->SetIndexedPropertiesToExternalArrayData()`, the same way a JS `Uint8Array` would.

Here's the relevant link: https://github.com/joyent/node/blob/master/src/node_buffer.c....

Reply on news.ycombinator.com