Hacker News Reader

piscisaureuson Zones – Robust error handling and flow control for Node.js

I'm sorry if it wasn't clear enough from the blog, but the attempt here is not to write a flow control library that solves the callback hell problem like promises does.

You should think of it more as an asynchronous try-catch block, where resources created in the "try" block are automaticially cleaned up.

I believe that flow control in the future will be done with generators; check out https://github.com/visionmedia/co for example.

piscisaureuson Zones – Robust error handling and flow control for Node.js

Note that the comparison page doesn't have the "zone" library we just announced on it. Zone.js is conceptually a bit different and it targets browsers.

piscisaureuson Zones – Robust error handling and flow control for Node.js

> I'm really unclear on how the zone library is doing the magical interception of errors

It monkey patches the entry points into javascript from the binding layer (some wrapping happens at a higher level), for the purpose of keeping track of the active zone. At that level we also wrap a try-catch block around all callbacks.

> Does a library (say, one that came from npm and not the core library) have to be zone-aware to function correctly with zones

It depends. Most modules shouldn't need any modification. However there are certain not-so-common cases (for example, when you're implementing a connection pool) that the module would need to be made zone-aware. It's necessary when the module wants to call a callback (or any function) in a child zone from a parent/ancestor zone.

Right now that's not very easy - you'd have to use a Gate (https://github.com/strongloop/zone#gates). W're working on an easier API.

piscisaureuson Zones – Robust error handling and flow control for Node.js

When control reaches there we know something went wrong within the zone that's sending the response, so it's nice to send a HTTP 500 error back.

However it could be that some data has already been sent back, in which case writeHead would throw. If that happens there isn't much more we can do.

piscisaureuson Zones – Robust error handling and flow control for Node.js

> I'm having a really hard time seeing how this is an actual improvement on the status quo

What we're doing is make it unnecessary to add error handlers everywhere and still get caught by surprising failure modes. If the zone catches an error it cleans up (closes resources, cancel operations when that makes sense) and then routes the error to the zone's result callback so you can handle it in once place.

The 'status quo' is that node has domains, but zones should be conceptually cleaner and thus have less corner cases.

> one of them even has a null exception catch

I'm not sure what you mean here. The http demo catches an exception and prints the stack trace. We can do that because the zone cleans up the socket so it's not necessary to exit and restart node.

piscisaureuson Android OpenSSL has heartbeats disabled

Node statically links OpenSSL and uses gyp (the chromium build system) to build it.

Initially we used the chromium project OpenSSL fork which had these flags set [1]; when we moved to maintain our own gyp-ified fork we kept these flags [2], under the assumption that the chromium folks would know what they're doing.

You may argue that this is naive and you may be right.

However, just looking at that FAQ, OpenSSL provides a justification why it is safe to ignore the specific valgrind error that -DPURIFY surpresses, so I currently see no reason to change it.

    When OpenSSL's PRNG routines are called to generate random numbers the   
    supplied buffer contents are mixed into the entropy pool: so it 
    technically does not matter whether the buffer is initialized at this 
    point or not. Valgrind (and other test tools) will complain about this. 
    When using Valgrind, make sure the OpenSSL library has been compiled with 
    the PURIFY macro defined (-DPURIFY) to get rid of these warnings.

[1] https://github.com/joyent/node/blob/e80cac622569dda0f6753b45...

[2] https://github.com/joyent/node/commit/7eaea7f9e501db0072455f...

Edit: fixed link

piscisaureuson Collaboration not Derision in the Node Community

Well, there you go: https://github.com/joyent/libuv/pull/1015#issuecomment-29568...

piscisaureuson MegaPipe: A New Programming Interface for Scalable Network I/O

Well sure you could do that. But I don't believe that reading say 50 bytes at a time is very efficient even when you factor in automatic syscall batching.

But this is all speculation. I didn't any consideration for this problem in the paper at all so I don't have the impression it was on the researchers' radar.

piscisaureuson MegaPipe: A New Programming Interface for Scalable Network I/O

The way they do mp_read() is not right IMO.

If you have to pass a target buffer every time you call mp_read, that ends up using a lot of memory in a scenario where there are a lot of connections open but traffic is sparse. Since data might arrive at any open connection, you'll have to have at least one "pending" mp_read operation for every socket at all times. This quickly adds up in terms of memory usage: if you were to read into 64k-sized buffers (that's somewhat arbirary - but using smaller buffers tends to be very bad for througput) you’d need 625mb of memory for just these buffers alone to handle 10.000 connections. In a C10M scenario (http://c10m.robertgraham.com/p/blog-page.html) the same would need a staggering 625GB of memory. The good old select model lets you allocate these read buffers "just in time" before you read data into it.

This was a major issue when implementing libuv (node.js) for windows. Windows overlapped I/O has exactly the same problem (and so has that new RIO thing).

A better model would be to have a "buffer pool" that is kept reasonably full by the user-mode application. The kernel could then take a buffer from the pool as soon as data actually comes in from the network.