On problems with threads in Node.js
future-processing.pl
future-processing.pl
The thread pool in node is only used for a limited number of APIs. Pretty much all networking uses native async IO and is unaffected by the size of the thread pool. Things like Oracle's driver are rare exceptions: the typical MySQL/PostgreSQL/redis etc drivers all use native async IO and are unaffected by this.
The author only glosses over this briefly. As a result this article leaves the impression that the problem described is the norm, which is not the case.
Is there any reliable way to check if the libs you're using are subject to this issue?
* kerberos, unused (dependency of mongodb)
* protobuf, for serializing data
* snappy, for compression
kerberos isn't actually used in our app, so it doesn't matter, but we send a lot of data through protobuf and snappy, so it may be worth us profiling this a little more.
Are they still using threads to get the "magic" working? I'm referring to this sentence: "But how did that happen? To the best of my knowledge node.js, is not powered by magic and fairy dust and things don’t just get done on their own."
Pretty much all event loop based programs work the same way: instead of blocking on a single request for IO, they use system calls (e.g. epool_wait) that block until any of the many descriptors (sockets) has some event (data to be read, client connecting, etc). It gets a bit complicated when there are queued tasks for the thread pool and timers involved too, but its the same principle.
That said, using a driver in another thread leads to various complexities that can be avoided when running in the same application (javascript/v8) thread/event loop.
https://www.npmjs.com/package/querystringparser for query string parsing. Depending on content, you may get massive improvements (5x-20x)
https://www.npmjs.com/package/fast-url-parser for url parsing (the built in url parser is the main reason why node is so far behind on the TechEmpower benchmark - with this replacement the benchmark shows about 60-80% improvement in served req/s)
https://www.npmjs.com/package/cookieparser
For very large post bodies, I use JSON in conjunction with OboeJS - http://oboejs.com/ . Its not too much slower than native JSON.parse (about 5-7 times) however its non-blocking. Still haven't found a solution that is close enough in speed to native JSON.parse
From [here](http://docs.libuv.org/en/latest/design.html), it sounds like all file IO is always based on blocking primitives, and native async file IO primitives are not used, although such async file IO primitives do exists and were tried out in libtorrent (http://blog.libtorrent.org/2012/10/asynchronous-disk-io/). The result of that experiment however was mostly that the thread pool solution was simpler to code (I guess).
From the libuv design doc, the overview is:
* Filesystem operations
* DNS functions (getaddrinfo and getnameinfo)
* User specified code via uv_queue_work()
I wonder whether this is really the best solution of if some combination of a thread pool and native async disk IO primitives could perform better.
and uniformly asynchronous (native async operations may not cover e.g. file copy or filesystem operations, furthermore filesystems may block during submission of IO ops which makes the operations effectively synchronous) and have higher throughput (they support read/write vectors).
And the higher throughput seemed only to be a problem on MacOSX, so we could fallback to the thread pool there, but use the async IO on Windows and Linux.
Edit: apparently I was wrong.
But if you have also read the following blunt criticism from Linus http://yarchive.net/comp/linux/o_direct.html he also outlines a better alternative way of implementing asynchronous disk io on Linux at least.
Also, the code says it will print the time taken since the start of the program, which again doesn't go with the output and the conclusion being made!
Anyway, how come the output isn't in order?
> However, watch what happens if we double the number of iterations
The order of the output is dependent on when each call finished - they run in parallel, so it's not guaranteed that functions will end in the order they were invoked.
For some reason I was thinking the readdir would run in series so output would go up by ~1s each time.
for (var i = 0; i < 3; ++i) {
namedFunction(i);
}
function namedFunction(id) {
fs.readdir('.', function () {
var end = process.hrtime(start);
console.log(util.format('readdir %d finished in %ds', id, end[0] + end[1] / 1e9));
});
};Now if you want to get good performance on an SSD (ie the rated iops) you will need a decent queue depth, like 32 or so, so it wont work but thats a specialist use case.
No big deal really. I have multiple servers running node.js under load for 3+ years and never had an issue with this.
In fact, I found it helpful. If your database is under load and is already running 4 heavy queries, not giving it any more jobs is actually a good thing.
In general, this looks like it could be a performance bottleneck for highly-concurrent applications.
Actually the threadpool shouldn't be a factor then, unless there are no nonblocking/async drivers for the database: internally, libuv uses its threadpool to asyncify operations for which no async/nonblocking version exists (Erlang does the same IIRC), for the most part it's filesystem operations, getaddrinfo and getnameinfo (I'm guessing it could include other stuff depending on what the underlying OS provides).
Socket or file IO should have native non-blocking APIs, so it does not need to use the threadpool.
User-interfaces 101: don't sacrifice the latency of short-lived jobs for finishing lengthy jobs.
Also, let the OS figure it out. That's what it was made for.
The OS is dumb and does not understand the nature of load. If we applied your logic, everyone would still run Apache instead of nginx.
Your user-interfaces 101 fails flat on its face when you have hundreds of short-lived jobs that need to happen. Each one is insignificant on it's own, but if you overload the server will all of them at the same time just can starve system's RAM (forcing it to swap) or increase disk seeks by order of magnitude if short-lived jobs are asking for different data that is all over the place.
The only true answer here is: it depends.
Are you aware that people like you are destroying something that was once brilliant?
HN is going through an Eternal September.
There is however a deeper underlying issue; decorum is important and communities that exhibit genuine 'niceness' are nice. Communities that allow, or worse, overlook dark behaviour degenerate.
Flagging and down voting is one part of the solution, but when the nastiness reaches a level that the nice people start to disengage and go elsewhere, it's clear to me that we need another element of control. Perhaps algorithmically detecting repeat offenders? Perhaps more granularity with down votes?
There's are differences between a down vote because one disagrees with the author, and a down vote because one believes the author is ill-informed and spreading misinformation, and a down vote because the author is being downright juvenile.
A number of hits on the third case against a given author on multiple comments could conceivably constitute an automatic warning and / or banning system.
I don't want people to be unable to express their views, but when the mean-spirited people who contribute nothing but nonsense start to represent a large percentage of a community, it's reasonable to see if anything can be done.
The difference is that the first two should not be voted down. If you vote down, you should not comment. If you comment, it means at the very least the comment added to the conversation, unless your comment is also not worth posting and you should be voted down as well.
It's fairly simple: does the comment bring value to the conversation? If it does so directly, vote up. If only indirectly, than don't. If it does not, vote down.
Whether you disagree or not is irrelevant. And someone being ill-informed should be corrected. At the very least, by writing an incorrect comment, they are presenting an opportunity to be corrected.
> I don't want people to be unable to express their views, but when the mean-spirited people who contribute nothing but nonsense start to represent a large percentage of a community, it's reasonable to see if anything can be done.
Things can already be done. Vote down and don't reply. That is the best way. Vote down and ignore.
I also think pointed, honest replies like yours above go a long way. The best way to get people to assume good faith is to show it. (Reyk's Second Law)