Windows 3.1 All Over Again
tomjoro.github.io
tomjoro.github.io
Python has this problem with "async". If you make a blocking call, you stall out the "async" stuff. Go gets around this by having a "fiber" or "green thread" mechanism underneath. If you block on a lock, that doesn't tie up a resource needed to run other goroutines. This is why retrofitting async to a language has problems.
Starts off nicely with a good premise, but instead of digging into the interesting details it glosses over them and starts to preach Elixir.
I don't have any stake in this, I use neither js or Elixir, but this article is too shallow to be really useful.
10 print "mike is cool"
20 goto 10
while (true) {
print("mike is cool");
if (shared_global_flag === true) break;
}
for example
var x=1;
while (x<10) {
output(x);
x+=1;
}
I don't see the wisdom of disaallowing something due to the specter of race conditions when its absence seems to cause a much easier to cause debilitating problem.
https://blog.codepen.io/2016/06/08/can-adjust-infinite-loop-...
I'd be ok for a light pre-emtion that would happen between javascript statements. Instead of genuain preemtion have the Javascript engine include break checks. A check of a flag and conditional branch (usually not taken) is not a huge expence. The cost can be especially ameliorated if you do the check only on statements that jump to earlier statements.
That fixes the problem of long running code stopping other things. It doen't need system level threads or parallelism, just a js engine doing spot checks.
If you have a long-running synchronous algorithm, you should not be running it in Node, or alternatively you could dispatch to another process/C lib and have it run in a true thread and asynchronously wait for the result.
No remotely relevant app is only ever IO-bound. There are always some long lists to sort, some graphs to walk, some intricate canvas renderings to do.
This is probably one of the worst usecases for node.js
A lot of microservices that these apps could be split to certainly are.
Define "relevant".
I work on a giant [Health|Fin|Defense]Tech monolith which has been around forever, has do to everything for everyone, and has been worked on by hundreds or thousands of developers with radically different skill levels. It connects to many databases, external services, etc., and does some immensely complex data munging just to render what you'd think are simple pages (since the inflexibility of the backing model and limited space to make denormalizations of really big data mean everyone has to do super complex aggregations in the app server, across data sources X, Y, and Z, all to show the user a 10 row table).
In short, it's huge, ugly, and computationally expensive.
I was asked to quickly research the benefits of switching its platform (a single-threaded scripting language) to NodeJS (I wasn't told to research anything other than NodeJS, despite my objections).
I figured the savings would be minimal, since all our application servers are constantly running out of CPU (page loads crazy expensive, see above) or mem (aggregations crazy expensive, see above).
So I broke down what the app was doing on some representative servers, working both from the coarse level (dtrace/system resource usage) to the fine level (flame graphs of calls/wait time/yield events within the application runtime itself). I didn't profile the batch processing services; they were RPC'd to via the renderers, and used more appropriate languages/patterns for huge-data manipulations. As far as my profiling was concerned, they were functionally databases.
The result? 88% on average time spent waiting on IO or blocking non-block-file system calls. P90 was 99% blocking.
That went totally against my assumptions.
Sure, our webservers were overloaded with non-IO load, but if we were to switch to non-blocking IO and buy more webservers, we'd have gotten a massive performance increase without having to change the fundamental architecture of our webapp.
That was when I started seriously considering the benefits of reactive-style programming in a single thread, a la NodeJS; it hits a nice balance between "programmers that aren't necessarily super skilled having to engage with a full/real concurrency system" and "do everything blocking one per process".
There are tons of downsides, of course. Switching to nonblocking IO after spending so long in a blocking world would require both massive technical expenditure, and would probably also require reorganizing the capacity planning of all the other services/databases the app servers talked to, since they'd be fielding a lot more requests. Basically, the blocking nature of the render loops was an informal rate limiter on database queries. Parallelizing the render loops via processes gave much more direct control of changes in resource utilization, which is nice for proactive scaling. Additionally, node/callback style is still harder to learn (even with async/await sugar) than plain 'ol top-to-bottom sequential code. All that said, we'd be rewriting code in a new platform that looked different, but the code could still do the same things, per render, in the same order, which is a huge benefit.
A platform that hides preemption/concurrency while allowing people to program in the sequential style (e.g. Erlang) might have been a better fit, but . . . we were already using one of the best M:N resource schedulers in the world, the Linux process scheduler, to multiplex concurrent sequential processes that were just . . . linux processes. At the end of the day, I gained a lot of respect for the power and balance struck by single-thread/event-loop-driven reactive runtimes like Node.
Edits: grammr.
That's very interesting; you were given a huge old legacy app that had scaling issues (cpu and memory). Presumably the business was tired of throwing hardware at it? Or they hadn't gotten to that stage yet? Continuing, and tasked with diagnosing the performance problems you looked at the system and application. I can read up on dtrace, but how did you profile the application level stuff (time/yield)? Was it some functionality provided by the run time? Ie java visual vm for example?
This is a problem many, many companies have; ill performing legacy apps that the "legacy" staff aren't capable of handling (because the talented people left long ago ie "don't move my cheese"). It'd be really educational to see a write up of this!
1. Simple resource usage (system time vs. user time, memory, etc.) got the metrics for how long the OS thought the app was spending waiting on IO.
2. Dtrace was able to slice those up by where/how they were being called, which syscalls were being made, and what was being passed as arguments. This was important for filtering out syscalls that would remain a constant, high cost (e.g. block local file operations on old versions of Linux, which we have, get farmed out to a thread pool in NodeJS, so I pessimistically budgeted as if that thread pool were constantly exhausted due to volume + filesystem overuse).
3. In-runtime profiling. We have the equivalent of java visual VM (well, more primitive; more like jstack plus some additional in-house nice features we built, like speculative replay), but for our scripting language platform. That generated flame graphs and call charts for processes. Those were somewhat useful, but tended to fall into black boxes where things called into native code libraries, which was where the dtrace-based filtering data was able to help disambiguate. Using this we got a comprehensive map of "time spent actually waiting for IO".
There was a lot more to it than that, though:
Since different syscalls both had different call overhead (and different call overhead depending on arguments supplied) and different blocking times, all 3 steps were necessary.
For example, an old monolithic chunk of code that did ten sequential queries to already-open database connections is going to issue select(2) (or epoll or whatever) at least ten times. Conversion to Node, and it's single-poll-per-tick model, would make that cost vastly reduced, making it move the performance needle a lot. Of course, that's only true if the ten queries in question can actually be parallelized, which typically requires understanding the code . . . if it can be understood, which is not a given.
However, a page render that called ten different HTTP services would make ten full-cost connect(2) calls in the worst case, ten low-cost (keepalive'd) connect calls in the best case. Node would still have to make those same ten calls, making it a less needle-moving thing to move into nonblocking IO (though the time spent waiting for connect to complete or time out would still not be paid directly in the render, which had to be accounted for as a positive). And it goes deeper: depending on the services being hit, keepalive window, and rate at which they were called during a typical server's render operations, we had to calculate how often, say, a 50-process appserver worker pool would be redundantly connecting to those services (because separate sibling processes can't share the sockets if they're initiating the connection, and before you ask, I would not like to add to the chaos by passing open file descriptors over unix sockets between uncoordinated processes thank you very much). If the redundant connect rate was high, Node might offer significant savings by allowing keepalive'd sharing of connections within a single node process (we'd need many fewer of those per server than appserver workers). If it was low, fewer savings.
TL;DR it's complicated but possible to measure this data using established techniques. You don't have to get super far down the rabbit hole to get a decent guess as to whether it will be beneficial to performance, but transforming decent into good requires a fairly thorough tour of Linux internals.
And, as always, the decision of whether or not to switch hinged primarily on the engineering time required, not the benefits of switching. C'est la vie.
If you can design an application where this kind of separation won't overcomplicate everything, Node.js is probably pleasant to deal with.
Just a few weeks ago there was an interview with Ryan Dahl where he said:
"...I think Node is not the best system to build a massive server web. I would definitely use Go for that. And honestly, that’s basically the reason why I left Node. It was the realization that: oh, actually, this is not the best server side system ever."[0]
[0]https://www.mappingthejourney.com/single-post/2017/08/31/epi...
What would be the point of saying that Elixir is superior in this situation, were it not?
Async != FP
Back in my high school days, FP was almost strictly about usage of pure functions above anything else.
The late comers to the party, the React guys, then bunched FP with async stuff, and function polymorphism. That was wholly apocryphal to the original FP idea.
I can still cite my high school teacher citing Niklaus Wirth: "If a function does not interact with external data in a any way other than reading them from its inputs, the whole program can be represented as a spreadsheet"
The original advantages were states as:
1. a program written in this manner is very easy to optimize. 2. an interpreter or a compiler executing such program is easier to program, and is less error prone 3. for as long as a kind of middleware sits in between the "spreadsheet function," and the data, it is possible to write a continuous execution log with ease, as well as to do tricks with getters/setters to start execution from arbitrary points in code.
All of that is stuff from times when Oberon and Modula were "hip" (seventies)
I dunno, maybe you could serve one of your 100 other concurrent requests?
> You can do something useful in the meanwhile and get the result later
and then immediately...
> It has nothing to do with hogging or stealing resources.
It has everything to do with this. If you're blocking on something waiting for a result while you could do something useful in the meantime, that is the definition of hogging resources.
In theory their performance and behavior should be the same. BEAM and other N:M green thread systems being maybe faster if they can efficiently use multiple threads. But since they use message passing anyway (needed for isolation), they are again almost equivalent to running multiple Node/reactive request handlers.
N:M is probably useful for desktop apps. But for that single thread is (should be) always enough, so same behavior.
Yes, I see, if you have data crunching in the event loop, you have to "manage" it, by either delegating it outside of it, or chunk it up into small pieces, and yield between.
But that should result in the same performance as BEAM, because you can run 1 big runtime or many small event loops on each box, and put a load balancer in front (which is needed even for BEAM, because if you have a more threads/processes than cores throughput AND latency will suffer).
Agreed on the DoS thing, but nothing, sort of a proper timeout (and/or hard RT) will save you. Yes, with Node et al. you can lose a lot of requests when you inevitably kill the runtime. But it's just as easy to DoS a different runtime. (BEAM just gives up a few seconds later.)
This isn't about performance, it's about how you write code. In Erlang/Elixir (and also Go and Haskell), you aren't sitting there constantly explaining to the runtime over and over again how to plumb together bits of "async" code. Essentially, all code is simply presumed async, and the plumbing is always already there for you; the act of calling a function includes, for free, all the "async" plumbing.
I expect anyone who has been doing this for a couple of months should have noticed it's the same plumbing, over and over again. Here's the long-running IO task. Here's what to do with it when it's done. Here's what to do with errors. Here's the next long-running IO task. Here's what to do with it when it's done. Here's what to do with errors, which is pretty similar to the last thing. Here's the next long-running IO task. Here's what to do with it when it's done. Here's what to do with errors; whoops, forget that one, which is gonna cost me during debugging but nothing in the code, runtime, or syntax affordances particularly cares that I forgot. Here's the next long-running IO task....
Yes, they do come out roughly the same in speed for Erlang/Elixir and JS, and Go can quite easily be substantially faster... and easier to write.
And if we want to discuss performance, Go still isn't as fast as it could be. There's a solid reason why N:M threading can't be the fastest way to run code in general, but I don't see much reason why it couldn't be something more like 20% away from C speed in general, with some pathological cases where it's much slower and out of the question (very fast packet handling with those fancy software TCP cards, for instance).
From what I've done async/await in ES (and C#) is easier to write than Go's equivalent, but we're hitting your mileage may vary territory.
That's exactly the problem with node.js and other promise/callback/deferred/futures mechanism - it mixes concerns. You business algorithm probably doesn't look like "1) authenticate & authorize 2) return a callback then go to select loop to process 100 other connections 3) add book to cart etc". It probably doesn't have 2) in there, and it shouldn't. The point was that that particular request doesn't and shouldn't have anything else to do besides wait for the connection from database to return a result.
Sure there maybe there are 2 million other (https://blog.whatsapp.com/196/1-million-is-so-2011) connected clients wanting to buy a book, but the logic for a single doesn't have to care about it. That's the beauty of it.
> If you're blocking on something waiting for a result while you could do something useful in the meantime, that is the definition of hogging resources.
That's not something you should be doing that high up the abstraction levels. If you have to worry about it that high up there is something broken with the framework / language / library.
Like the OP says, it is basically the equivalent of Windows 3.1. Nobody sane today would put their crown jewel, latest production systems running on Windows 3.1 but yet we do with when it comes to our choice of frameworks and libraries.
For my last side project I was evaluating go and erlang. Just to test out how faster go is, I quickly coded a simple HTTP request in both language(I am not an expert in either).
The server would get a request with large body, cache it to a file on the disk, then load it in 128k chunks, get the hash of the chunk and save both the digest and the 128k data to a database.
To my surprise, erlang was consistently slightly faster for each request. Happy to share the code snips of both if anyone want to audit/scrutinize.
Edit: Code for both: https://pastebin.com/aBQWqkG3
On my laptop a single request(1.2MB) takes around 230ms for Erlang and 280ms for Go. Not a scientific test, but it gives an idea.
I believe it might, depending on the database driver, also cause another issue. Since you're not closing the rows right away, and not even consuming the rows, this may force the driver to mark the internal connection as busy until the defer runs, meaning the next database call would have to open a new connection. Try closing the rows as soon as you can.
If you don't expect any results, you can also use Exec instead. Then there's nothing to close.
Will try with Exec and without the defer and see if it makes a noticeable difference.
The driver is the MySQL one for what’s worth it. Postgres was a bit slower.
Exec vs Query didn't make a difference.
Yet another update: things get really interesting when I run wrk. I don’t want to spoil it for you. I can let you try it ;)
The program is mostly i/o with sha256 being the only CPU intensive part. If sha256 (or any compute-intensive part) was implemented in Erlang, it would be at least 10x slower than Go version.
Go version probably has a bug: file.Read() doesn't guarantee to fill the buffer, you need to use io.ReadFull() for that.
Both versions unnecessarily write a copy of the file to disk (and don't seem to delete those randomly-named files from disk).
In Go saveToDb() would take io.Reader as an argument and you would pass r.Body directly.
Go is much faster than Erlang, it just doesn't show up in every toy program.
I think the request in my test is quite representative of what an actual web server will be like: mostly IO.
As a general programming language, go is much faster, I completely agree with you.
Btw: I don’t think deleting the files would make a difference. Also I save the body to file first because with cowboy I had trouble splitting it to the exact chick sizes. Will try without it in go and update.
IMHO it is not as huge as everybody makes it look like.
Again, as you very correctly pointed out, this is of course only for IO intensive applications.
It's a sort of a "it's fast enough until it isn't" type thing. Python is fast in the same way (and faster with each release!).
For anything that isn't dominated by network and database time, language and runtime performance matters, and OTP/Elixir is slow in that way. In terms of "real world performance", it can sometimes matter more that the incremental cost of preempting your beam process is low, more than whether your process completes in more or less time.
What saddens me is that only Erlang/Elixir get the “it’s slow” argument although they are in the same class as python and ruby.
For everything else to be faster you need asm, c, c++, fortran, or rust.
Loading an applet over a modem in 1998 doesn't count...
Maybe jigsaw is one piece in the puzzle to reduce the time it takes to start.
I doubt it's so bloody complicated, except AOT compilation of all system jars, I suspect it's mainly a matter of getting the time and resources to look at what happens when hello.world is executed.nu
That and relabling executable jars to jxe and the desktop is Java's to grab (that last part was a joke)
Then we went through with the full POC of our toy webserver project and the performance difference between Go, Elixir, and Scala was pretty much insignificant (within 5% of each other).
10 A = 1
20 A = A + 1
30 PRINT "HELLO " + A
40 PRINT "YES"
50 GOTO 20
Yes, this requires making sure the code doesn't do anything CPU intensive, but for the application I have running, that is the case. But for me, using Lua coroutines means the main business logic is still sequential, even in an event driving environment.
Also, because you call a function to preempt execution, you don't need to save the entire CPU state. For C on the x86-64 bit, this is just six registers: https://github.com/spc476/C-Coroutines/blob/314607cf058352be...
>An Erlang process is not a system process. An Erlang process is a lightweight process. But here’s a key difference: there is no way in Erlang for processes to share memory
I'm confused. Is there real concurrency in Erlang? If yes, what Erlang processes use under the hood: system processes or threads? If threads, why they aren't sharing memory?
The language has no concept of pointer, so there is no way to create a reference to memory owned by another process. You can send a message to another process, but internally the VM will copy the message to that process’ heap.
To properly reason why the approach is wrong, shouldn't you consider all significant reasons, including backwards compatibility as probably the biggest one?
"New language that learns from the mistakes of languages before it" would generally be better than "language and runtime that is keeping compatibility with a programming language that was heavily rushed just to fill a feature point for a web browser"
One might as well state that the grass is green and sky is blue, no?
The only reasons there is no preemptive multithreading in Node is just that the V8 interpreter is under GIL.
Is that a mistake with "HELLO3"? Should it not be HELLO2? Or am I stupid?
See also the "help" link next to the edit box: https://news.ycombinator.com/formatdoc
Sometimes there's a 3rd choice: actors. imo it's much easier to manage than threads
Sequential access to the DOM can be ok because we are the only user of the browser. Single processing is not so ok on the backend because there could be thousands of users there. We scale Ruby, Python and Node with multiple processes (I'm doing it.) I'm also developing an Elixir application using Phoenix. The approach is similar to writing Rails or Django code (I never sent a single message in all the application) with the convenience of not having to manage sidekiq or celery for background jobs (they're in the language) and autoscaling.
This is easy to test: we just have to wait until we can ship byte code through the browser, and see how long it takes for another layer of abstraction missing some important feature to pop up on top of it.
As a baseline, you have Asm.js[1] + Web Workers[2] today in most browser places (actually, all places, if you creatively polyfill.) For newer-fangled browsers only, you have Web Assembly[3] + Web Workers, which takes in-browser performance to a whole 'nother level.
Now, the Web Assembly spec isn't stopping at replacing Asm.js. There's going to be support added for SIMD and vectors and whatever cool stuff newer processors can do. The big deal will be once Web Assembly gets the ability to do syscall-like thunks to DOM API and the other myriad JavaScript API. At that point, the browser's a full OS platform capable of hosting most applications. This new thing will then be able to go beyond any previous application platform in terms of reach and ultimately, capability.
Just forget about the term "browser" when the thunks appear. This will be a new "Platform 1.0" that will span all modern operating systems and devices. Node.JS will implement a compatible "Platform 1.0" subset for server-side platforming where client and server will be able to share binary libraries.
This is coming soon. If you've got some great idea for a next generation online app, it's time to start working hard and have it target "Platform 1.0"!
[2] https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...
True, your JS code all runs in a single thread, but all the heavy lifting is behind asynchronous interfaces, delegating the work to a threadpool in the background.
What a Node.JS application really is, is a supervisor thread (your JS code) giving work for a bunch of worker threads. Doesn't sound very single threaded to me.
If not, then calling it multi-threaded is misleading to a programmer who needs such things, and instead you'll need to find a more accurate word.
process.env.UV_THREADPOOL_SIZE = 10;
function startOne() {
const start = Date.now()
require('crypto').pbkdf2(String(start), 'secretSalt', 100000, 512, 'sha512', (error, result) => {
console.log(`${result.length}b took ${Date.now() - start}ms`);
startOne();
});
}
for (let i = 0; i < 10; i++) startOne();
If you are talking more generally about anything we can run within Node.JS, is using Node.JS user-facing APIs and works without doing any modifications to Node.JS, the answer is yes. You can easily achieve that with node's native module support, and I would encourage you to do so over committing the folly of doing anything CPU intensive in JS.
That is if you manage to find any CPU intensive task that isn't already handled by node built-ins or some library out there.
Edit:
Now you may say: "Ahah! You can't use threads using [limited usage of node]. Also threads must be usable for arbitrary workloads in order to consider a thing multi-threaded. Node.js is single-threaded!"
I just can't argue with that. It's just a matter of opinions now about what makes a thing/environment single-threaded, and what makes it multi-threaded.
To me it's simple: [x] Thing uses multiple threads to perform work.
Yes, delegating to the C++ modules in the background, which do support multi-threading.
These modules have nothing to do with Node directly so meme stays - node itself is single-threaded no matter how you spin this off.
Eh, what are these if not part of node?
What kind of definition are you applying so that a thing is suddenly no longer the sum of its parts?
Edit: JavaScript may be single threaded. Node.JS is not. JavaScript is but a part of Node.JS.