And that the operational improvement once written in Rust was notable enough to write a paper.
Imagine the K8S team porting from Go to some other language for similar reasons.
Use the right tool for the job. The PHP website is running PHP, but it isn’t running a web server written in PHP. Web servers are system software; PHP isn’t for writing system software. Same thing applies here. Node does fine with business logic, but isn’t really architecturally suited to serving the “data layer” when you have control/data separation.
Also, I would assume any crypto in v8 is already written in C, with JS calling into it.
The rust implementation was more efficient than the JS one. A CPU bound service of course is bottlenecked at the CPU, and this benefits from efficiency.
At scale, it makes sense to replace this with Rust. Javascript did the job, but did not provide the same efficiency as Rust.
Except the things that end up being memory-bound instead, but the NPM database isn't large enough for that.
Not in a relative sense. If authorization is 5% of the work, scaling it leaves it at 5% of the work, and it's never a bottleneck. Authorization was being a significant bottleneck, not a tiny percent, and that is somewhat surprising.
Perhaps the new worker threads may alleviate this, but I'm not sure (it's still an experimental API).
Are you sure? I'm not familiar with the PHP.net architecture, and there may be less gains from how PHP has traditionally tied itself as a module to web servers in the past, but Rails (and any number of other dynamic language frameworks) are actually web servers implemented in that language, with an optional separate web server such as NGINX or Apache you can run in front to handle the stuff they aren't as good at (static file serving, etc).
Now, that is a framework, and not the language proper, but I wouldn't be all that surprised to find python.org running on top of a Python framework.
"NOTE: Some of our maintainers prefer to use web servers other than Apache, such as Nginx. While this is permitted (as long as everything ultimately works as directed), we do not officially support these setups at this time"
I don't see how any of this is critical of JavaScript, which isn't even really discussed in the paper and still runs the rest of the infrastructure. If anything the paper is more damning of C, C++, and Java but I still think damning is far too extreme to describe what the paper said.
I don't see it as damning on either of these. On C++ it says "we didn't want to learn it." Which is fine. Maybe after learning they would have decided different, or not. On Java they said "we didn't want to learn how to operate it" as they feared the complexity of an Java application server for a single small service, which they can create in a way which hooks into their monitoring infrastructure. No damning their either.
However their company's purpose is to push Javascript and they are saying "operating JavaScript is hard, doing this in Rust is easy" which directly goes against their business.
Is this maybe parsing data structures itself?
Publishing is a relatively rare event compared with reading, but in a world of private packages, orgs, and teams, the "can {user} read {object}" gets more complicated. It probably wouldn't be CPU bound if not for the sheer scale we're dealing with, but once all the IO bottlenecks are resolved, you still have to check to make sure that a login token is valid, then get the user associated with it, then check the teams/orgs/users with access to a thing (which might be public, in which case, the problem is a lot simpler, but you still have to get that info and check it), and then whether the user is in any of those groups. So there's a straightforward but relevant bit of CPU work to be done, and that's where Rust shines.
k8s was originally written in Java, so they have already done this once.
There is a FOSDEM 2019 talk about it.
"The clusterfuck hidden in the Kubernetes code base" https://fosdem.org/2019/schedule/event/kubernetesclusterfuck...
> We look at what it would take to begin undoing the spaghetti code that is the various Kubernetes binaries
Well at least the developers are being frank about it I guess.
> The audience walks away feeling empathetic that they aren’t alone in their journey to writing idiomatic Go and is now equipped with strong refactoring techniques developed by some of the world’s top engineers for the Kuberentes project.
As an occasional user of kubernetes, minikube etc. it's not something I would have guessed to have been developed by the word's top engineers.
I mean kubernetes tries to, and probably manages to provide an useful abstraction, but at a few million LOC and a few man-month of full time senior engineering effort to run anything in production it's not exactly a epitome of elegant and efficient engineering.
https://adtmag.com/articles/2019/01/17/project-marble.aspx
If you want to have some reading fun, check /r/androiddev/ every time there is a "stable" release.
The interesting thing with Kubernetes is that it's basically a re-imagining of Borg, which one assumes was not a few million lines of code when it was already running all of Googles infra more than a decade ago. It's obviously not solving the exactly same problem (e.g. Google correctly recognized that DNS isn't so hot and wrote their own replacement protocol, BNS which wouldn't fly for external adoption etc.). But I'd be curious to know how Borg's code quality and size back when it became the standard way to run stuff at Google maybe 12 years ago compares to Kubernetes today.
Every projects of that scale are going to have issues.