JavaScript Containers
tinyclouds.org
tinyclouds.org
This is entirely besides the point. The emphasis is mostly on V8, which we all know is first and foremost a JavaScript engine. But it is also a WebAssembly engine, meaning several languages beyond JavaScript can execute with the approach he's talking about.
What the post doesn't really go into detail on is how V8 is arguably the most secure runtime in the world. The browser runtime is one of the most battle tested pieces of software ever built. At its core, it enables remote code execution on anyone's machine. It has a robust security model. Your program can't do whatever it wants on the host operating system, unless granted by the user.
Docker Containers were an improvement over Virtual Machines by enabling dependency snapshots and compiled programs on top of an OS. V8 Isolates (translation: chrome tabs) are an improvement over Containers. Cloudflare's discovery here is what enabled them to build their edge network. With a containerized server deploy, you can execute dozens of concurrent, isolated V8 programs without needing to spin up a new container for each of them. The server also has complete control over execution time and memory. If a V8 program consumes more than its budget, it can easily be terminated.
Runtime performance concerns about JavaScript are also misguided. With Rust, it's easier than ever to write native functions that can be executed in JS or WASM. Extending the V8 codebase has historically been so difficult that most people don't even consider it a possibility. With Deno's V8 bindings, snapshots, and core crates, anyone can extend the language ecosystem with relative ease. Programs operating in this model on the edge have already shown to be significantly more performant than their traditional server deploy counterparts. End-to-end latency is all but eliminated, until that edge program needs to fetch data from a datacenter. And for that, the "global state" problem is aggressively being worked on by the major edge providers (sorry Cloudflare KV, we're having a hard time relating with you). Once this has been solved, the web is prepared for some serious optimization.
If you still think "the edge" is a fad in 2022, I'd recommend spending more time learning about it. It's not just JavaScript. Ironically, this architecture is the solution to the web's JavaScript problem, and will be the demise of heavy clients and SPAs.
Similar to Chromium, Deno's permissions model is embedded in their CLI crate. Would love to see this get extracted into their core crates so that custom JS runtimes can leverage it more easily.
Javascript the language will be around forever, but I don’t know that I’d call it futureproof. Just because it’s there doesn’t mean people will continue to want to use it, particularly if wasm as a build target for other languages becomes more realistic and/or practical. However, javascript the technology (i.e., highly optimized and hardened runtime coupled with wasm) is pretty remarkable.
[1]: https://twitter.com/swyx/status/1521973694414864385?s=20&t=I...
Is cpu limiting also enabled by v8? Ie noisy neighbor?
This appears to be self contridictory. Putting things on the edge means downloading more code to the client and heavy client code and more logic on the client side, and things more like full-fledged apps (I'm avoiding saying Single Page App here because the concept of "single page" is related to website navigation). I don't see how this solves "the web's JavaScript problem", whatever it is you mean by that. Can you expound upon this part of your comment more?
Here's a web framework that is focused on this architecture: https://remix.run/
Here's a PaaS that is focused on this architecture with wide language support: https://fly.io/
Despite all the rhetoric that makes this claim, I continue to fail to see that server-rendered apps are somehow less interactive and have slower response time (both time to first byte and time to first interaction) than SPAs. I don't think I've ever seen a single page app respond faster and achieve interactivity sooner than an completely server side HTML templated site, but we say it feels faster for some reason. The Linux kernel cgit served interface¹ beats the pants off of almost everything else and it has way more data in its backing store (a git repo containing the Linux kernel) than many sites. Admittedly, there's little to no per-user custom state management there, but that's not something that gets removed by having the server closer to the user or only sending page patials anyway.
I feel like we spend, on average, more time staring at spinning circle gifs waiting for SPAs to request their data and document fragments than we want to admit.
(Yes, yes, I used some hyperbole here to make a point)
¹ https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
Definitely. I think that's the core problem remix is trying to solve (simple server rendering with the nice devex of our modern react toolkits). It will be interesting to see if the approach can works at scale and if the edge computing part of it remains a big selling point.
No. I'd wager it's much more likely _any scripting language but JavaScript_. Statistically speaking it's probably Python, though I wouldn't love that universe either, being a Rubyist.
Browser JavaScript has 30 years of technical debt in the form of the general inability to make non-backwards-compatible breaking changes lest half of web pages stop executing properly. This has been a boon for legacy code, but the language itself has languished for so long and in so many ways. It shouldn't be anyone's North Star, especially given a choice. Web Assembly gives us real choices, and I think it will eventually result in the end of JS's dominance in frontend when we all come to our senses and support improves a bit more.
JS now has: async/await, lambda arrow functions, generators, optional chaining, null coalescing, spread and rest operators, destructuring of objects and arrays, block scope with const and let, default values for function parameters, for-of loops, iterable interface that works in any loop construct, template literals, dynamic object literal keys as {[foo]: 'bar'}, and many new APIs such as Symbol, Map, Set, WeakMap, Typed Arrays, Workers, BigInts natively as ####n, and so much more.
Historically, the language has been progressive in having: first-class regular expressions as /.../, value-preserving non-boolean-coercing logical operators such as || and &&, closures, polymorphism of any flavor (even multiple inheritance) through prototypes, and so much more.
The language is concise, beautiful, and the reason so many people like yourself have an ill perception of it, is because you wrote or worked with codebases that had shitty JS code that looked like PHP spaghetti. That isn't JS's fault.
I've always written JS code into separate classes or modules. I've always found the language to be rather elegant and it's only gotten better in recent years in precision and conciseness.
We now have better standard library objects and better DOM methods. The syntax and actual operators are so much more powerful.
And lastly, it is fast as fuck for a JITted language.
Nitpick: it's fast for a dynamically typed language. It's not faster than, say, Java.
Perl 5 had already all those features in 2000 (Perl 5.6), or even earlier.
Python and Ruby are not in the same category IMHO, and while they are extremely good at what they are designed to do, I doubt that they could reach the lingua franca status js has. I would evaluate JS with PHP, Bash and C together, and I know how weird that sounds.
JS having it in in the browser, which is basically guaranteed to be on all devices and doesn't have to be setup before use is it's true advantage
> the language itself has languished for so long and in so many ways
When I read this, I thought "I wonder what they're talking about?" and then came up with numerous counter-examples of things that have improved immeasurably since I started writing Javacript. For example, there is now a `class` keyword with truly private properties. When I started writing Javascript, you had to manually manage the prototype chain.
But, I don't know what you're thinking of with that generalization. So it could be that there are many things that I don't notice anymore, but are still really crufty (like array reverse & sort mutating in place). Bringing those up specifically would improve the discussion.
Another example is the mess with for each loops, with .forEach, and the for..in syntax, and the peculiar behaviors therein where nothing does what you think it would coming from any other programming language.
Then you have things like "should I use var or let". It's embarrassing frankly. And don't even get me started with the uneven and confusing state of modules across the different ecosystems that exist now.
For someone who has worked in js since the pre ES5 days, it's fine, because we know the history and we know "oh, forEach is a legacy thing, don't use that if you can avoid it" but to newcomers it is extremely overwhelming having this sea of available functions and syntaxes and not knowing what to use. I know because I have had to talk through this with hundreds of coding bootcamp students as I used to serve as a mentor for one of the major ones. I tell all of them that the JS of today is a spaghetti soup of 30 years of tech debt, because it's true.
What's worse, is for newcomers, there is no way of viewing just the new/blessed syntax without going through each ES version and reading through the changes. It is very hard to distinguish the skeletons from the things you should use unless you know the entire history, and that fact is javascript's biggest smell to date. And it's also impossible to explain the new syntax in isolation without a story about what it replaced and why what it replaced fell out of favor.
JS has been adding new language features at an alarming rate, doing so poorly, and the reason for this is they have to walk on eggshells so as not to break existing web pages. Multiply that by 30 years and you get the mess that is everything since the netscape days. Things really got off the rails with ES6+, though, in terms of adding "awesome" new stuff without fixing/reworking old stuff. It's just plastered on in time-sensitive, fragile layers.
Prototypal Inheritance is considered one of JavaScript’s strong points, and many, including myself don’t think the class syntactic sugar that was added has much value. If anything it just confuses new developers.
A lot of JavaScripts detractors simply don’t understand the language very well.
Another language that has continuously accreted language changes is C++, and I've really struggled to learn it for that reason. It's so hard to find a "how to do it right" kind of guidepost for C++. Like modern JS, there's so many build systems to pick before you even get started...
Python is not popular because of it's lack of technical debt etc..
JS has added a ton of nice features while continuing support the skeletons in its closet. But you're free to ignore the vast majority of skeletons. You can block the old cruft using linters (e.g. preventing usage of `var`)
For example:
https://developer.okta.com/blog/2022/01/28/webassembly-on-ku...
https://training.linuxfoundation.org/blog/how-wasi-makes-con...
The only think I can find right now is Krustlet, which is not really what I'm looking for. I want something that I can run on my own hardware. Have you (or anyone else 'round these parts) hears of a WASM replacement for Docker in development?
We're looking for a WASM runtime that can handle orchestration of multiple assemblies on the same machine, I suppose: https://www.docker.com/wp-content/uploads/2021/10/Docker-Web...
EDIT: I found what we're looking for! Docker Engine is a container runtime. What we need is an alternate container runtime that will work with WASM. K8s-compatible options are described in the following article: https://kubernetes.io/docs/setup/production-environment/cont...
It turns out that both CRI-O and containerd can run WASM assemblies via the WasmEdge runtime in crun:
CRI-O: https://wasmedge.org/book/en/kubernetes/cri/crio.html
containerd: https://wasmedge.org/book/en/kubernetes/cri/containerd.html
And you can create WASM containers without docker hub images by controlling crun directly: https://wasmedge.org/book/en/kubernetes/container/crun.html
That is a pretty bold statement to that scripting languages allow business logic to be written faster and cheaper. I think that it depends completely on the size and complexity of your business logic, and on whether you factor in the cost of maintaining that software over time.
I'm sure dynamic languages were more productive than 90s-era Java, C, and C++, but I don't think those productivity claims hold today.
Maybe don't use Python where performance matters? Just a thought.
In many cases the largest cost indeed
Everyone likes s**g on JS/Node.js, but things are a lot more nuanced than this. In Javascript it's been normal that first there's some user-land fix to some problem, then it's implemented within the language core, and then things move over, including (not surprisingly) ESModules. The problem is that the way ESM was implemented was totally incompatible with CommonJS, making the transition a really painful one. On the other hand and TBF, Node.js innovation has slowed significantly and instead focused on stability, which is good to some but not to others (see the founder creating Deno with an implementation closer to the browser).
So what has happened is, effectively, what has always happened: Node.js ("userland" for the standard) introduces new concepts, then the ECMAScript body makes it a standard that it's similar but not the same, then Node.js has to change everything to adapt to that standard, and it's a PITA and leaves everyone scalded and burned out. Node.js in particular had to innovate a lot since it's running the same language in a different context, so all of these have followed that path: AJAX, fetch(), crypto, pipes, promises, request handling, Buffer, ESM/commonjs, etc. Most of the things we use day-to-day.
What you're describing is not just "normal", but the way standards works in the web space, both in theory and in practice, driven by both browsers and the standard bodies.
1. Developers/browser developers want to be able to do something in browsers
2. One engine implements it, developers start building with it
3. Second engine implements it
4. Work begins to standardize it
5. Both (all) engines starts moving from their implementation, to the standardized one
If the step after one of the steps doesn't happen (like only one browser implements a new API), the next step doesn't happen. Standards are not created for one engine, but if at least two engines have implemented something.
Javascript is a fine language for client-side software because the client is paying for it (and the client's computer, tablet or phone is running idle most of the time anyway)
Firms that run web services pay dearly to run their servers on the cloud, expect to run at a high utilization fraction, and if they ran their infrastructure on a slow language like Ruby they would be paying cloud bills 10x what they'd get for infrastructure written in Go.
The people who make the decision are paying the bills and that means they make a very different decision.
I have been writing back ends in Java and C# for more than a decade and it is widespread for people to take advantage of multi-core systems in two ways if they can: (1) threads sharing data structures such as system configuration and caches (e.g. it is no problem to have 10 or 100 megabytes of configuration data for a Java-based system) and (2) using Executors to split up tasks into smaller pieces and running them concurrently.
In Node.js, Python, and other GIL languages you can't do the above and slow down from "configuration at the speed of RAM" to "parsing configuration over and over again", "configuration at the speed of the database", etc.
I see people using Node.js for build systems but I think it's still an unusual choice (like Python) for a back end for a commercial system.
In regards to Node, as Python is anyway mostly CPython 99% of the deployments.
It would kind of seem like a wash to me, naively.
Let alone the detail that the goroutine is full blown native code, while the node/Python process is interpreted, and even if a JIT is used, many C2 level optimisations are out of reach for dynamic languages.
It is no wonder that even with the herculean effort that has gone into V8, for the ultimate performance it needs help from GPU shaders and WebAssembly, both typed.
The perf difference here probably isn't that great (although Java/Go will likely still be faster), but if you're at the point of running multiple processes you may well find that it's less dev effort to write your code in Java/Go (assuming you are familiar with both).
While it's never going to match compiled languages like C or Go, it's definitely the fastest interpreted language out there; I've seen studies putting it in the same order of magnitude as C (i.e it's 2x slower, not 10x). Moreover, its concurrency model (callbacks / event handlers) makes it uniquely suited to handle IO-bound workloads easily, which is 99% of the web nowadays (getting a request, sending a query to the database, and sending the reponse back).
According to a 2021 performance test¹, Lua with LuaJIT is slower than JavaScript with V8. Note that the "quite slow" JavaScript running on V8 is nearly as fast as Java.
¹ https://eklausmeier.goip.de/blog/2021/07-13-performance-comp...
JS is a single-threaded language, there is no concurrency
That said, I've had to eat my words recently on this -- serverless compute is just so damn easy to use and scale, it almost doesn't matter how inefficient the language you're running on it is as long as there are good ways of optimizing around cold starts (i.e. pre-warming), etc.. General efficiency of the backend language mattered a lot more when we had long-running web servers where memory leaks and stability issues would rear their heads inevitably after the server has been running for a few days. With serverless, these things almost no longer matter, because every micro VM is so short lived, there is no opportunity for these types of issues to arise. When it comes to raw performance, scripting languages _are_ sufficient if all you're doing is CRUD, in which case the main bottleneck is going to be your database anyway, especially if you are using a robust ORM like ActiveRecord which heavily optimizes the server side portion of this.
At Arist (YC S20) we are using Ruby on Jets in production, and it's quite efficient. We have a ~0.996 Appdex score, pages load typically within ~80-150ms, and many of our pages are server side rendered. I also operate a personal project that uses a Rust-based web framework. If I run this project on a raw EBS cluster (instead of serverless), I can get pages to load from the server in as quick as 20-40ms, but my point is that the difference visually is usually imperceptible -- 110ms is fast enough 99% of the time. When you factor in the fact that scripting languages, and in particular, Ruby, has significantly higher productivity than some of the systems languages, it becomes pretty obvious why a lot of startups stick with scripting languages.
All of that said, bring on the good and robust Rust and crystal web frameworks!!!
Similarly, caching is universal and is the answer to nearly all API issues. Raw compute is such a non-factor in a significant portion of use-cases.
JavaScript is totally fine for serving thousands of reqs per second. In many cases the bottleneck will be third party APIs or the DB. The cost of running that will be negligible.
If you have hundreds of thousands (or millions) of reqs per second, then of course cost can become an important factor. But at that stage you probably have the resources to build whatever you want with the best possible language for the use case.
Js has many tricks up its sleeve that cant be underestimated
They'll choose Ruby over C++ any day for purely financial reasons (obviously there are more consideration, but if it had been only about money).
Yeah ... I'm just going to have to go ahead and um disagree about that.
But I agree with the point that a higher level and well defined universal container/vm can be valuable. I think it'll be wasm-based though, not js.
> Instead of invoking Linux executables, like shell does, the JavaScript sandbox can invoke Wasm. If you have some computational heavy lifting, like image resizing, it probably makes sense to use Wasm rather than writing it in JS. Just like you wouldn’t write image resizing code in bash, you’d spawn imagemagick.
I agree with you all. WASM should be the solution to the problem in this space.
I do believe scripting languages are terribly useful for scripting the browser, a game, a tool, for writing glue code, small tools and use once code. But not for everything.
People should try to learn and use more than one programming language.
I remember I recently saw a lecture by an industry professional who took a Python algorithm and improved it 100 000x, in parts. That is, in stages he explained how to utilize the hardware on that particular machine, to improve the algorithm so much that it was hard to believe. Anyone have a link? I looked and I could not find it. It was a recent thing that I believe I saw on this site.
When it comes to configuring web services: It is extremely important to configure them in such a way that you can fully or partially cache content, reducing costs. I get it, if you are a startup you need to move fast, but with the way things are going these days, you should have someone on your team who can at the very least deal with the basics of caching.
You also cannot pre-initialize v8 as far as I know, so you inevitably end up putting it inside an emulator anyway, and guess what happens to the performance then? I could be wrong.
You can create a snapshot/image with V8 to reduce startup times (https://v8.dev/blog/custom-startup-snapshots).
Node already does this for some of its core libraries (https://github.com/nodejs/node/issues/17058). There are plans to expose this functionality to users so that any application and libraries can be snapshotted (https://github.com/nodejs/node/issues/35711).
I suspect it's this one:
Yes, as the article said - this is what WebAssembly is for. Shell : Executable :: Javascript : WebAssembly. WebAssembly supports wide SIMD instructions (https://v8.dev/features/simd) and many WebAssembly runtimes pre-compile the web assembly to native machine code (eg https://crates.io/crates/wasmer-compiler-llvm).
In a way, you could think of WebAssembly like a more portable LLVM bitcode - it’s a compact, partially optimized representation of a program ready for an optimizing machine-specific compiler to lower to the native architecture. But, it’s also possible to interpret it in contexts that prefer fast start up.
> I remember I recently saw a lecture by an industry professional who took a Python algorithm and improved it 100 000x, in parts. That is, in stages he explained how to utilize the hardware on that particular machine, to improve the algorithm so much that it was hard to believe. Anyone have a link? I looked and I could not find it. It was a recent thing that I believe I saw on this site.
> When it comes to configuring web services: It is extremely important to configure them in such a way that you can fully or partially cache content, reducing costs. I get it, if you are a startup you need to move fast, but with the way things are going these days, you should have someone on your team who can at the very least deal with the basics of caching.
> You also cannot pre-initialize v8 as far as I know, so you inevitably end up putting it inside an emulator anyway, and guess what happens to the performance then? I could be wrong.
> The awful performance you can be stuck with when you can only use emulators and v8 is going to come back to haunt some of you, I believe. It is indeed easy to sell something like this, as it looks easy to work with, the same way that no-code looks easy. But inevitably even native performance is not nearly enough and you need to employ algorithms that use 256- and 512-bit vector operations. At that point, we are talking about running natively on the CPU, and for that you need something that isn't just a web app.
> I remember I recently saw a lecture by an industry professional who took a Python algorithm and improved it 100 000x, in parts. That is, in stages he explained how to utilize the hardware on that particular machine, to improve the algorithm so much that it was hard to believe. Anyone have a link? I looked and I could not find it. It was a recent thing that I believe I saw on this site.
> When it comes to configuring web services: It is extremely important to configure them in such a way that you can fully or partially cache content, reducing costs. I get it, if you are a startup you need to move fast, but with the way things are going these days, you should have someone on your team who can at the very least deal with the basics of caching.
> You also cannot pre-initialize v8 as far as I know, so you inevitably end up putting it inside an emulator anyway, and guess what happens to the performance then? I could be wrong.
> The awful performance you can be stuck with when you can only use emulators and v8 is going to come back to haunt some of you, I believe. It is indeed easy to sell something like this, as it looks easy to work with, the same way that no-code looks easy. But inevitably even native performance is not nearly enough and you need to employ algorithms that use 256- and 512-bit vector operations. At that point, we are talking about running natively on the CPU, and for that you need something that isn't just a web app.
> I remember I recently saw a lecture by an industry professional who took a Python algorithm and improved it 100 000x, in parts. That is, in stages he explained how to utilize the hardware on that particular machine, to improve the algorithm so much that it was hard to believe. Anyone have a link? I looked and I could not find it. It was a recent thing that I believe I saw on this site.
> When it comes to configuring web services: It is extremely important to configure them in such a way that you can fully or partially cache content, reducing costs. I get it, if you are a startup you need to move fast, but with the way things are going these days, you should have someone on your team who can at the very least deal with the basics of caching.
> You also cannot pre-initialize v8 as far as I know, so you inevitably end up putting it inside an emulator anyway, and guess what happens to the performance then? I could be wrong.
I also wonder about applications where WebAssembly isn't the most performant thing — image resizing, one of your examples, is also an example of a situation where you get significantly better performance with SIMD instructions. The experiment was tried here:
https://www.libvips.org/2020/09/01/libvips-for-webassembly.h...
addEventListener("fetch", (event) => {
event.respondWith(new Response("Hello world"));
});[0]: https://developer.mozilla.org/en-US/docs/Web/API/FetchEvent
However there is one big caveat - it's super easy to mess things up with it and it is harder to test and debug (Chrome is the best tool for this IMO as a fan of FF). You better have a very straight forward way to update your service worker right off the bat.
I thought the WWW was already here
Probably should say "the World Wide Web will STILL be here in 10 years."
what happens to Lua then?
You can cat business logic to /dev/null. That's even faster and cheaper.
Correctness? Reliability? Speed? Extensibility?