How I built a fast JavaScript framework | Hacker News Reader

How I built a fast JavaScript framework

medium.com

45 pointskrapans61 comments

I am often frustrated in this ecosystem by how people build things without any clear conceptual model, and then they document them by saying "it is so fast, it just works like magic". It is impossible to compare two approaches because there are no facts available to compare them on.

Like for this one from a peek at the source code it appears to try to parse JavaScript at runtime using regexes (https://github.com/radi-js/radi/blob/master/src/index.js#L4) which is great for demoware and not at all something I'd want to use in production, but that is not mentioned anywhere in the site or blog post.

I don't mean to just dump on this project in particular, which could just be a fun toy side project for the author, because this problem is endemic. (But then why go to all the effort to make all the marketing materials for it without actually writing any docs?)

Looks like the regex is only used to strip comments?

https://github.com/radi-js/radi/blob/eace84e9b74c944b233f631...

It parses the source code of the view function at run time, matching parentheses, but ignoring whether those are present in strings or regexes. Likewise, the comment stripper will turn

    "/*whoops*/"

into the empty string...

In other words, the parser is buggy :-/

Having taken a closer look, it looks like the problems go even deeper than that: it rewrites `l()` calls in functions through a regex/replace on `fn.toString()`. This is similar to the mechanism that broke shorthand Angular.js DI in production. So code can break if it goes through some bundler than renames the `l` variable, or if a minifier does it, etc.

I'd imagine closures also break.

Closures will break, yes. It looks like it is intended to react to mutations to the properties of `this`...

I hadn't thought of minification, yes it pretty bad :-/...

It looks like the author is smart but has little JS experience (`var last = temp.splice(-1)[0]` where `temp.pop()` would do tells me that it may be one of his first JS project).

The line I linked to, sure. But under what conceptual model would a framework need logic to strip comments (an aspect of source text) at runtime? Follow it to its conclusion.

They compared Radi to React Stack, not React Fiber. That's being sneaky.

If you run the React Fiber demo, you'll see it's just as fast: https://claudiopro.github.io/react-fiber-vs-stack-demo/fiber...

I'm not primarily a Javascript developer, but I seem to remember that attaching event listeners to Object fields and modifying the DOM in event handlers is how Javascript development was done 5-10 years ago. Am I missing something? Are we going full circle?

In some sense, I feel like the big frontend frameworks are very useful for large apps but they became the default for even the most basic interactions. Now some developers are realizing that for their basic needs it is simpler to go back to the basics. For ambitious SPAs I still think something like React or Vue is the way to go.

> In some sense, I feel like the big frontend frameworks are very useful for large apps but they became the default for even the most basic interactions.

Exactly, I'm using react at work when working on a huge web app and jquery at home for my little side projects. It's so nice not having a build step for my side projects. While at the same time it's really nice not using jquery in a project with dozens of developers.

Yup. I find that most cases of "JavaScript fatigue" are really cases of "I want tools custom tailored to my use case, but don't want to do the legwork to figure out what they are".

The only thing wrong with jQuery is that it's perfectly fine until it isn't. Conversely, if you'll never need React's power you don't need it's complexity either.

React's (or other SPA frameworks like Vue's) complexity is much higher than it should have is probably because of the tooling (Babel, Webpack, NPM etc) and additional build process required even for a simple hello world app. That coupled with the rapid revision of the underlying ECMAScript standard and the race by developers and framework authors to adopt them causing further chaos.

Another issue was the rapid deprecation of age old JQuery plugins and the lack of equivalent plugins in the React landscape causing confusion whether you should build it yourself or not etc. I had hoped WebComponents somehow got standardized as the underlying architecture for SPAs, just so that the plugin/widgets wont get fragmented further, but it looks like it wont be happening anytime soon.

> React's (or other SPA frameworks like Vue's) complexity is much higher than it should have is probably because of the tooling (Babel, Webpack, NPM etc) and additional build process required even for a simple hello world app.

I wouldn't frame it as excess complexity, I would frame it as power. My problem is not writing simple hello world apps, it's writing and maintaining very large production applications with 5 other developers. Two years ago I took a week to really learn the tooling stack and it's paid dividends ever since.

We are 10 developers just using vanilla Javascript here. No mess, and a joy to work with. We use good old design patterns like observer, proxy, composite and so on. We also use inheritance and template literals a lot.

The key is good documentation, having an onboard plan for new developers, keeping things stupid simple and stop being bleeding edge. We want our code base to just work, so we can focus more on new features and less on maintenance.

I'm planning on writing a deeper blog post this year about why and how we feel is a good structured simple way to do frontend development. I believe there are several frustrated frontend developers out there that just want to make their life easier but currently all the arguments they find just favor 'React'....

Our whole Javascript codebase is 50k + lines. But the clients only fetch what is needed.

> joy to work with

Color me skeptical. React is quite literally the only framework I've ever found to be enjoyable (and maintainable).

This is false or misleading. There is complexity, yes, but those tools abstract it away and simplify everything. Then they have configs with great documentation for when you need to deviate from a default or add a plugin. Next, these things are pre-configured for you. "More complex than it should be" is when you try to reinvent the power of these tools for literally no coherent reason.

> "I want tools custom tailored to my use case, but don't want to do the legwork to figure out what they are"

I feel this quite often. It doesn't help that the tools move so quickly that once you develop a kind of gut feel for what frameworks work in what situations its already changed and you get a bunch of "lol [X] was 2 years ago we're all on [Y] now".

Dunno, but it wouldn't surprise me. The resemblence between tables and flexboxes is also quite funny. We took 20 years from declaring the structure of table-like markup in a table to declaring it in CSS with a flexbox.

Cool, but things like this have a marginal positive effect for developers, at the cost of having to learn yet another thing, and making it harder and harder to make a reasonable webpage.

Wait till you take a look at CSS grid

https://css-tricks.com/snippets/css/complete-guide-grid/

Basically a table but instead of HTML elements, using CSS properties? I loathe how "let's move the concept from one separation layer into another one, or better yet - let's mix the concepts together in the same separation layer" is becoming the next big thing recently.

CSS Grid does the opposite of what you claim – it fixes the mixing of concepts that previously tied HTML elements to their layout, not make it worse.

table, td, tr, etc are perfectly valid and semantic HTML elements. Are HTML5's footer and header elements broken as well, shall we expect "footer-" and "header-" CSS properties in 5 years?

I have no idea why you keep bringing up <table>. It is totally unlike CSS Grid and completely unrelated in every way.

CSS Grid is for layout, not tabular data. It allows you to move the presentation of your elements around at different breakpoints regardless of your markup structure.

In general I recommend understanding a feature before you spout off nonsense about it.

> I have no idea why you keep bringing up <table>. It is totally unlike CSS Grid and completely unrelated in every way.

I disagree. Conceptually, it is exactly how you would do layout in the old days. This still works, btw, it's just considered bad practice.

It's a bit like saying that a kitchen knife is totally unlike a dagger and completely unrelated in every way, because the former is for slicing food and the latter for stabbing.

I'd say it's more like comparing a horse to a car:

Horses were never really "vehicles." We just misappropriated them that way, because they just happen to be big and move around and we can sit on them. Cars are purpose-made to be vehicles for humans.

If a designer shows you their wireframes with their grid lines drawn on top of it, I don't think many people would think "ah! so it's like in a science paper when they're printing the raw data from their experimental results in tabular form."

The fact that some web developers consider them alike is a historical accident.

I think the fact that, in order to "move fast and break things", we left behind the standards development process, is the saddest part. It's totally true that the standards processes (W3C) often got bogged down or lead us astray. Freedom to experiment is good as well.

But the fact that we have become dependent on frameworks that solely exist in JavaScript and external tooling, created by mainly large organizations, using techniques that work around the basic browser behavior, had me worried for a long time. I say that as someone who has been building websites and web applications since around 1997.

The flip side of this is, without all that experimentation being done by people outside of the standard process, perhaps discoveries like this one (how to do something similar using essentially standard tech) wouldn't be found. And the progress we made before the latest round of shadow and/or virtual DOM and SPA applications is what enabled us to get to the point where we could fully conceptualize them.

So there you have the interplay between the community, industry, de facto and formal standards in a nutshell. There is good and bad, especially when you are in the middle of massive technology shifts.

I for one welcome rolling some of the advancements that we have discovered into formal standards that become built into and fully supported by browser vendors. I welcome a return to "how can we simplify" where we've gotten to and roll that into the standard realm more.

And personally, if we can simplify the use of external tooling as much as possible... I think that would be a huge boon. That tooling is, I believe, going to be extremely brittle over the long run and is a risk factor.

I wish the framework developer the best of luck and thank them for looking back at what we could already do in an attempt to simplify!

Btw, I think that what I am talking about is basically the process of "exnovation" [1].

[1] https://en.m.wikipedia.org/wiki/Exnovation

I'm not a frontend developer and don't know really know JavaScript frameworks. That said, it seems like something very well established like React would choose to have a virtual DOM for a good reason.

Surely there must be some tradeoffs that the author makes for his design to get better performance. Could somebody explain the cost/benefit of doing things one way versus the other?

Virtual DOM was invented not to make things faster than with jQuery, etc., but to make a better and less error-prone style of web app development not be intolerably unperformant.

I think if we could see a larger Radi example, say, the standard TODO app, the costs of its approach would become obvious.

That's not true. Virtual DOM was created to avoid recreating huge DOM elements each time a re-render is needed, which was the way apps used to be done in the days of jQuery.

Eh, not exactly right either. In the jQuery days, the problem is that you needed different procedural code paths to go from one DOM state to another depending on what action you took. For example, adding a row to a list would be `$('.list').html(html)`, removing would be `$('.list :last-child').remove()` and updating one item might be `$('.list #' + id).text(value)`.

Templating engines such as handlebars provide a declarative abstraction that represents how the DOM is supposed to look like given some rules, rather than procedurally specifying how to get from A to B. This is similar to how you write HTML declarative as opposed to how you have to write canvas code procedurally.

Virtual DOM is just an implementation detail of some templating libraries that have the same goal of providing that declarative abstraction. The problem was that popular KVO systems (e.g. Knockout and Ember) were notoriously slow and people were looking for alternative ways to speed up declarative rendering libraries. The community has since learned a lot about performance and is realizing that KVO systems _can_ be performant after all.

The main drawback of KVO systems is that their API bleeds into the data layer because they require you as a developer to manually register data as observable entities. In contrast, virtual DOM works with plain Javascript objects which are much more familiar and easier to manipulate and interop with. A third approach, called dirty checking and popularized by Angular also had the desirable property of working w/ plain js objects, but it was also very slow. Some systems (most notoriously Vue 1) tried to bridge the gap between KVO and POJOs by providing a POJO-looking data layer that has monkeypatched array methods etc that provide observability/reactivity under the hood. Without JS proxies, though, this still left some weird edge such as the ones handled by `Vue.set`/`Vue.delete`. Vue eventually moved to vdom because the way it mounted subcomponents didn't scale for recursive mounts (think comment trees such as the ones in HN or reddit). This is because it had to mount each level to find out what subcomponents needed to be mounted, and as well know, multiple repaints is a big no-no for perf.

With that said, virtual DOM is not perfect either. It's inherently memory intensive and can block the UI in cases with large DOMs and few changes. Various frameworks have various techniques to cope (such as shouldComponentUpdate and friends) but they typically require developer intervention, whereas a KVO system would not suffer from this class of problems to begin with. Another problematic aspect of virtual DOM systems is that they are "naked" js and thus, difficult to optimize as libraries from an algorithmic standpoint. This is why Svelte does better in benchmarks than vdom systems.

TL;DR: if you want to evaluate the performance characteristics of a KVO system such as Radi or Surplus, the main things you should ask about are API lock-in in the data layer and performance of recursive mounts. If you want to evaluate perf of virtual DOM systems, you should ask about diff latency for needle-in-a-haystack scenarios and ease of use for hooks to selectively diff a tree.

> It's inherently memory intensive

Even if we pretend that other architectures doesn't have any memory overhead, vdom memory overhead in a large UI app will be less than a single small decoded image. But in reality, KVO can be way much more memory intensive, especially when we consider cases that involve filtering items in large collections.

I was reading this thinking, gee, here's someone who knows a lot about JS frameworks. Only later noticed the username. Hi Leo!

Surplus [1] is the fastest framework in the big JS benchmark challenge, and it too uses the native DOM directly rather than a virtual DOM. Looking forward to seeing Radi.js added to that benchmark suite to really put it to the test.

[1] https://github.com/adamhaile/surplus

The problem with not using a virtual DOM is that you can't merge in changes to stateful DOM elements. For example, an INPUT field has state (its value, plus its scroll position). If you regenerate the INPUT field from scratch, you risk losing the state. In contrast, if you use a virtual DOM, the vdom mechanism can merge in any changes for you (the INPUT field will stay the object!)

Of course, you can recreate DOM elements with their complete state, but if you'd try to do that with e.g. a VIDEO element, then you risk flicker and reloading of the actual video.

This isn't quite correct. Modern rendering libraries aren't going to recreate an input from scratch unless they absolutely have to, vdom or not.

What's tricky about reconciling app state changes and dom state is dealing with things like cursor position if e.g. the input value change comes from a socket and the user had the input focused and the cursor in the middle of the text. But this has nothing to do with vdom

To prove you wrong, consider how DOM elements are created (remember in Surplus there are no virtual elements). Ignoring all dependency tracking, the code to show some text has to be (eventually) of the form

    x = document.createTextNode(v);

Now, consider the dependency v changes. The only option is to run the code above again, in its entirety. And thus, the DOM node corresponding to the text is new.

Of course, a smart library will ensure that the amount of nodes that has to be visited is at a minimum. But that doesn't mean DOM nodes don't get rebuilt from scratch.

> The only option is to run the code above again, in its entirety

Not at all. The reactive pattern typically boils down to something like this:

    el = document.createTextNode(initialValue);
    parent.appendChild(el);
    reactiveValue.onchange(v => {
      el.nodeValue = v;
    })

Ironically, in this specific example, vdom libs typically do this when possible:

    parent.textContent = v

which does in fact replace the underlying text node, but they do so for the sake of performance

Yes, you can code like that, but ultimately that block will be contained inside a closure that will get triggered by some change. Of course, unless you are very careful.

The element creation part will be triggered if a creation is required. It won't be triggered by a prop update, for instance.

In the following example, I change the button id on click. It's still the same button element after the change:

https://jsfiddle.net/dwn36aw4/8/

But it's very difficult to prevent unnecessary dependencies from triggering in all cases. Consider a map (dict) with some values. Some outer code depends on the map. This code builds some DOM elements and sets values based on the values in the map. Now, if the map changes, all the elements will be recreated, even if effectively some irrelevant part of the map changed. Of course you can prevent this from happening by doing smart diffing on the map, but now you've effectively moved the problem from dom-diffing to map-diffing.

I wrote about that (and the drawbacks and alternative takes) here: https://news.ycombinator.com/item?id=16540223 but yes, in a nutshell, "move the problem from dom-diffing to map-diffing" is basically what a reactive system is supposed to be taking care of.

You're absolutely right that reactive systems are not trivial to implement (let alone implement well), but inefficient DOM recreation under these systems is a symptom of poor granularity in the reactive layer (which may be a problem with the reactivity library just as well as it could be poor usage of its API), rather than a limitation of the rendering library itself.

Ok, that makes sense.

I'm also worried about debugging accidental dependency triggers. Imagine that suddenly the scroll-state of some div is reset; how do you determine which dependency caused the DOM element to be recreated? With a vdom you avoid the whole problem.

> how do you determine which dependency caused the DOM element to be recreated

Normally, I would toss in a throw and look at the stack trace. But yes, the stack trace gets noisy as hell.

what you discribe is basically what https://github.com/google/incremental-dom is all about and in certain cases it probably is as fast as vdom and also less memory heavy.

I don't see why this would be any more difficult without a vdom. For the textarea, either way you're merging two text blobs, and if the vdom can maintain scroll position while doing this, then so can the non-vdom using the same mechanism.

Let me try to explain. Let's say you have a textarea element which has DOM-state (value, scroll-position). And let's say you change the value through your render-framework. With the vdom approach, only the value component of the DOM-state is modified. With the non-vdom approach, you're creating a new textarea element, and set its value. Its scroll-position is lost, unless you keep track of it.

Now with VIDEO elements, things are more problematic. If you recreate parent-nodes of a VIDEO element, the video will stop playing, or will flicker, or jump back to the beginning.

> With the non-vdom approach, you're creating a new textarea element, and set its value.

This isn't necessarily true. In fact, it's not true of Surplus and Radi.js, which is partly why they're much faster than every vdom framework out there.

They're faster because they are not computing DOM (or vdom) nodes when that's not needed.

But they're creating new DOM nodes when values-on-which-the-code-depends change.

What primarily makes surplus fast isn't necessarily that it doesn't use a virtual dom, but that changes are tracked with a data reactivity library reminiscent of knockout called s.js instead. On the DOM side, it has a compiler like svelte that generates JIT friendly expressions, e.g. el.className = val instead of el[prop] = val.

> but that changes are tracked with a data reactivity library reminiscent of knockout called s.js instead

Surplus is fast because it doesn't have many features that available in many other libraries. For example, component model with lifecycles.

In this benchmark, "select row" should be the best case scenario for library like this, but even in such scenario it is slower than some vdom libraries.

Lifecycles are only needed because of the vdom, it's not a desirable "feature" since it overcomplicates a conceptually simple process.

Lifecycles has nothing to do with vdom. Even web components has lifecycle hooks https://developer.mozilla.org/en-US/docs/Web/Web_Components/...

Have you tried to build a components library, or have you seen any component libraries that is built on top of a framework that doesn't provide lifecycle hooks for components? Something like https://ant.design/docs/react/introduce or https://developer.microsoft.com/en-us/fabric#/components

The term "lifecycle" in web components has a completely different meaning compared to its use in vdom implementations. Compare the semantics of "lifecycle" events in web components against React: https://reactjs.org/docs/react-component.html

They have virtually nothing in common. Web components are essentially just new DOM events, albeit more restricted.

I started this thread by mentioning Surplus, which has no lifecycles because it operates directly on the DOM, and so React's lifecycle callbacks have no use.

> They have virtually nothing in common.

If you think so, then there is really nothing to discuss ;)

Lifecycles are important if you need to do some arbitrary DOM manipulation upon mounting an element to the DOM tree. For example, you may want to render a google map when navigating to some route.

"Overcomplication" is in the eye of the beholder. You can't tell your client that his wanting a map in the contact page is "overcomplicating a conceptually simple process". At some point, if a system's restrictions are too strict for the sake of simplicity/beauty/etc, then it ceases to be useful in the real world.

Does S.js keep track of dependencies that are no longer needed? Is there some sort of garbage-collection implemented?

This looks similar in many ways to mithril[1] -- somewhat similar api - mount, view, r("input") vs m("input"). The counter example[2] looks similar[3] I'd be interested in seeing comparisons between the two.

[1] https://mithril.js.org/

[2] https://github.com/MithrilJS/mithril.js/blob/5956314e3655a3c...

[3] https://github.com/radi-js/radi/blob/master/examples/counter...

Hi, Mithril author here.

In terms of API, frankly all the frameworks are more or less similar. Mithril has been around for a while, so obviously all the common observations apply (i.e. it's been more battle tested, it has a more mature community, more libraries, etc)

Taking a quick glance at the Radi.js code, I noticed a few things that could still use some improvements (e.g. the comment regexp someone else had pointed out, the handling of attributes not accounting for things like SVG, etc)

I didn't see support for lifecycle methods in the Radi.js code, and it's not clear to me from the docs whether keyed lists and fragments are supported. Keyed lists are extremely important in cases like `someArray.unshift()` and any list containing stateful dom (e.g. inputs or link tabindex). Lack of fragments would not be a deal breaker, but IME they're super nice for building lightweight abstractions, especially in Mithril's case, where they can have lifecycle hooks.

Inside, the difference is that Mithril uses a virtual dom.

Outside, with Mithril you don't need the l() function to bind attributes to vdom nodes. I also suspect there are things this framework can't do that vdom frameworks like Mithril can.

Aside, this framework's author seems better at marketing than the Mithril community. Notice the multiple comments on this article lamenting how React's size and complexity make it less suitable for small projects even though it has advantages for large ones. Mithril is suitable for both large and small projects, but you'd think it didn't exist.

I recently did a project with morphdom (https://github.com/patrick-steele-idem/morphdom) using only ES6 template strings and a render loop that checked for changes to a global state object and patched a re-computed DOM string using morphdom. Couldn’t have been an easier and more pleasurable experience.

The reason diffing and virtual doms are used are to prevent unnecessarily repainting the browser window 1000 times a second. If I have 5 updates to make, I want to do those in one paint, instead of 5. So the process is change -> change -> change -> paint.

With this framework's model, the process would be change -> paint -> change -> paint etc. etc. Sure, you get nice FPS (because it's painting a LOT), but that will destroy battery performance on mobile phones and I don't need 60 FPS in my shopping web app.

Reply on news.ycombinator.com