krausest

Feel free to post any issues to the github repo and I'll try to help. Building is indeed not trivial, since there are so many languages and tools and thus builiding is currently only fully supported on linux (rather a docker linux build). With a few exceptions building on windows and OSX works. A simple workaround is to delete the folders of those implementations that cause errors.

The startup benchmark tries to capture that, but doesn‘t simulate network performance. Still I think it shows some interesting results.

Is your question what is keyed (answered in the blog) or what‘s vanillajs (javascript without a framework)?

That would be great. Just send me a pull request!

I also hoped to see a bigger improvement, but it turned out that Javascript is already very fast for nbody. Maybe I should have picked a well known numeric benchmark where Javascript is still far behind - any suggestions for that? Or are the Javascript VMs already too good for numeric benchmarks?

I didn't. I believe that is's better to run a few times and take the best run. I payed for the turbo mode CPU and I'd like to know the performance on my machine. There's a lot going on my machine, and even more when a browser is running. The only thing (besides running on a clean machine) one can do about it is to measure multiple runs. Three runs are maybe a bit to little for scientific results, but I considered them good enough for a casual benchmark.

I added riot 2.5, but I need help. Can you assist? https://github.com/krausest/js-framework-benchmark/issues/13

Running all the benchmarks takes a few hours.

I'll keep that in mind for the next round. Let's see if that works nicely. Would you still drop outliers?

You're basically right. Thus we've decided to perform 10 runs for each benchmark and drop the worst 4. This eliminates cases where the gc or a background system process causes a slow run. In consequence this also strongly reduces variance, but is blind on the gc eye. It would be interesting to include gc, but I have no idea how to do it in a real good way.

The results are comparable (but nowhere equal) between runs and the difference between the frameworks is in most cases large enough that the ranking stays consistent (but you wouldn't prefer preact to cycle.js v7 for having a 0.01 better result in the average slow down, would you? For ember, mithril and cycle.js v6 performance might be something to consider depending on your use case.).

If you look at the console output this is just an approximation, the real measurement is performed on data from chrome's timeline using selenium. The console measurement uses a timer to guess when rendering was performed. Of course this is more inaccurate than using the timeline.