Comment by vanderZwan | Hacker News Reader

Comment by vanderZwan | Hacker News Reader

Parent Full thread

> Using Rust and Event Sourcing for instant UX over 150,000 records — far past where JavaScript crashes with a stack overflow error.

Nit: while I fully support making the web more efficient and hope that Nue is successful in promoting that, I'm skeptical of the "crashing" JS claim here. You can do amazingly efficient things with TypedArrays, the forgotten middle child between JavaScript and WASM.

Having said that, this requires building a few clever data structures on top (I've gotten a lot of mileage out of "typed" dequeues, and (sparse) bitmaps myself), often emulating structs in a language unable to express them properly. So the resulting code is not exactly idiomatic JavaScript — which I presume is what they implied here. Plus in this case it probably still wouldn't be as fast as a well-written WASM implementation, since almost all of the work is likely spent searching with only a little bit of DOM manipulation at the end.

So I'm both agreeing and feeling like it slightly overgeneralizes. I guess my point is: yes, let's be critical of bloated web frameworks, but let's not equate React with JavaScript either.

Author here. You’re spot on. I took away the React association and added a direct link to the JS code [1] that hit the ‘Maximum call stack size exceeded’ wall. TypedArrays can do wild stuff, no doubt, but yeah, it’s not exactly vanilla JS—and still lags a tight WASM setup. Appreciating the nuance here!

[1]: https://github.com/nuejs/nue/blob/master/packages/examples/s...

The new wording is much more nuanced, appreciated! Also the fact that you link to the actual code that broke down, making it possible to verify the claim. Less ammunition for the people who actually are skeptical of the project instead of just nitpicky like me ;)

Speaking of the code in question, it looks pretty sensible - there's a bit of low-hanging fruit where one could avoid creating redundant arrays by turning chained calls to map and filter into one for-loop and such, but that's about it.

What confuses me is that there's no point in the code where I see any recursion, here or in the other JS files that seem relevant, so how does one end up with a call stack overflow?

(not questioning that it crashed for you, it's just that I don't see any obvious flaws in the implementation, so I'm curious to lean what the "gotcha" is in case I ever bump into something like it myself)

The stack overflow is caused by an `arr=[...]; events.push(...arr)` in `add_events`. Replacing that with `for(const x of [...]) events.push(x)` gets rid of that issue, and the JS-backed build is then snappier to search & filter than the WASM/Rust version.

Oh duh, yeah using spread syntax for function calls is definitely limited to fewer than 150,000 arguments in any browser that I know of. Don't expect everyone to know that, but I sure did, stupid that I didn't spot that. Thanks for pointing it out!

(funny enough I tend to use for(let i = 0; i < arr.length; i++) loops most of the time anyway because the iterator protocol adds too much overhead for my tastes, so I wasn't likely to ever bump into this in the first place)

I would imagine browsers optimize the iterator protocol away for arrays. At least on https://jsben.ch/iJFZ4 for..of is a smidge faster than an old-school loop.

You accidentally included the array initialization in the benchmarked code ("setup block" vs "boilerplate block"), dwarfing everything else. If we're talking about an already allocated array the difference is pretty big:

https://jsben.ch/FgDA6

… but in a way you're correct that this is rarely the actual bottleneck in the surrounding code. Still, "death by a thousand papercuts" and all that. Plus having old-school for-loops as a habit makes it easier to spot the true botllenecks.

... derp, yes. It was way too early in the morning, and I misread "is part of the benchmark" as "is not part of the benchmark"...

In your defense, the "use it for data initializing" suggestion isn't exactly helpful text either.

Also, for the record: I presume you tested on Chrome, but on Firefox the difference between the two loop styles is quite a bit larger (and generally slower than Chrome, so perhaps also a consideration when optimizing for "slowest browser" as the bottleneck).

What does event sourcing have to do with searching or "instant keypresses"? It's a storage pattern for recording events, not a way to search.

Like, you're searching a single table for text messages, not a stream of events.

Reply on news.ycombinator.com