Perspective: Streaming Analytics via WebAssembly | Hacker News Reader

Perspective: Streaming Analytics via WebAssembly

perspective.finos.org

36 pointsArtWomb10 comments

Are you able to avoid going through HTTP requests and use directly UDP for streaming when using WebAssembly? Or the only advantages of WebAssembly are the improved client-side rendering and processing performance?

In that case, if client-side processing speed is important, wouldn't it be a lot more important and faster to render the output using OffscreenCanvas than by frequendly updating DOM elements?

Currently, WebAssembly doesn't can't directly call any of the browser APIs; it has to call back into javascript to make any network requests, make WebGL calls, read to/write from the DOM. Projects like wasm-bindgen make it look like you're directly calling browser APIs, but it's actually calling out to a Javascript shim.

I think this project's use case is that you have a fixed query, but the dataset over which you're running that query is changing over time (getting streaming updates). The wasm part is doing the work of running the query over the dataset and updating the query results as new data streams in. So the assumption is that rendering isn't the bottleneck, but running the query is.

So the dataset would be a local dataset (that gets frequently updated from the network) and the query is a local query that runs over that dataset?

I still am a bit confused about the use-case.

Oh, I still I'm starting to get it.

Is it if you have a large dataset locally stored (eg. in JS memory) that receives constant updates from a server? Are there any example apps that have this architecture?

Any kind of pivoting/roll-up over streaming updates (financial trading, IOT, transportation). A lot of streaming dashboards across a number of industries are effectively just pivot tables over ticking data.

WebAssembly cannot utilize UDP, or any other APIs not exposed by the Browser - just HTTP and Websocket in this case. However, Perspective can read Apache Arrow using the C++ client library without decoding the data structure to an intermediate JS representation first, which in our benchmarks has a substantial impact on overall update throughput.

I think WebAssembly System Interface will address using other protocols.

https://github.com/WebAssembly/WASI

Seems neat but doesn't seem to work in the latest Firefox (though I do have a hardened user config, I don't think its relevant here). The error showing up in the console is "SyntaxError: invalid regexp group".

Apologies, this was fixed a few days ago but hasn't made it into a release yet. https://github.com/finos/perspective/pull/1065

Ugh, went back to HN to post the same comment.

On FF this usually means that the regexp uses `lookbehind`: https://caniuse.com/#feat=js-regexp-lookbehind

Reply on news.ycombinator.com