To Jupyter and Beyond: Interactive Data Science at Scale with OmniSci | Hacker News Reader

To Jupyter and Beyond: Interactive Data Science at Scale with OmniSci

omnisci.com

80 pointsdomoritz3 comments

Looks very interesting. But when would I want to use OmniSciDB and OmniSci charts? I am not a working data scientist so my understanding here is limited.

OmniSciDB is useful when you need really high-performance from a relational database. Think <100ms query times over billions of records. This level of performance is useful for streaming use cases, operations, repeated drilling into subsets of data, etc.. Note that OmniSciDB refers to the open-source relational database portion of our product.

Using any of the OmniSci open-source charting libraries is useful when you want to do something using only OmniSciDB (i.e. not wanting to purchase Enterprise Edition), or when you want to create a custom visualization that isn't provided by OmniSci Immerse (our commercial visualization tool). Some of the work referenced in the blog post was in partnership with the creators of the Vega.js library as research into high-frame rate and large data interactivity in browser visualization

RE:Streaming, Graphistry wrote much of the Apache Arrow JS lib to optimize for this case of streaming GPU DBs -> GPU browser frontends, you can see a pretty striking before/after here: https://www.graphistry.com/blog/experiencing-the-future-of-g... . JS networking devs can handwrite that stuff.. but it stinks, and this way, everyone benefits. I recall Vega folks had some fun experiments with GPU DB -> CPU frontends here (ended up part of an HCI or VLDB paper?), not sure what happened since. Our production focus on that layer is now more on scaling up node js cpu+gpu code <> pydata GPU code (Nvidia RAPIDS).

For OmniSci folks: Cool to see Jupyter embedded! Our more advanced users love this path, but we find versioning & team collab tricky in big enterprise settings, so every additional company and (hopefully) contributor here should help. Getting there :)

Reply on news.ycombinator.com