NeroVanbierv

A very good suggestion and something we need to look into. Thinking about it, the techniques will probably be similar.

Python + React in electron

Sorry for the huge delay, I missed your comment. Thanks for the feedback! - just to confirm: this means your screen is has a height < 800px? - do you remember what exactly went wrong on windows? - sounds very weird behaviour, do you have a screenshot?

Thanks!

(Mbaert his comment, but his account is throttled) We were worried about the performance of Electron as well. All data intensive work is done in a separate process, once in react/electron no data manipulation is needed anymore. So far it works quite ok, performance upgrades we find in electron usually don't have anything to do with the data.

Thank you for the info, we're still finding our position in the market. Figuring out the perfect use case and integration in a certain workflow will be key indeed! Feel free to contact us in private if you like to discuss more in detail about your experiences, we'd love to hear more :)

On your own laptop for now. This has the advantage that your data is already there on your local disk - no need for uploading. But with some minor changes we can run it in the cloud too

Which ones are broken?

Thanks, cool question. The insane performance gains we have when visualising are mostly because of conceptual decisions we made. One such thing is that we don't try to plot more datapoints than there are pixels on the users screen. But when a user does calculations on data (e.g. a low pass filter), we really have to do all the math. So in that case we are mostly using numpy (python) operations, which are optimised in C already. But I don't know if this includes vectorisation

Realtime and db are among the things that we will work on next. We still need to figure at a way to do this efficiently for us so that we don't need to integrate ourselves with every kind of db. The maximum was about 400 million datapoints in one visualisation, and that worked fine! We're also definitely considering a Metabase/Grafana approach where you can deploy it yourself.

There is no integration for any db setup at the moment, but this is one of the next things we will tackle for sure! Druid was not on our radar, seems like it is used a lot in automotive/aerospace. Never tested superset, I'll give it a try. But it looks to be more like a dashboarding tool. Our goal is to provide an investigative tool that allows the user to detect problems and in a very interactive way change the visualisation.

Hi HN, co-founder here! We designed Marple for engineers to make visual analysis intuitive and powerful. Most of our users are control systems engineers or test engineers (but not the kind that tests software). They often have to draw conclusions from millions of time series data points. As a quick example: logging data from a drone control system for 15 minutes at 1.5 kHz is already more than 1 million samples.

This creates some interesting technical challenges. We implemented a custom visualisation engine that efficiently displays only the data that fits the screen. In the same way, we also support applying functions such as low-pass filters, integrals, derivatives, etc.

Having a visual approach to analysing data is in our opinion still highly underrated. Common engineering tools such as Matlab or python+matplotlib scripts require you to reduce the number of samples yourself. And you end up with a plot that is not interactive: zooming, panning highlighting, adding a different signal are all cumbersome or impossible. Marple treats interactivity as a first class citizen.

We recently launched a beta version and we're currently looking for test users to try it out. If you sometimes need to visualise some data from a .csv file, python script, matlab script, simulation, ... you should give it a spin!

Cheers! Nero

ku·ber·netes - a swelling of the cornea beneath the eye, caused by fluid aspiration, typically occurring as a form of secondary hairlike cloud formation during movement

Belgian writer David van Reybrouck has written a very interesting book on this topic: Against elections. https://www.amazon.com/Against-Elections-David-Van-Reybrouck...