Show HN: Data Painter – A Different Way to Interact with Your Data | Hacker News Reader

Show HN: Data Painter – A Different Way to Interact with Your Data

docs.kanaries.net

179 pointsloa_observer29 comments

Data Painter is RATH's new feature to interact with your data. It helps you to specify complex visual patterns and find explanations and potential causals.

It allows you to do on-flight data labeling, cleaning or even create new features does not exited in original dataset. Everything can be done with a brush tool(painter), You can even play with your data with your fingers on mobile.

RATH is an open-source alternative to Tableau, but with more automations. Feedbacks and suggestions are appreciated.

RATH's Github: https://github.com/Kanaries/Rath

RATH's HomePage: https://kanaries.net/

Open articleView on HN

Interesting product. I first encountered this "data painter" paradigm in the 80's in a product called DataDesk. That software was created by Cornell professor Paul Velleman. Paul was a student of John Tukey and credits his influence on the product.

https://datadescription.com/

Wow I would have killed for this in my algo trading days :) Great looking tool

Children, this is why you to have be very careful when you hire someone for algo trading. Some of these people are killers.

ngl would've killed for more alpha back then

Yessss this is really great! I have a bunch of time-series for climate and remote sensing data sources I could use for this

I am hearing about this tool first time and the git repo examples look impressive.

Is this free and open-source in the sense that we can host it on our own infra with no license fees? Is there any guide or example for deploying this in a AWS environment with typical data sources like S3 data lake (CSV/Parquet/Delta) and Redshift?

RATH is now under AGPL license, we are considering move to a less limited license like apache if there are more open-source users and developer in its community.

The related docs will be published soon, we are working on connecting RATH to more infra now, you can join the discord to the latest discussions. https://discord.com/invite/Z4ngFWXz2U

I think AGPL is probably your best bet if there is a chance a competitor will extend your code and release it as a SAAS.

But also, the git repo has LICENSE with the AGPL, as well as LICENSE2 which just has some names followed by all rights reserved. It's a bit confusing, if there is dual licensing or some of the files are licensed differently maybe you should explain that in the LICENSE file, even if it means referencing a separate AGPL file.

In any case it looks pretty neat, I think I'll give it a try.

I was really impressed that it seems to be doing everything locally in the browser, but I see once you start trying to do more complex visualizations it POSTs the dataset to the cloud. Still neat.

I tested 1 million rows * 12 columns. It works fine with pure client's web-workers and indexedDB. For larger datasets(like CSV file > 150MB), especially for the "mega automation" feature, It turns to be slow and sometimes lead crash in browsers.

For now, I have some temporally solutions, setting a sample size is the easiest.

Another way is put some heavy computation tasks to scaleable OLAP services, like a clickhouse clusters.

I will continue to research more ways to make the client handle as much data as possible and fast enough so more people can deploy it by their own with few costs.

Dope tool, will try it exhaustively!

One thing tho, when I try the causal analysis feature, it shows in japanese to me (github show all in english tho) is there a knob to turn it to english?

If you are on Kanaries homepage (kanaries.net), the language setting is on the top right corner near login button. The language switch button icon looks like this one: https://www.flaticon.com/free-icon/language_4509733

If you are on RATH product page (rath.kanaries.net), the language setting is on the left bottom of the page.

I did it, the insiders > casual analysis is partial english/chinese

here: https://i.imgur.com/5QCXdoG.jpeg

Just fixed this, thanks for feedback.

Nicely done!

Tangent: I am looking for a different "data painter," i.e., a simple GUI where I can draw data points and have it generate a table or CSV file with them.

I thought about "draw to create data" before, but it brings problems when your data contains high dimensions (lots of columns). Painted data only defines limited number of dimensions, for other columns, Some prediction model can be used to infer the values. but with higher dimension in data, the prediction gets less convincing.

Great point. For my use case, drawing points on a 2d plane would be good enough. I wasn't considering higher dimensions. Maybe it would be possible to "sculp" 3 or 4d data using a cad-inspired direct manipulation GUI like SculptGL or Meshmixer [0,1, respectively].

0. https://stephaneginier.com/sculptgl/ 1. https://meshmixer.com/

I found this... Something like what you have in mind? (not Foss)

https://plotdigitizer.com/

Ed: or maybe this (gpl) - didn't realize this existed... https://markummitchell.github.io/engauge-digitizer/

Ed2: another along similar lines (pun not intended). Looks like most assume you have a bitmap of a graph, rather than let you just draw datapoints.

https://automeris.io/WebPlotDigitizer/

Thanks for links! I was thinking about having the drawing part integrated into such a tool, but what you shared will work too. Cheers!

I need this!

I look forward to testing it out.

Terrific, have been looking for something like this since Hadley's work on the grand tourr

https://github.com/ggobi/tourr https://www.jstatsoft.org/article/view/v040i02 https://www.researchgate.net/publication/346903245_High-dime...

ggobi is very cool. I was always surprised that the ideas it exemplified were so slow to catch on. Glad OP is working on rectifying that.

Same, but after working with folks on these issues, turns out mostly they just don't have clean, well formatted data to even put into these tools.

Still in the stone age etc.

I think the 'k6s' title is an unintentional misdirection

I was confused why the title bar says Kanaries but the product name is RATH. As someone coming in via a deep link with no knowledge of this product or company, it was confusing to figure out what was the actual product was called. I also couldn’t easily find an explanation of what RATH stands for.

Sorry for lack of information. Kanaries is the github organization's name, with a shorthand k6s.

RATH is one of an open source project under Kanaries. As for the meaning of RATH, it actually came from an animation called Sword Art Online. I was a little bit regret to named this repo with RATH and had deleted descriptions about the repo's name, but with no more good ideas for now.....

Brand cohesion is important. I'm in your target demographic, and arrived knowing that, but I see stuff like (k6s) up top and immediately I'm bouncing because it's confusing and I have a lot of tabs opened I wanted to see.

thanks for sharing your thoughts and experience! I am fixing the descriptions.

You should probably use a spellchecker too. I saw "Embeded" on the Kanaries homepage and "comming" on https://kanaries.net/products. Also, when navigating from the Kanaries homepage to https://kanaries.net/products it's not a proper page load and doesn't reset the scroll position.

Reply on news.ycombinator.com