Show HN: Bamboolib – A GUI for Pandas (Python Data Science)
bamboolib.com
bamboolib.com
How much work would it be to use Dask (and Dask-ML) as a backend?
I see the OneHotEncoder button. Have you considered integration with Yellowbrick? They've probably already implemented a few of your near-future and someday roadmap items involving hyperparameter selection and model selection and visualization? https://www.scikit-yb.org/en/latest/
This video shows more of the advanced bamboolib features: https://youtu.be/I0a58h1OCcg
The live histogram rebinning looks useful. Recently I read about a 'shadowgram' / ~KDE approach with very many possible bin widths translucently overlaid in one chart. https://stats.stackexchange.com/questions/68999/how-to-smear...
Yellowbrick also has a bin width optimization visualization in yellowbrick.target.binning.BalancedBinningReference: https://www.scikit-yb.org/en/latest/api/target/binning.html
Great work.
We are currently thinking about providing other dataframe libraries like dask or pyspark and similar. However, we are a little bit unsure on how to make sure that there is user demand before we implement it. It is not a complete rewrite but it would require some additional abstractions at some points in the library. And we need to check if some features might not be available any more. Would dask support be a reason to buy for you?
Great hint with yellowbrick and yes, we are considering some of those features as well if there is a useful place in the library.
In general, we are also thinking about ways how you can extend the library for yourself so that you can add your own analyses/charts of choice and then they will come up again the right point in time. In case that this is useful.
Pandas-profiling has a number of cool descriptive statistics features as well. https://github.com/pandas-profiling/pandas-profiling
There's a new IterativeImputer in Scikit-learn 0.22 that it'd be cool to see visualizations of. https://twitter.com/TedPetrou/status/1197150813707108352 https://scikit-learn.org/stable/modules/impute.html
A plugin model would be cool; though configuring the container every time wouldn't be fun. Some ideas about how we could create a desktop version of binderhub in order to launch REES-compatible environments on our own resources: https://github.com/westurner/nbhandler/issues/1
I guess that makes it easy to get started with pandas (and learn about the pandas api). I wonder how some advanced transforms such as join/union/pivot will look like?
For join in action, you can watch this video: https://www.youtube.com/watch?v=r59Q19oCMr8&t=3s
We also support pivot and melt. About union: what do you have in mind here?
pyfilemods generates a ReStructuredText document with introspected API comparisons. "Identify and compare Python file functions/methods and attributes from os, os.path, shutil, pathlib, and path.py" https://github.com/westurner/pyfilemods
If this doesn't work out commercially, would you consider open-sourcing it?
I am curious how many data scientists don't already have pipelines that do similar functions?
About open-sourcing: I cannot tell right now what the situation will be in the future. But I can tell you that we believe in Open-Source and technologies which dont provide a vendor lockin. This is also why we export the pandas code. So, you are always flexible with your code and you own the result of your work.
Basically, we want to strengthen people to use Open-Source software at the core but we also want to make it as user-friendly as fully proprietary solutions like Trifacta. So, you will have the best of both worlds without the vendor lockin.
We talked to many Data Scientists and some already started creating similar packages etc but they never got far because it takes a long and consistent effort to catch most of the cases. Also, it quickly becomes a software engineering challenge.
You can sign up for https://cloud.trifacta.com/ our SaaS offering for your data analysis.
In this times is normal for companies to create their own products using open source products, but to some people is not very good seen.
In my case I think is not worth it a tool that only works in some specific environment, that doesn't have many functionalities, and it costs more than all the products of Jetbrains. I don't like either that is a tool built on top of open sources projects trying to charge a big amount while it does not have almost any functionality.
I really don't think that is correct. It integrates into those / builds on them, but those projects absolutely do not have the features that I can see playing around with this product.
I don't want to demean this amazing work but I think the user persona and the price for the tier seeming to be a mismatch.
On top, bamboolib aims to reduce the training time for new Data Scientists.
In addition, bamboolib makes pandas available to people who are proficient with working on data but not specifically Python or coding. Thus, companies can let people with business knowledge work on the data transformations who then hand over the code to Data Engineers who deploy the code, or similar
What do you think about this?
After the demo I looked at the pricing and immediately decided it's not worth it by far.
From the viewpoint of a freelance software dev that does quite a lot if data cleaning lately, the price is so high that I wouldn't even bother trying it on binder.
As a comparison, I pay €53/year for PyCharm professional that I can install on as many machines as I like and pay for my Excel/Office a similar yearly amount. I switch between 3 computers, so having a license nailed to one of them is a dealbreaker.
Also, $49 + taxes roughly translates to 1 hour of income per month - every month if I use it or not. Plus I'd have to factor in the time it takes to setup and deal with license problems & bugs. Setting up licenses behind a company firewall is quite a challenge - unless you use a simple txt.file license option like jetbrains. BTW, Jetbrains also has a very cool feature in the license model: If you pay for at least a year, you get to keep the last version that's at least one year old for free. From my usage, I estimate that bamboolib could save me 1 hour per month max - currently I just paste to excel if I need to scroll in a larger data set or use the .sample() function to look at some examples.
So to tempt me there should be a freelancer license at a maximum of $49/year that covers at least 3 machines (only use one at a time) and should work offline.
BTW, the companies I work for all have not made the jump to Jupyter labs, yet. They are firmly Excel based and I'm constantly trying to drum up interest for Jupyter. I also do regular meetup talks on Jupyter (where normal business people show up) and many of them don't know that it exists, yet.
So having a very cheap or even free personal license would showcase your program to companies... and you could write the license in a way that companies need to buy a full price version.
Also, thank you for the licensing input - so that we can consider and support other options in the future.
Why do you think that you will only get 1h per month out of this? How many hours per month do you spend with pandas? Given this estimate, I can totally understand your price proposition of 5$/month because we also aim to provide at least 10x value. However, we assume 10h savings per month. Did you already see the data visualization features? https://www.youtube.com/watch?v=I0a58h1OCcg
Did I understand you correctly, that you propose offering a free version (because it might not make sense to charge less than 5$ per month anyway?) for business and another one for 49$/year for freelancers/businesses? Or do you also propose adding another company license?
Yet not for me as I do not like the 'click to create macro code'. They never have all the things I want and like to have my code in my syntax. But that's me.
What I want is to be able to add new functionality to the bamboo UI via new buttons for specific functionality that you don't have. Maybe if you had a plugin architecture.
I have code for this already and I'd just want to add it to the UI.
I think you should be much more up front that this is a commercial product, because very few things based on Jupyter and Pandas are.
About the communication: I can understand that. We try to communicate clearly that we both are commercial and support Open Data. It's not easy, however. Especially when people see you for the first time.
According to your perception, where could we be more clear on the communication?
I think you should mention it near the top of the demo notebook linked here, if it's intended to be a main entry point. I looked at the notebook and videos before heading to bamboolib.com, which has the first mention of pricing, at which point I felt like I'd wasted my time (because I'd gotten the wrong idea). I think most people are used to anything demoed in a Jupyter notebook being open source.
Anyway, I like the idea of making UI for Pandas, but I think that there should be more comprehensive software to make data science easier for non-coders.
Yet, I definitely agree that bamboolib especially nicely suits pandas learners and non-coders. Would be happy to have a direct exchange on your ideas on this topic (feel free to pm me at tobiaskrabel at gmail dot com).
To me the only use of this tool is to make available the more complex uses of pandas to individuals without the background/understanding of how to wield those.
But without that understanding, giving those people a UI of functions they have no understanding of is just a recipe for disaster.
All these tools that aim to lower the barrier to entry for data science without fully automating it are doomed to fail because they have no audience. The market for analysts who aren't also software engineers is shrinking to 0.
I can see use of the UI in a classroom setting to bridge the learning gap for people that are pretty proficient with the utility libraries like pandas offer, but lack the experience with Python and reading documentation at this point in time. I honestly fail to understand this critique, that sounds like saying we should ban Excel because people could use it to calculate something that doesn't make sense. It's not like pandas does something magical and every half decent Excel user understands the functionality behind the buttons I see in the bamboo demo linked here.
> The market for analysts who aren't also software engineers is shrinking to 0.
While I certainly get where this perspective might be coming from, I find it unlikely to be true. The recent acquisition of Tableau and growth of similar no-code tools shows that it's untrue from a business perspective (just read one of the HN threads on these topics, plenty of non-SE people making good use of them). Even from the code perspective, outside of production most of the data analysis code I see hardly shows any signs of good software engineering practice and yet fulfills the task it is written for.
They like the opportunity to not always have to write the code and just reach it via the UI. So they have a smooth workflow without inspecting the data in Libre Office/Excel or having to google so often.
However, what they especially like are the data exploration and visualization features because they save them a lot of time. You can see a video of those here: https://www.youtube.com/watch?v=I0a58h1OCcg
Why do you not want the UI panes to overlap? And what exactly do you mean? Because the pane does not overlap but you can inspect and reach the full table. Maybe it was confusing on the video?
About visualization: We also provide quite some automated charts as can be seen here: https://www.youtube.com/watch?v=I0a58h1OCcg
We are thinking about supporting other dataframe-like libraries and hopefully we can support all of them. However, it is a matter of priority here. The architecture enables this in general but we need to find users who actually want and need this. Any idea in that regard is appreciated and if enough interest builds up, we can definitely support this.
What do you mean by (3) with the shaded cells? Maybe you can give an example here?