aarondia

Heyo, one of the Mito creators here. Thanks for sharing Datasette. I haven't seen that one before. It looks neat!

You're right though, there are several tools that fit the general shape of: GUI on top of Jupyter on top of Python. There's a few general vectors to understand these tools by:

1. Excel-ness: Although most (if not all) of these tools incorporate some type of spreadsheet, the interface for interacting with the data in that spreadsheet differs greatly. Some tools, like Bamboolib [1] and Datasette [2] resemble Excel only in the spreadsheet. Other tools, like Mito [3], stick to a lot of the other Excel design decisions -- things like having a toolbar with buttons and menu items to access functionality, the ability to write spreadsheet formulas inside of the cell & formula bar, etc. In many ways, this Excel-ness design vector is a proxy for how easy it is to get started with the tool. What we see, is that users are able to download Mito and get something useful out their first analysis because the interface is one that they are used to!

2. Ownership of your analysis / lack of lockin: We believe that the most powerful low-code spreadsheet tools allow spreadsheet users to easily transition to full programming languages, if they want to. Instead oflocking users into a limited and proprietary product, it's better if users can transition to a full programming language (like Python) very naturally. This transition is super natural in Mito because we generate Python code for every edit that a user makes. So if Mito doesn't support the exact transformation that you want, you can use Mito as a starting point for your analysis and customize the script that Mito generates.

[1] https://bamboolib.8080labs.com/ [2] https://datasette.io/ [3] https://www.trymito.io/

Thanks for posting!

One of the creators of Mito, here. Thanks for your feedback. I wanted to share a couple of nuggets about Mito that have been helpful in talking about this with other users.

1. The core Mito product is open source. You can see our GitHub here [1]. We also have a pro version that has some additional, code visible, but non-open source features. The way that we think about which features belong in which version of the product is as following: Features that are needed to just get any average analysis done are open source features. On the other hand, features that are specifically useful in an organization -- connecting to company databases, formatting / styling data and graphs for a presentation, etc. -- are pro features. So if you are a team that is relying on our pro features, you're helping support the longevity & progress of Mito. If you are not one of those users and using the open source version, then you will always have access to Mito (and can even help improve it!). Of course the line between what features are specifically helpful in an organization and what feature are needed for an average analysis is a bit blurry, and is a moving target as we continue to expand Mito's offering.

2. Mito is designed specifically to not force users to make a big 'switch'. I've commented this elsewhere in this thread, but just to recap: Because Mito is an extension to Juptyer and because we generate python code for every edit you make, Mito is designed to improve your existing workflow instead of lock you into a new system. Many Mito users use Mito as a starting point! They do as much of their analysis as they can in the Mito spreadsheet and then continue writing more customized Python code to finish up their work.

Not requiring a big switch is nice for the user and its nice for Mito too! Lots of large companies have been able to get up and running with Mito in 30 minutes because it fits into their data stack.

Anyways, not that these are the only two reasons you might feel uneasy about adopting Mito, but at least wanted to share why the switch to Mito might be less scary than switching to other tools.

[1] https://github.com/mito-ds/monorepo

This is a great point and something that we're actively working on improving in Mito. If you have millions of rows of data, its not enough to just scroll through your data, you need tools to build your understanding.

Some of the tools that you mentioned exist in Mito today. For example, Mito generates summary information about each column (all of the .describe() info along with a histogram of the data). And we're creating features for gaining a global understanding of the data too.

In practice, one of the main ways that we see people use Mito is for that initial exploration of the data. Often the first thing that users do when they import data into Mito is to correct the column dtype, delete columns that are irrelevant to their analysis, and filter out/replace missing values.

Yeah, Mito is limited to the Jupyter ecosystem (for now). We want to expand to VSCode, Google Collab, and Streamlit!

For the time being, because Mito generates pandas code for every edit you make, you can always use Mito in Jupyter to generate code, and then copy it over to VSCode. Admittedly, its not as nice of a workflow, but it does work!

Heyo, Mito cofounder here, bridging that gap is one of the main ways that enterprises are using Mito today! Helping business users become data self-sufficient in a world where Excel's data size limitations make it a non-option is where Mito shines :)

Thanks for that feedback. Mito's approach to telemetry is that we never log any of your data or metadata about your data. We don't track things like the size, shape, or content of your data.

We do collect info about app usage, things like which buttons users click. This allows us to focus development time on improving the features that are used most often.

That being said, it's important to us that there is a way to be totally telemetry-less if users don't want any information to be leave their computer. Compared to most other cloud-based sass data science tools where you pretty much have no hope of total privacy, we're proud of the flexibility that we offer.

But of course, we're always open to feedback about how we can continue to improve our practices!

Heyo! Another co-founder here. Excited to see Mito on HN :) Thanks @alefnula for posting!

+1 to everything @narush said.

It's important to us that the software we build is empowering to users and not restrictive. This plays out in two primary ways: 1) Since Mito is open source and generates Python code for every edit, Mito doesn't lock users into a 'Mito ecosystem', instead it help users interact with the powerful & robust Python ecosystem. 2) Because Mito is an extension to Jupyter Notebooks + JupyterLab, Mito improves your existing workflows instead of completely altering your data analytics stack.

Excited to interact with you all in the comments :)

Hey, I'm one of the founders of Mito (https://www.trymito.io/). This is a super interesting perspective. I agree with a lot of your thoughts and wanted to respond to a few in particular.

> They also happen to be utterly awful at anything even remotely large-scale.

I think there's a few reasons why spreadsheets struggle to scale to large datasets and complex analyses.

When it comes to data size, legacy spreadsheets like Excel were just built for an age with different data size expectations and its hard to upgrade that monstrous code base. That's why Mito uses Python to make all of the transformations. Python still has limitations, but it works for tens of millions of rows of data.

Complex analyses are the other big cause of pain when using spreadsheets. Specifically, spreadsheets can quickly get super messy when using a mix of tabular data and singular cell results. Once the structure of the spreadsheet loses consistency, it takes a lot more mental effort to untangle the spreadsheet.

These complexities arise because Excel is super un-opinionated about what types of analyses make sense for a spreadsheet and how those analyses should be structured. Because Mito is designed specifically for working with tabular data through pandas dataframes, we're able to make design decisions that enforce a bit more structure into the analysis. 1) All data in Mito must be tabular -- it both preserves the structure of the spreadsheet and fits the ideals of pandas dataframes. 2) Every edit you apply in Mito applies the entire column (or dataframe for ops like filter, sort, pivot, etc.).

The result of 1 + 2 + the fact that Mito generates the equivalent pandas code for every edit makes it fairly easy to understand what transformations are applied to the data at any given time.

In practice, we see complexity explosion is the result of combining data exploration and analysis. In the exploration phase users apply temporary filters, column transformations, etc. But they don't want to take those transformations with them. What is exploratory and analysis work is often not known until after the analysis, so its a hard problem to design for, but its something we spend a lot of time talking about. Our most recent work to address this area of complexity is optimizing the pandas code that we generate. We can use obvious cues like if the user deleted a column or dataframe that they had previously created to tell us that work was only part of exploratory work that they no longer want. As a result, we can safely delete the python code used to create those columns/dataframes.

> I want a tool I can open and use right now, not one where I have to make a whole new Python environment and notebook and so on just to do a simple calculation

I totally agree with this! Even as the creator of Mito, if I have to do some quick ad-hoc analysis, I'll end up opening Excel instead of launching Jupyter and then Mito. We're looking into ways of improving this though! One idea is to create a command like mito <file path> that automatically launches your juptyer server and opens the file in Mito. Another is to add support for Jupyter Lab desktop so you can get closer to launching with the click of a button.

Lastly, I'd love to engage with you more about this since you clearly have a lot of interesting thoughts. If you want, reach out to me aaron <@> sagacollab (dot) com.

In the spirit of dreams and poetry:

__________________

Spreadsheets are my desired cuisine. Excel the dish I adore.

But fill it up with too many ingredients, and you’ll find Excel on the floor.

It happens all the more frequent now, as my grocery store is now a warehouse

So I pound my fists and curse and yell, and leave Excel in the guesthouse.

Lost without Excel, in a ‘landscape where nothing officially exists’

I saw a single cell appear above the dune in a hazy mist

Running up the dune, the grid displayed with all its might

A new spreadsheet designed for my growing appetite.

__________________

I’m one of the founders of Mito [1] (and a very bad poet). Like Spreadsheets are Dreams discusses, we believe that spreadsheets are the most powerful low-code tool because of their versatility to capture the simplicity or complexity of nearly any analysis.

As my poem tries to highlight, existing spreadsheet tools like Excel are not designed for today’s growing data sets. Today's spreadsheets should help users leverage their Excel skills while upgrading to a more robust and powerful environment like Python.

Mito is a spreadsheet extension to your Jupyter environment. You can display any Pandas dataframe as a spreadsheet, and edit it in a very similar way to Excel. For each edit you make, it generates the corresponding Python code below for those edits. Practically, you can think about Mito as recording a macro, but instead of generating VBA code, it generates Python.

[1] https://www.trymito.io/

A lot of comments here point to users eventually running up against the boundaries of no/low-code environments and having to make the decision to graduate from them or push their limits to a place where they were really not intended to be used.

I've spent the past 1.5 years or so building a no-code data analytics tool called Mito [1] -- a spreadsheet extension to JupyterLab that lets previously Excel-based workers make the switch to a Python based environment. Unlike other no/low-code data analytics tools we spend a large amount of engineering time making sure that users are never locked into Mito. We do this in 2 ways: Firstly, every time you make an edit to your data in the Mito spreadsheet, we automatically generate the equivalent Python code for you. And secondly, to make it easy to work with that code, Mito is an extension to your JupyterLab environment. By existing in the IDE that you would otherwise be using if you didn't have the low-code environment, its really easy for users to switch back and forth between working in the Mito no-code environment and writing code themselves.

Practically, what we've seen over the past year is that giving users ownership of the code that they create using Mito and existing in JupyterLab has been incredibly important. The most active Mito users are those that use Mito in conjunction with custom Python code that they or their company has written. They do things like incorporate API calls to access their data, use advanced pandas/Python functionality that we don't support, etc.

TLDR: We believe that no/low-code are most powerful when they are augments to the "real deal" code-first workflow instead of a substitution.

[1] https://trymito.io/hn

I think it’s a cliche at this point, but what has really been helpful for me is setting boundaries. I used to do some work 7 days a week, trying to do less on the weekends. But because I didn’t have strict guidelines, I would have anxiety that I should be doing work humming in the background all weekend. When I switched to the model of doing absolutely no work on the Saturday (even if I didn’t have anything better to do), it allowed me to completely separate work from recovery, and made both more enjoyable.

I can attest first hand to how great Plotly is. I'm building Mito[1], an interactive spreadsheet that converts every edit you make into the equivalent Python code, it's also a Jupyter extension. We use Plotly for all of our visualizations -- letting users configure bar plots, histograms, scatter plots, etc similar to how you could in Excel. Plotly's API is really simple to use and it allows for interactivity right out of the box. It would have taken us months to recreate their functionality that we were able to get in days of implementation instead.

[1] https://trymito.io/hn

Sorry for just getting back to this, but yes it is! Almost nobody picks up on it, so color me impressed.

I've been reading about D3 and am thinking about integrating it into my React project. But I've come across a number of posts that discuss how React & D3 don't play nicely together because they both manipulate the DOM. [1] is a good starting point for this deep dive.

Does Observable Plot resolve those issues or do we need still to be careful integrating it into a React environment?

[1] https://medium.com/capital-one-tech/a-comparison-of-data-vis...

Glad to hear you're a fan! You're right, Mito is not designed to cover all of Excel's use cases. Mito focuses on rectangular datasets that fit nicely into a pandas dataframe ... at least for now. So the types of use cases that we're looking for is data cleaning, pivoting, etc. If you're building an LBO, Mito is not the tool for you.

It seems like what your proposing is almost a wrapper around pandas functionality to make the language easier to read for Excel users. I think that's a super interesting approach which we honestly haven't thought that much about. As a rule of thumb for Mito right now, any spreadsheet formula gets generated as a Mito formula (ie: using an IF statement in the Mito spreadsheet generates the code IF(A > B, 1, 0) instead of the Pandas code) and anything else is raw pandas code (ie: pivot tables, merges, add column).

In general, we've been thinking about trying to move more of the code to the raw python approach since we've heard things like "not seeing the raw script makes the code unproductionizable" etc. But I also see your point that beginning Python users might prefer readable code over Python code. If we took that approach, users would still get the reproducibility, auditability, and ability to use a spreadsheet interface on large datasets, they'd just sacrifice any semblance of learning Python. That's great food for thought!

One example is https://www.pyxll.com/ which lets you write Excel Add-ins using Python.

I've never used it myself, but I have tried to use the Excel JS API, and it was quite a pain.

After talking with Excel users for the past year to understand what makes it such a ubiquitous tool, I don't think that its the ability to write complex formulas, but instead the reactive, visual environment (read: IDE). Of course, for some users the ability to nest 100 VLookups or create UDF's is the important part, but for the vast majority of people, their workbooks aren't __that__ complex.

When it comes to data analytics IDEs there is a fundamental tradeoff between staring at the data or staring at the transformations. Excel makes sure you are brutally aware of each edit you make to your data at the expense of reproducibility and auditability of your transformations. Python takes the opposite end of the trade -- obscuring the underlying data, but bringing the transformations to the forefront. For non-programmers trying to learn Python (especially for data analytics), the biggest hurdle is losing touch with their data.

I've been building a Python package, Mito (https://trymito.io/hnc), to try to address this tradeoff for those who want to analyze data with the intuitiveness and data-first-ness of a spreadsheet, but with the power and traceability of Python. Mito is a Jupyter Lab extension which gives users an interactive spreadsheet that automatically converts your spreadsheet analysis to the equivalent pandas code. You can write Excel formulas, merge datasets, create pivot tables, etc.

This year was a great experiment in person hygiene. It was the healthiest I have ever been for a full year, no colds, stomach bugs, anything.

I'm not sure which practices I'll continue, but its definitely shown me that I have wayyyy more control over my health than I previously thought.

Curious what others think are the most effective continued practices. Seems like social distancing isn't super reasonable, but maybe continuing to wear a mask in crowded places?

+1 to Nielsen Norman. Whenever I'm doing any design work, I always look for their recommendations first. They have the best UX guidance that I've been able to find. Most other resources I've found focus more on styling (fonts, spacing, colors, etc.)

I've been working on/with spreadsheet users (and some hardcore spreadsheet financial models) for the past 1.5 years - and I like to think about spreadsheets as a reactive, visual (and 2d!), _functional_ programming language. Except if you use circular references, in which case it's no longer functional (b/c you can use use circular references to store internal state!).

If you're interested, I've spent the past 6 months working on Mito -- https://trymito.io/launch1 -- - it's a spreadsheet that sits inside a Python data analytics workflow, and generates Python for you while you edit. Our approach to spreadsheet programming is a little bit more on the nose :)

When I receive an html email, I default to thinking there is nothing of value in it, and I don't read it. I use email as a more formal text messaging, or often group text messaging, system. Marketing newsletters, etc are just a clutter. But I do get a lot of them, so maybe they work for others.

They were the first newsletter that my friends and coworkers consistently read, although I never got hooked.

They speak in the tone that appeals to college students, and on the topics that they are all starting careers in.

I'm a big fan of this one too: https://spritesmods.com/?art=veccart&page=5

As a bit of background on Mito, it works by passing the parameters from the frontend spreadsheet to the Python kernel backend, which transpiles the spreadsheet formula into Python/Pandas code [0]. So what we're hoping to do down the line is let the user pick which language to translate to, SQL and R being the obvious next steps. But that's a ways away :)

[0] https://trymito.io/blog/transpiler

I totally agree that this makes sense for certain types of Excel work -- the data analysis, report generating workflows. You can see my comment above about how we're addressing some of these issues with Mito.

However, not sure that these three points hold for the type of financial modelling work that is done in Excel today. There isn't a great unbundling of Excel for the LBO, etc world (yet), so the inability to save or have columns with multiple data types/formulas seems quite limiting for that world.

There's also a workaround in Google Sheets where you can store your data in BigQuery and use the spreadsheet to interact with it.

You're definitely correct that its a nuanced question whether for a given (user, analysis) pair they are better off in Excel or Python/R/etc. Specifically with respect to building intuition for a dataset, however, there is a huge benefit of having an interactive data representation (if only for the ability to scroll and see all of your data).

Because you can think of Mito as a frontend interface to Pandas, using Mito doesn't prohibit you from building intuition or analyzing your data in the same way you would if you didn't have the spreadsheet frontend. It just helps you write the Python/Pandas code faster + see the most up to date version of your data set in live time.

The typical Mito user uses Mito multiple times throughout an analysis. A common pattern is: start by just visualizing the data in Mito, create a few graphs to help understand the distribution using matplotlib (right now we only have a tiny bit of graphing support), passing the data back into Mito to do some filtering and cleaning, then lastly creating a pivot table output using Mito. Of course, it varies greatly from user to user, but that's a general flow we see often!

Lack of interoperability is an interesting angle, but not one that I've ran into quite as much. I've usually gotten around that by converting to a csv file in the case of data, or screenshotting graphs, etc.

Would love to hear a bit more about your workflows where you're trying to process an .xlsx file in another system. I'd imagine it would be a nightmare, but haven't ran into it myself :)

I'm less familiar with the R ecosystem, but there are tools in the Python world to go that direction -- https://xlsxwriter.readthedocs.io/