narush

Hiya kite_and_code - thanks for the question + good to see you here :)

Our understanding of our license is evolving - we're first time open source devs, and as I'm sure you know it can be a tricky process. That being said: we totally support Mito users using Mito from notebooks hosted in the cloud!

Currently, we have quite a few users using Mito in notebooks hosted through AWS, GCP, etc. We’re aiming to be good stewards of open source software, and want to see Mito exist where ever it is solving users problems!

We’ve had lots of folks in lots of environments request Mito, and are actively working on prioritizing supporting those other environments. We added classic Notebook support last month (funnily, I thought it’d take weeks to support, and it took 2 days lol) - and are looking into VS Code, Streamlit, Dash, and more!

EDIT: due to comment below, I edited this comment for clarity that we 100% support users using Mito from notebooks in the cloud!

Def check these all out! Lots of cool tools out there. For anyone who's tried a bunch of these... that's a great topic for a Medium post :)

Fancy seeing you here, writing the same comment as me... :}

To pull the curtains back a bit: we probably spend about 85% of our product and development time on open source code. Just this week, we developed copy and paste, nan value filling, and spilling a text column on a delimiter - all of these are open source features.

As we've begun to engage with larger teams, we often take features that we build out for their workflow and open source them as well - a few of the teams have been explicit proponents for the open source tool, which is awesome to see.

I'm sure our thinking on this will evolve over time, but we are highly focused on developing just a _great_ piece of open source software. And for folks that need more power, we want to give them the chance to get it - while also supporting Mito's development :)

P.S. Check out our Mito Pro roadmap here: https://www.trymito.io/plans#mito_pro_roadmap. Feedback appreciated!

Hey everyone. Mito cofounder here. Thanks to whoever posted this - was a real surprise to find it here :-)

Mito (pronounced my-toe) was born out of our personal experience with spreadsheets, and a previous (failed) spreadsheet version control product.

Spreadsheets were the original killer app for computers, and are the most popular programming language used worldwide today. That being said, spreadsheets have some growing to do! They don’t handle large datasets well, they don’t lead to repeatable or auditable processes, and generally they disrespect many of the hard won software engineering principals that us engineers fight for.

More than that, as spreadsheet users run into these problems and turn to Python to solve them, they struggle to use pandas to accomplish what would have been two clicks in a spreadsheet. Pandas is great, but the syntax is not always so obvious (not is learning to program in the first place!)

Mito is the our first step in addressing these problems. Take any dataframe, edit it like a spreadsheet, and generate code that corresponds to those edits. You can then take this Python code and use it in other scripts, send it to your colleagues, or just rerun it.

We’ve been working on Mito for over a year now. Growth has really picked up in the past few months - and we’ve begun working with larger companies to help accelerate their transition to Python.

To any companies who are somewhere in that Python transition process - please do reach out - we would love to see if we can be helpful for all your spreadsheet users!

Feel free to browse my profile for other spreadsheet related thoughts, I’m a bit of a HN junkie. Of course, any and all feedback (positive or negative) is appreciated.

My cofounders and I will be trolling about in the comments. Say hey! :-)

Hiya! I'm a cofounder of Mito [1] - where a spreadsheet extension to Jupyter Notebooks / Lab. Our most used feature is our exploratory graphing and Excel-like pivot tables - which seems super relevant.

Totally no pressure (although this is clearly a pitch), but if you get the chance to check it out and have feedback on where we fall short of being as awesome as Excel, your feedback would be super appreciated!

[1] https://trymito.io

My (novice) highest-level summary of the argument in this paper: firms emerge as a result of transaction costs between people. By being in a firm together, people build shared structures, and as a result can reduce these transaction costs. A firm is like a ball of low transaction costs, pretty much.

This paper is the first economics paper I ever read (a long time ago, excuse my if my summary is awful lol), and still one of the most thought-provoking and interesting papers I've encountered.

The fun-to-think-about questions that it leads me to:

1. What sort of transaction costs between people today are _practically_ the most important to leading to a creation of a firm?

2. What if we built technology that reduced those transaction costs to near zero? E.g. what would it mean for there to be less incentives for a firm to form?

3. How does questions of transaction costs relate to market structure and monopoly?

I guess mostly this paper is amazing b/c it made me realize I never really thought to ask the question "why companies in the first place?"

I'm reading his book Build right now - it came out last week, so assuming that's why he's appearing all over all the feed). About 2/3rds of the way through currently.

My thoughts: 1. Lots of Steve Jobs talk. There's a whole chapter on the distinction between real assholes, and assholes that just really care about the product quality / customer. The distinction drawn was in motivation - but I wonder if it might just be a winner-writes-the-history situation. 2. Some advice goes heavily against current startup orthodoxy. He rejects fail-fast / figure it out later mentality, and argues for a lengthy product design process (for both atoms and bits!). 3. Lots of good details about how to think about managing people, managing and scaling team, and issues with scaling, etc, taken from his days building Nest.

Both (2) and (3) stand out to me as functions of his specific background. Building a company like Nest is obviously wildly (!) hard, but doing it after building the iPhone is a different game. I don't think his advice about lengthy design processes make a ton of sense for my startup [1], for example.

Generally, I always try and remind myself when I'm reading company-building advice books: this is probably good advice for someone in the same context as Tony Fadell (e.g. someone who is launching a second company after building... the iPhone). For me, it may or may not be relevant.

The good thing about failing fast is that you can use it at a strategy-level, aka fail fast _at_ failing fast, and switch to longer product cycles if you think that might work better. Starting with a long-product cycle doesn't give you much of a chance to try again if your first swing is a miss (which Mito's first attempt was...).

[1] https://trymito.io

> fundamental invalidity, rather than practicality

I'm not sure there's any difference between these in practice - same with the difference being degree or kind. At the end of the day, the question is: what can I actually implement in my own brain to make good decisions.

I think the biggest place we might disagree here is about _what the goals_ are of prediction in the first place. "if you can't track your predictions over time you'll have a hard time improving" might be true, but for me, the terminal goal is _not_ to make increasingly accurate predictions.

My goal is to operate as effectively as possible. We both agree that explicit Bayesian calculations aren't useful in a startup world. We also both agree that Bayesian techniques are very easy to footgun with in the wild. For me, both of these point pretty squarely in the direction called: we need non-Bayesian decision making tools.

To paint the shape of what I think these decision making tools should look like:

1. They avoid mechanistic explanations.

2. They are explicitly not mathy.

3. They apply simple and robust heuristics to generate lots of options.

4. They exist in a purposely constructed iterative environment so that you get lots of attempts.

These decision making tools are built around a) acknowledging that we cannot mathematically reason about the uncertainty we're dealing with in any legit capacity (without just foot-gunning), and that b) spending time on _decisions_ vs. on _execution_ is silly in most contexts, as execution is where you actually learn things (and thus you should make lots of quick and dirty decisions), and c) more good options are always valuable!

I can talk more about what this looks like in practice, as it's sort of the shape of how my cofounders and I run our startup. It's a WIP of course, but in practice we find the iterative, quick and dirty heuristic approach to lead to much faster and more robust growth than long-term predictions (which we used to do a lot of).

Also, do you have links to work that has informed your thinking on superforcasters? Or links to the specific set of superforecasters you're talking about? It seems you're thinking about a specific set of people, and I'd love to learn more about em!

Thanks for the points and taking the time to write up so much! I am enjoying this interaction, even if we don't fully agree :}

> The difference in irreducible uncertainty between "shuffled cards" and "anything else" is a matter of degree, not kind.

Difference in degree or difference in kind is not the important point to sort out. The important question is what computation methods actually work in practice for humans. Some algorithms that work well on degree-small datasets fail to terminate in the course of the universe for large-datasets.

If you've ever operated in a particular uncertain environment (like, as a first time startup founder), it immediately becomes clear how totally useless explicit probability calculations are when making decisions. I was a self-proclaimed rationalist and tried reasonable hard to be a good Baysian; I ended up deciding I might as well sacrifice a goat and read it's entrails.

> The point isn't to follow the numbers off a cliff, but if you're well-calibrated (as in, have a track record with a decent brier score, or something), then pretending those numbers are totally useless is a bit silly.

I'm actually trying to make a stronger point that these numbers are "totally useless." I think in practice many applications of explicit probability calculation are actually quite harmful.

Rationalists love to talk about map and territory as the two items to be concerned about, but there's also another thing called "agent's belief in the effectiveness of their map." What models/explicit math/probability gets you in improved mapping, it takes away from you by making one's belief in the effectiveness of their map much much more.

If you look at large-scale model failure in practice, it's almost always because someone thought the math they were doing captured the full system in a complete way (and then it didn't). 2008 is a fantastic example of this.

> Well, you'd be able to do that in a world where we cared about keeping track of that sort of thing. Too bad people keep finding reasons not to do that, huh?

I'd love to start tracking predictions generally! I agree with you here!

> Surely you aren't telling me that you've updated your priors on observed evidence, and as a result have a different expectation about future world states (that is, "how useful would explicit Bayesian reasoning be if I tried using it")?

To be clear: it's the explicit probability calculation and mathification that I take my major issue with. I am most definitely not against learning from what I observe :-)

> Their level of performance is not compatible with the "the small number of people out millions for whom the the coin flip came up heads 10x in a row" hypothesis

Can you link this math? I'd love to see it - not flippant, genuinely looking to check it out and be educated here!

> Predicting the next draw from a deck of cards is subject to "out of context" model violations the same way that any other prediction over future world states is.

The observation that the uncertainty is only present in the observer is certainly true in the card context, but really questionable in the world states one. For one, don't we need to assume a really aggressive "deterministic evolution of future world states" to argue this uncertainty is only present in the observer? I, for one, am totally uncomfortable with this assumption...

Also, there's a clear computational difference between these two settings. You kind of point this out by acknowledging that explicit Bayesian calculation are unreasonable in many settings - but in practice, I'm on a rationalist email thread where folks are trying to calculate explicit probabilities about the increased likelihood of nuclear war over the past 6 months. It's totally silly.

We need to look at the actual way that Bayesian tools are actually used (and usable in practice) by it's adherents. As far as I've observed, it's mostly just silly signaling games where people make up numbers to justify whatever story they want to tell (given that the rationalist communities spend so much time worrying about confirmation bias as a fundamental one, I'm not sure this is even surprising).

Also, superforcasters are most certainly not "proof that explicit Bayesian calculations are still extremely useful." There are literally _millions_ of experts who make prediction - there's no version of history where this isn't a random subset who performs dramatically better than average just due to statistical chance! Misunderstanding this as a proof of the usefulness of the probabilistic reasoning tool is the classic example of being fooled by randomness.

Also a dev on Mito. Would love to hear if anyone has strong opinions on best practices for UI documentation! Thanks a ton :-)

I'm a cofounder of a small (and growing :-)) startup called Mito [1]. We don't do A/B testing currently, for two main reasons:

1. We're a locally installable product with opt-in updates. 2. We don't get enough new users per week to make the turnaround time on experiments quick enough.

We're big proponents for local-first software, so though we might have a hosted offering at some point for users who prefer that, the locally-installable + opt-in updates are gonna be around for a while. These make A/B testing hard for obvious reasons - it's no longer just flipping a switch to get both sets of users on the same branch after the experiment terminates, nor is it as easy to randomize people into two different groups.

We also just don't get enough new users a week to make the experiments make sense. For the effect sizes we're hoping to measure, we'd have to wait a few weeks to to draw conclusions - and a given that it's even more complicated to run more than one experiment at once (given the above local install + opt-in upgrading mentioned), it becomes really expensive to do any sort of A/B testing.

There's also the question that this post leaves out: what changes are worth A/B testing in the first place?

I know folks at a gaming company that A/B tests every single change they make to their games for the effect it has on rev/user. They are a mature company, and the changes they make to their apps are, on balance, fairly small. This makes sense to me, especially given the tooling they have built to facilitate this - although it is clear they are just trying to extract value from their existing user base rather than dramatically improve their product and grow.

For early-stage startups, IMO often the best bet is to be a user of your own product, and to just test your product changes on yourself - it's usually pretty obvious which direction you should take things. We recently overhauled our graphing capabilities to add about 5x more graph types and actual graph configuration options. Given how limited our previous graphing capabilities were, was pretty much a no brainer and obviously better. A/B testing would have just been a waste of time, methinks!

Feedback and thoughts on the above greatly appreciated. We're always looking for ways to improve our product/technical processes!

[1] https://trymito.io

Gotcha. Pretty much the template language is close enough to the destination language that you don’t need an AST for the template language at all.

That makes sense! Seems like a legit simplification to take advantage of.

Do you have any plans to make the template language more advanced/complex?

Super cool. Bidirectional code generation is something I've spent a bit of time thinking about: I've been building a spreadsheet that generates Python code when you edit it [1], but some of our users also want the ability to edit the Python code they generate and have that reflect in the sheet itself.

Template -> Code -> Template is one really hard part of this, and something this tool seems to take a really good approach to. If you're interested in related subjects (like transpiliation), I'd recommend this overview article as a great approach [2]. From this article: "Now, there is a single biggest mistake we see in persons trying to implement a transpiler without experience in this: they try to generate directly the code of the target language from the AST of the original language."

Question to the OP - you mention you parse the Ruby AST - do you also transform this into an AST of your template syntax before generating the template? Aka, do you avoid this sin, or is it not an issue for you?

With Mito, there is additional complexity beyond just going from Template -> Code -> Template, in that we also need to understand _which_ variables are being changed and in what way. This is necessary because a spreadsheet stores other data about your variables beyond just their current value. As an example, which of these columns in a dataframe are a result of a formula vs. being in the original dataset isn't something that is just stored in the dataframe itself.

I haven't tried too hard, but I don't think there's a general solution; it feels like it requires some sort of symbolic execution in the general case, and is tough to do well even in simple cases. Our "fail loudly and early" equivalent feels like it would be a lot higher than the 10-20% this tool can deliver!

Anyways, bidirectional spreadsehet code generation is low on the priorities... but it's a fun one to dream about :-)

[1] https://trymito.io [2] https://tomassetti.me/how-to-write-a-transpiler/

Yeah that’s fair feedback. Our Pro features are a WIP currently, so this might evolve in the future. It was important to us that there is a way to be totally telemetry-less if users prioritize that - vs most other cloud based sass data science tools where you pretty much have no hope of total privacy.

Mito isn’t just a spreadsheet that works with Python, but a spreadsheet that allows you to generate Python code when you edit!

If your just looking to work with spreadsheets with Python, I’d also reccomend checking out XLWings - I haven’t used it myself but some of our users do and love it!

The main benefit (to our users) of being in Jupyter is that we don’t have to force them to switch up their whole workflow. If they want to layer on a spreadsheet, bring an extension mak it easy to do this. We don’t want to lock people into our platform vs just provide the best spreadsheet experience we can!

For us, it’s nice because we don’t have to reinvent the wheel(s) that Jupyter comes with :-)

Warning: I'm a founder working in the spreadsheet space, so take the rest of my this comment with a (large) grain of salt. I've written before [1] (HN and elsewhere) about how I think spreadsheets are the most popular programming paradigm ever, we just don't talk about it much. As this article mentions, there are many ways we can push this forward.

I personally think the most powerful low-code spreadsheet tools we can build are those that allow spreadsheet users to easily transition to full programming languages, if they want to. So rather than locking users into limited and proprietary product number #115 (some of them are mentioned in this article), IMO it's better if users can transition to a full programming language (like Python) very naturally. Som I've spent the past 2 years building Mito [2].

Mito is a spreadsheet extension to your JupyterLab environment. You can display any Pandas dataframe as a spreadsheet, and edit it in a very similar way to Excel. For each edit you make, it generates the corresponding Python code below for those edits. Practically, you can think about Mito as recording a macro, but instead of generating scummy-crummy VBA code, it generates Python.

We're open core [3]. Feedback greatly appreciated!

[1] https://naterush.io/blog/Spreadsheets-are-the-Ultimate-Progr... [2] https://trymito.io [3] https://github.com/mito-ds/monorepo

This is a cool article. I like the narrative of "Change to a life where you are in control of your direction."

I am probably the most privacy conscious person of my friends (not that I do a particularly good job of it), so I spent lots of time thinking about how to communicate about privacy in a way that is effective. [We're all mid-twenties, for context.] The main issue, I find, is that people just mostly don't care.

That being said, most of my friends do relate (and dislike) the loss of agency that comes with giving your brain to an algorithmic feed that decides what you eat. The narrative of "control your life and decisions" would be an effective piece of rhetoric, I think!

The other argument I've found really effective is one that convinced me after reading Edward Snowden's memoir Permanent Record. A sketch of the point:

Person 1: privacy matters. Person 2: I don't care about privacy, because I don't have anything to hide. Person 1: Historically, the folks who were hurt because of a lack of privacy weren't us (young professionals), but rather the civil rights movement in the 1960s and the Vietnam anti-war movement, etc. Privacy is about protecting those people who the government/institutions/etc already squash down.

It's an argument that comes from Permanent Record, pretty much, and I think for me it is the most compelling reason that I care about privacy! Not as much for myself (although that's nice), but mostly for the people that privacy protects who really need it.

In my experience, it goes over very well, as people can see that privacy isn't just about doomsday preppers not wanting anyone to know where they bury their gold, but rather about protecting those people who need/deserve/would benefit from protection!

P.S. I'd recommend reading Permanent Record. I was only 15 or so when all that stuff went down, and really didn't know anything about it except "Snowden good or bad idk," but the book is a fantastically interesting and well-written story. I think he's kinda awesome.

Spreadsheets are the best way to represent data, but there is a very long-tail of tasks one might to perform with their data. Requiring all of them to be built into a visual UI pretty much insists you end up with Excel - what feels like 1 billion features, where each user knows <1% of them (and costs millions of programmer-hours to build).

I don't think users can/should leave spreadsheets for the tasks spreadsheets make sense for (basic data munching, pivoting, many formulas, etc). But being able to easily transition your spreadsheet to other tools in a easy/native way is a huge win - and why at least half of our users are actually just Python programmers who use Mito because it makes that transition back and forth to spreadsheet/code so easy!

I don't think anyone should abandon tools if they are working for them :-)

The question of "what is a better visual data model than VisiCalc" is just one of the questions we can ask ourselves about how to create Excel 2.0. In the authors original post, they point out that Notion and Coda answer this question not with an evaluation on "cells" and the data model, but rather by extending the model to include a word processor. This isn't a purely visual change, but a functionality/integration one.

There are a bunch of different angles to consider the evolution of a spreadsheet, and, as I say in my response above, I personally think focuses on changes to the UI/display of data miss the point: what's missing in Excel 1.0 isn't a better display of data - IMO, it's giving the modern, powerful analysis tools that us programmers have access to the beginner-end of the programmer spectrum!

Different spreadsheet startups certainly have different theses on this. Subset [1] (the OP) seems to focus on side-by-side grids on an infinite canvas. Monday [2] (also referenced by OP) seems to focus on different "views" for a spreadsheet for project tracking, etc. Mito focuses on allowing you to integrate Python and spreadsheets as easily as possible. Clay [3] seems to focus on spreadsheet integrations into APIs/other data.

(Disclaimer: all the above are just my understandings of these tools, but I haven't used most of them directly mostly am just going of marketing materials... I highly recommend you check them out, though - they all look quite cool!)

My post def was a plug for Mito - I'll try and make my response to the post/thesis more clearly delineated in the future. I think this post is an awesome chance to get feedback on our spreadsheet thesis (and potentially hear back from OP on this thoughts!).

Rock on, spreadsheets :-)

[1] https://subset.so

[2] https://monday.com

[3] https://www.clay.run

Thanks for posting! I love the topic - I've written before [1] about how I think spreadsheets are the most popular programming paradigm ever, we just don't talk about it much.

I personally think that the evolution of spreadsheets is less about changing the UI, and instead making it possible for spreadsheet users to easily transition to more powerful programming tools in a natural and easy way. So I've spent the past 2 years building Mito [2].

Mito is a spreadsheet extension to your JupyterLab environment. You can display any Pandas dataframe as a spreadsheet, and edit it in a very similar way to Excel. For each edit you make, it generates the corresponding Python code below for those edits. Practically, you can think about Mito as recording a macro, but instead of generating scummy-crummy VBA code, it generates Python.

We currently have two types of users. 1) Excel users from a huge variety of industries who are somewhere in their journey to learning Python - and Mito helps them write Python scripts quickly and make that transition easier. 2) Python users who prefer using Mito because of it's visual interface. I pretty much only use Mito when I'm trying to pivot or graph data - some things really just are better visually, especially when you get code out that you can edit if you want!

We're open core [3], and also sell a Pro and Enterprise versions of the tool with advanced functionality. We've been steadily growing for the past year or so, as the product has improved (first time founder here!).

Feedback greatly appreciated!

[1] https://naterush.io/blog/Spreadsheets-are-the-Ultimate-Progr...

[2] https://trymito.io

[3] https://github.com/mito-ds/monorepo

Can you give a little color on where the difficulties lie? Is it an efficiency question, or is determining "which changes" hard in the first place?

I did miss this section in my skim, but it's also the first place I went as well. It doesn't seem like you could ever be sure that no old messages would be sent with an earlier timestamp if you don't know the set of nodes that are participating in the protocol.

Moreover, if you allow the set of nodes you do know to make operations maliciously (e.g. a single bad node enters the set and tries to cause conflicts), things get much more complicated, and I don't really think you can escape the impossibility results in relation to consensus protocols...

Ok, having read through the core of the algorithm, cool! Seems like the main key here is that move operations have timestamps associated with them, and this allows you to effectively linearize the operations (and ignore ones that result in invalid state).

The tradeoff: you need to store all the operations you've ever seen, as you need to be able to detect conflicts between old ones.

Question for the authors (if they are around): any ideas for how we might get to efficient pruning of old operations?

My first thought was that you could have some notion of epoch, and once you move beyond that epoch, you accept no new operations from it. But then you realize that this requires nodes to agree on moving between epochs, which seems like it might require some fancy consensus protocol / a known set of nodes / mad complexity.

I don't have any ideas. Do you all have some idea of how we might be able to prune this old log? Or even sketches of ideas?

Super cool. Martin Kleppmann, the lead author on this paper, is the author of Designing Data-Intensive Applications, which I haven't read but have heard pretty awesome things about.

What I have done: watched his other talks about CRDTs (conflict-free replicated data types). I'd recommend watching this video [1] for an overview of what CRDTs are, why he cares about them, and open problems (one of which he solves in this paper!).

A (potentially inaccurate) TLDR: local-first collaboration software presents is a compelling alternative to the current way collaboration is implemented in a client-server model. Specifically: if we build tooling that makes it easy to build collaborative experiences into your app without a server, then we'll potentially be able to achieve the goals of 'decentralization' (or, fuck the middleman or whatever you wanna call it) in some real sense. Note that I'm not talking about crypto coins here!

Very cool, very exciting!

[1] https://www.youtube.com/watch?v=Qytg0Ibet2E

Doesn't this feel like a self-fulfilling prophecy? It's like saying "if things are going bad, then just try and extract value."

But if extracting value makes things go bad, then it pretty much leads to a death spiral.

I feel like a more helpful framework is: if you're giving up on competing, then financialize, while recognizing that a) it's probably gonna kill your company, and b) value extraction from users [especially those who have little choice in using your products!] is no fun for anyone!

Interesting post! We use a monorepo at Mito [1], it’s open source if you want to check it out [2].

We solved the test flakiness issue by getting rid of all of our frontend tests, which means we spend more time manually testing our product. We’re happy with the tradeoff because we like being forced to use our product. It’s something a velocity focused team can forget to do… at least something we can forget.

Merge races aren’t really a huge issue for us. They happen sometimes, we catch them, and we fix them. Putting thought into making them easier to fix isn’t something that makes sense at our scale.

That’s being said, we’re a tiny team of 3 first time founders - so the above choices make a lot more sense for in our context than at stripe :-)

A reminder that you need not design your systems how the big companies do! Read their informative blog posts, and then design them for your goals and constraints! There’s a whole world of solutions to problems you don’t have when your small, and it can be very easy to feel like you need to adopt all of them. Hint: you don’t.

[1] https://trymito.io

[2] https://GitHub.com/mito-ds/monorepo

Woah. This paper looks freaking awesome - and surprising! “The proposed scheme can tolerate any num- ber of Byzantine nodes (making it immune to Sybil attacks)” - this is really intriguing because of the strong impossibility results in consensus algorithms around how many faulty nodes can be tolerated. I’m looking forward to reading more than just the abstract tho (confession).

Martin - thanks for all your cool work!