Representing Python notebooks as dataflow graphs
marimo.io
marimo.io
I run it with uv and --sandboxed which makes it much easy to share notebooks with teammates and not have to worry about limiting dependencies. Any issues I've had were were Python libraries themselves (specifically graphviz).
I really like how much easier it is to reason about interactive components vs Jupyter. The mo.ui.altair_chart method has got me to migrate off of matplotlib because charts can be fully integrated – as you can see in the demo being able to lasso data points or scrub a chart and analyze specific time periods is awesome.
One thing which I don't like about reactive notebooks is that you have to be much more mindful of expensive and long running calculations. There are feature to help, like adding a run button, but often I end up just disabling auto-run which does reduce the value of the reactive flow. For those use cases I don't find myself using marimo over Jupyter.
I think the entire marimo team deserves a shoutout, the quality of the software is excellent, they've moved very quickly, and they have been very receptive to issues and feature suggestions.
We're committed to having an excellent experience for working with expensive notebooks [1]. At least for my own personal work, I find that there are many reasons to use marimo even when autorun is disabled — you still get guarantees on state, rich dataframe views, reusable functions [2], the Python file format, and more. If you have feedback on how we might improve the experience, we'd love to hear it.
This seems not necessarily very hard to me? All you have to do is keep yourself honest by actually trying to reproduce the results of the notebook when you're done:
1. Copy the notebook
2. Run from first cell in the copy
3. Check that the results are the same
4. If not the same, debug and repeat
What makes it hard is when the feedback loop is slow because the data is big. But not all data is big!
Another thing that might make it hard is if your execution is so chaotic that debugging is impossible because what you did and what you think you did bear no resemblance. But personally I wouldn't define rising above that state as incredible discipline. For people who suffer from that issue, I think the best help would be a command history similar to that provided by RStudio.
All that said, Marimo seems great and I agree notebooks are dangerous if their results are trusted equally as fully explicit processing pipelines.
Needs a bit of extra config but tis a really nice feature.
It's one thing when I'm relying on my own attention to detail to make sure all the intermediate results have been correctly recalculated, but it's entirely another when I have to rely on even trusted co-workers' attention to detail, much less randos on github. As a sibling comment points out, the "reproducibility crisis" numbers are very much not in favor of this approach being the right idea.
... Or you could work in a format that makes incorrect / out-of-date intermediate state impossible (or at least hard) to represent, which is (I believe) what marimo is an attempt at.
Yet from my experience, you quickly learn to "restart kernel and run all" before sharing to make sure everything is good.
All but the most novice users get caught by the "out of order cells" trap, and those will
1) not use anything that adds complexity, because by definition they are novices
2) fall in any other trap on their way because anyway that's how you learn
Thus, IMHO, these flow tools are only seen as useful by _real devs with savior syndrome_, pushing dev solution to exploratory research users, and that will never catch on.
It's harder when some of your cells took hours to execute, which mine don't. Then I think you should be using data files.
The tools that fix this problem seem to evoke "the cure is worse than the disease." At this point, basic Jupyter notebooks are by far the most reproducible way I've ever worked.
Anything else we can help with?
I forgot the link: https://docs.marimo.io/guides/editor_features/overview/#conf...
Habitually running restart and run all works okay for very lightweight notebooks, but it's a habit you need to develop, and I believe our tools should work by default. It doesn't work at all for entire categories of work, where computation is heavy and the cost of a bug is high.
From the blog, you will see that reactive execution not only minimizes hidden state, it also enables rapid data exploration (far more rapid than a traditional notebook), reuse as data apps, reuse as scripts, a far more intelligent module autoreloader, and much more.
marimo is not just another Jupyter extension, it's a new kind of notebook. While it may not be for you, marimo has been open source for over a year and has strong traction at many companies and universities, including by many who you may not view to be "real devs". The question of whether marimo will catch on has already been resolved :)
We describe why we opted against “disabling” the graph at the end of this blog: https://marimo.io/blog/lessons-learned
I would love to see DAGs like in SSA form of compilers, that also supports loop operators. However, IMHO also the notebook interface needs to adjust for that (cell indentation ?). However, the strength of notebooks rather shows in document authoring like quarto, which IMHO mostly contradicts more complex controll flow.
This is one of those things that is blindingly obvious to people in adjacent sectors of the industry, but maybe there just hasn't been enough cross-pollination of ideas in that direction (or in either direction).
I personally fall into both camps--both horrified at the execution model but also, it's not actually caused problems for me as it's second nature to restart the kernel when changing upstream cells.
That said, marimo feels like a significant step forward and I'm rooting for them to succeed.