Hacker News Reader

Comment by _l7dh | Hacker News Reader

It's probably an unrelated post (apologies in advance) but I wanted to shoutout to the Marimo (https://marimo.io), it's the only Jupyter alternative that really got me excited, it's like Streamlit and Jupyter had a kid (and the kid took the best genes from both).

View on HN

>> marimo notebooks are pure Python and stored as .py files

That sounds like a solid improvement. I’m going to give this a test drive. I feel like modularity is one of the hardest aspects of Jupyter notebooks in a team environment.

I’d be interested to hear if anyone has cracked a workflow with notebooks for larger teams. Notebooks are easy for solo or very small teams, and the literate programming style benefits still apply in larger teams but there’s a lot of friction: “hey just %run this shared notebook with a bunch of useful utilities in it - oops yeah it tries to write some files because of some stuff unrelated to your use case in there (that’s essential to my use case)”

My current best that I know of is to keep “calculation” (pure) code in a .py and just the “action“ (side-effectful) code in the notebook. Then as far as physically possible, keep the data outside of notebook (usually a database or csv’s). That helps avoid the main time sink pitfalls (resolving git conflicts, versioning, testing etc) but it doesn’t solve for example tooling you might want to run - maybe mypy against that action code - sure you can use nbqa but… interested to learn better approaches.

I actually think the problem you are describing is actually sometimes helpful from a design perspective, if you can be conscientious enough to periodically review your notebooks and figure out what is the actual useful code which should be properly integrated into the codebase vs what is the “one-off” / non-modular code. Like you mentioned, calculation vs side-effects is one way to help you decide but not the only. There’s definitely no single answer. The key is to just periodically figure out what ought to be refactored into library code, which notebooks should just be straight up deleted (hopefully as many as possible - you can always get them back in your git history if needed!), and so on.

I'm mostly in the camp that notebooks aren't that great for software development, they thrive as an "excel for coders" of sorts, but take a look at nbdev from fast.ai.

The literate programming aspect is very nice and I wish it was explored more.

marimo is really cool, albeit "pure python" is only true insofar as the diff is concerned. other than that, it's an unconnected group of functions that need the marimo runtime to stitch together.

would be cool if marimo could "unroll" the compute graph into a standalone python script that doesn't need the marimo library

it’s already possible to do this: marimo export script nb.py

Pure-python also helps to work with existing tools out of the box: formatting, linting, pytest, importing notebooks as modules, composition, PEP 723 inline metadata

If you are going to do that, you could stick with Jupyter + nbconvert.

I rarely use notebooks directly anymore unless I require the output to be stored. Do most everything in VSCode with interactive .py files. Gets you the same notebook-y experience + all of the Python tooling.

marimo notebooks are actually DAGs as cells, reusable out-of-the-box in three diffferent ways: as interactive computing notebooks in a reactive environment with no hidden state, as Python scripts, as data apps. So Jupyter + nbconvert (or perhaps you meant jupytext) is not a replacement for marimo.

Reply on news.ycombinator.com