Notebooks Are McDonalds of Code
yobibyte.github.io
yobibyte.github.io
A REPL, by its name, is a very narrow version of the broader paradigm of interactive computer programming environments. But Notebooks are not REPLs, unless you use REPL to mean "interactive programming environment" and not REPL. Notebooks are much broader than a REPL! In a notebook, you can go back and edit and run individual lines of the notebook without re-running the whole notebook from the start and without re-computing everything that depends on what you just edited. Behavior like this makes it super hard to track the actual state, and super easy to lose track of how things are how they are. That's pretty terrible!
The parent article links this great talk that goes into more detail than the parent post and is much easier to understand: https://www.youtube.com/watch?v=7jiPeIFXb6U
Isn't this a standard on REPLs as well? You can select the code you wish to run, and press Ctrl+Enter or what ever. I must admit, I've programmed Python for about 10 years in Spyder and VS Code now, but I haven't used notebooks at any point. Just either ad-hoc scripts or actual source files.
My definition of a "notebook" is an ad-hoc script, split into individual "cells" which are typically run as a whole. On my workflow, I just select the code I wish to run. Sometimes it is one expression, one line, 100 lines or 1000 lines depending what I've changed on the script.
Not usually, no. Type `python` at the command prompt - what you get is a REPL. Type `clisp` at the command prompt, or `wish`, or `psql`, or `perl` or even `bash` - those are al REPLs.
Very different to a program that presents an editor, and then lets the user selectively choose which lines/expressions in that editor to run next. For example, type `emacs somefile.sql` in the command prompt. The application that opens is most definitely not a READ-EVAL-PRINT-LOOP.
For the same reason that adding (working) wings to a car makes it not a car anymore.[1]
I mean, to my mind, when something is satisfying a different primary use-case, then that thing is a different thing.
I'm sure there's some fuzziness in the distinction between "This is a REPL/car and this is a Notebook/plane".
Usually it's very easy to see the distinction - the REPL is waiting for the next command and the next command only while the notebook takes whatever input you give it, determines whether it got a command or content, and reacts appropriately.
[1] Tons of examples, TBH. I don't refer to my computer as my calculator, even though the computer does everything a fancy calculator can do. People don't call motorcycles 'bicycles', even though the motorcycle can go anywhere that a legal bicycle can go. More telling is how people don't call their computer monitor 'TV' and don't call the TV a 'Monitor' even when the same actual item is used for both (i.e. I repurposed an old monitor as a small-screen netflix-box, and now an item that used to be called 'monitor' by wife and kids is called 'TV' by wife and kids).
Jupyter Notebook absolutely is a REPL, see my sibling comment above for the WP link describing it as such. It waits for input, then evals the input, the prints the return value, and then loops.
“A Jupyter Notebook application is a browser-based REPL containing an ordered list of input/output cells which can contain code, text (using Github Flavored Markdown), mathematics, plots and rich media.”
I’ve got an ever growing tool set of F# data related functions that I’ve moved to a personal lib in my dotfiles and use in scripts, notebooks, etc.
Then you still get a digital lab notebook that ties together scripts, plots, and documentation, but the scripts remain usable standalone.
Edit: jinx!
(1) The calculational methods we used - could either be a set of mathematical equations or a description of the algorithms. (2) The results of evaluating these equations/algorithms for different parameter values. Usually some graphs, and some discussion of their meaning.
A Jupyter notebook is designed to replicate that process but make it easier because the figures are produced by code right there. Personally, all my notebooks include a discussion in the markdown cells what I am doing, and why. It includes discussions of the code. And directly from the code, some graphs or numbers, with a discussion attached.
With the script workflow, I would have two different files. One with the code, and one with the results pasted in. It's annoying when my primary goal is to develop and test the algorithms under discussions. Best thing is, if done right, my work is completely replicable. Just run the notebook again.
Just because some people misuse the tool doesn't mean the tool isn't useful.
This is especially useful with large datasets. Even if serialization is straightforward, if you have enough data (or the data is remotely hosted), loading it might take anywhere from 2s to multiple minutes, and even 2s is enough to get you out of the flow if you are working rapidly and want quick feedback.
It's also useful when one of your cells goes and queries a slow API for a bunch of data — I do this all the time with Datadog.
if your calculation is long running you would not be as productive as could be in notebooks
In contrast to Jupyter, you’re still working with plain text files and not JSON, and you don’t end up saving the cached data in the same file as the script.
In contrast to Quarto, Jupytext, etc., this is still just a code file and not a MarkDown file with code blocks. Not all editors have fully working “go to definition” etc. in MarkDown code blocks, and in any case, many people need a standalone script that can be placed in an HPC job queue after initial local testing is done.
For example, there are a lot of cases my team uses notebooks for proofs of concept where we make a large expensive call to load a large chunk of data, slice a small piece of it, iteratively try to reprocess the piece until you get the reprocessing to occur the desired way, validate it reprocessed correctly, and then extend the reprocessing the the rest of the data set. That can all be done after only making 1 expensive call. Further more, if the last cell evaluation fails, it just resets you back to the line before and you can retry it.
Can you do this with a script? Absolutely. You can write a script to download the data, and a script to process the data, and sub scripts for the individual steps. But that's not the path of least resistance; the path of least resistance involves you having to edit a piece and recompile everything and reset the entry point. Avoiding really makes it easier to brute force to the desired state ASAP.
Can't really do that in a script unless you're running TempleOS
If your script requires loading 12+ GB of ml models into a gpu before running anything at all this is the difference between a few seconds and a minute to see a change also if the output isn't text you can see the image or chart result inline to that code.
Then I started mentoring a junior staff member who worked on another project. The lead of that project was a physicist who wrote primarily in Jupyter notebooks. Like, thousands and thousands of lines of code. This junior staff member spent like 90% of her time confused and copying-and-pasting between notebooks. She had no idea what a virtual environment was and her go-to solution for solving import errors was to nuke and re-download the repo and re-run the setup script, or create a new notebook from scratch containing all the prerequisite user functions.
Was top 10 most horrifying things I’ve ever seen. I advised her immediately to stop using notebooks for development and sent her a few Python tutorials. Luckily though she just left the project and got on a better one instead.
Good SWE practices are 100% not taught in school.
All the greats carried one and I too carry one.
Out-of-date cells happen because Jupyter works like a buggy makefile that doesn't reliably rebuild dependencies, forcing you to run "make clean" when anything weird happens. There are better build systems.
Observable notebooks will automatically rerun cells that changed, like a spreadsheet. It works nicely for calculations that aren't too heavy, but it might not be what you want for a heavy batch job.
Their newer tool, Observable Framework, works more like a regular build system. You can still have it automatically build when you save a file in your editor.
A second complaint, that it's browser based, is basically an editor preference. You can open Jupyter notebooks in VS Code if you prefer.
F# based .Net would've made a fine math-centric environment, but it's a language with a high initial barrier, and though .Net community has been making a decent effort at porting over a lot of NumPy/SciPy, it's not fully caught up after many years.
We have Julia now, it's jitted, multicore/GPU friendly, and has interesting REPL innovations, and yes, serializable state (though, ugh, ligatures). But every time I reach for it, it's like, oh no, yet another thing I want to call is in the Python ecosystem. Especially all the modern deep learning/tensor stuff, where the assumption is Python's speed and the GIL don't matter because you're just gluing together GPU calls.
In terms of NumPy port, for more involved implementations you might be better off using built-in numeric primitives together with upcoming Tensor<T>(kind of like ndarray on steroids) and TensorPrimitives (BLAS).
There are undergoing efforts to improve numerics performance on Smalltalk. Pharo, for example, uses JIT, and already beats pure Python. But numerics in Python is mostly outsourced to fast modules written in C/Fortran. So people are in process of making Pharo's counterpart to NumPy/SciPy (PolyMath) leverage BLAS/LAPACK integration as well. See:
That working with numbers is a craft in itself. And the primary driver for the craft are the numbers. Not SWE practices.
You can't "feel" the numbers just by coming up with a huge testsuite.
And pretty often, the feel goes missing when you translate your prototype notebook into "real code".
Socrates, as you recall, famously argued that writing was a detriment to thinking. The parallel that notebooks are a sign of lazy thought does not go unnoticed.
Being against coding in a code notebook is more like saying "yellow highlighter on sticky notes is a bad medium for serious writing"
On the other hand, it accomplishes a goal other than getting a programmer paid. So in one important sense is objectively better than probably 80% of the code I've seen people paid to write, no matter how nicely it was constructed.
That backend had decent integration tests, and it took less than 5 minutes to add a small new feature. Most prod code doing comparable things where I work is worse-tested, less reliable, and at least 10X more expensive in terms of SWE-time, so I think modern programming standards are actually nonsense even though I can play along with them.
Now when business requirements change and you're constantly sending people into that 1600 line backend function or whatever, it's likely worth it to start doing some refactors. But otherwise if I don't have a good reason to be in there I pretend I don't see it.
Could still be the best approach, if by "modify" you meant "added new queries to an array of strings".
I spend a good deal of time covertly teaching software engineering best practices to folks who claim they don't want to be "software engineers", yet they are in front of Jupyter most days.
I'm a so called "scientific" programmer, and I use notebooks. I care about good programming practices, and have made an effort to improve my craftsmanship.
And I don't want to be a software engineer. I'm surrounded by SWE's in my work area, and their job doesn't really interest me. Division of labor. I'm glad somebody enjoys it.
I still have masses of admiration and respect for him, but no longer so much when it comes to software engineering - due to this, among other things. It seems like a great idea, but is clunky, hard to refactor and to integrate with other things, in my practice.
Wrapping production code in jupyter cruft doesn't work for me. Or, any code, really, that doesn't stay very small and stand alone.
I use notebooks occasionally for the university classes I teach. I like how I can write equations and text to explain ideas, and present interactive graphs to illustrate the ideas.
But it's quite a lot of work to set things up, and there can be a problem when components of the work take a long time.
In my research work, it is common for a calculation or a graph to take hours to days. Doing work like that in a notebook is just a non-starter. I use scripts (in various languages) along with Makefiles that "know" if I've changed my code or my data, and only rebuilds a result file or a graph when require. Almost always, I separate code that does analysis from code that creates graphical displays. And of course the text (of a paper, course notes, etc.) is written to incorporate these numerical and graphical results. The whole point is to subdivide complex tasks into smaller tasks that can be executed in an organized way.
I don't see the sense in using one tool (notebooks, say) for simple tasks and other tools (scripts, a strong editor, tmux, unix, etc.) for complex tasks. So, apart from the toy tools that I make for some teaching tasks, I stick to simple unix-style tools for the tricky stuff.
It's important to have many tools in the toolbox. Notebooks can be useful for some things, but I wouldn't want to frame a wall using an awl.
Certainly you can shoot yourself in the foot with notebooks, but you can also footgun with many other tools.
You generally shouldn’t put code you want to reuse in notebooks, but we haven’t found this to be much of a problem. And >3/4 of the people who’ve worked on our team don’t have software engineering backgrounds. If you set clear expectations, you can avoid the really bad tar pits even if most of your co-workers are (non-software) engineers or scientists 0-3 years out of college.
It's harder to breach that barrier in embedded, because someone has to approve of getting the hardware made.
In most organizations, there's a kind of no mans land between hacked code, and professionally written software. That's because the real SWE's are always overbooked, and for good reasons -- business and career wise -- should be working on the biggest projects. This leaves software needs that are urgent for somebody but not urgent enough to rise to the top of the priority list for the software team. The somebody could be an internal or external customer.
That explains a lot. I have seen a lot of SWE that believe they are working on bigger project but they are not. Probably 90% of them.
These are the guys that will produce a lot of useless abstractions without producing any real value.
I guess they aspire to work with kafka/k8s at Google but ended up at the local shop.
And nothing wrong with the local shop. This is probably the place where you actually have the possibility to produce real value.
Technically, I was using gradio to create a localhost webpage and then piping it through cloudflare. The website would only work when the notebook was running on the cmd line of EC2. Hey if it works, it works! Notebooks allowed me to do a 2 week project in 2 hours.
Like most things, notebooks have their pros and cons. One of the biggest adv is very rapid experimentation. Even a regular script takes a while for python interpreter to run and that time (even if only a few seconds) adds up in lack of creativity ("Bret Victor - Inventing on Principle" [1]). And if your script is loading a big database, then notebook is a no-brainer.
One of the biggest disadv of notebook is mis-ordering. You are allowed to declare a variable in cell 3 and then go and use it in cell 2. Even worse, you can declare a variable in cell 3 and then delete cell 3 and still be able to use that variable. That I believe is the biggest dis-adv of notebooks. It adds way too many subtle errors. One way to bypass this is to write everything in functions - no global vars.
I am willing to accept the issue of mis-ordering in order to get rapid experimentation. It's subjective whether you think the pros outweight the cons. I definitely think they do.
So... your alternative framework was too enterprisey for your purposes?
One of my family members literally maintains a software platform for test bedding production NASA spacecraft at JPL and it is all on top of Jupyter notebooks.
People are literally on course to build AGI and there’s a good chance that large portions of that work will be done within Jupyter notebooks.
To suggest real work doesn’t get done in notebooks is simply asinine.
As I have embedded further into data science and machine learning teams within my career I have come to learn that a lot of the things software engineers care about around maintainability, reusability etc. are often completely antithetical to data science productivity. In many cases practitioners would consider some of the listed cons of notebooks as features rather than bugs.
There is obviously a time and a place for pulling out those practices in such settings but blindly applying software engineering standards to this kind of work is both all too common and often rather counter productive.
If notebook style dev doesn’t jive with you personally I get that, and god speed. I’ve worked on CV/ML teams with a lot of CS degree grads and they don’t always want Jupyter notebooks. However, I’ve noticed a tendency for folks to extrapolate that to blanket statements about notebooks in general. As someone who has inherited and had to clean up many jumbled/bad notebooks I’m empathetic but I would challenge folks to lean into considering more about the circumstances the previous notebook author was working in and what kinds of productivity outcomes they were responsible and prioritizing rather than casting judgement on said author or their tools of choice.
Citation needed.
However, the problem is that those things are not trivial and have to be learned before they can be applied. And that is where the value becomes questionable - the time spent learning all that could instead be spent on learning other things. But if you already have a software engineering background, you do benefit from doing things "right" to some extent.
I agree that notebooks are best reserved for exposition.
What are the "right abstractions" that everyone seems to miss?
This is actually one of the best things about jupyter is that you can just tunnel into the computer and use the exact same env... this article is WACK
That said, when I'm writing code, if I want it to be good quality code then I might use org-mode and its literate programming facilities to type up my thoughts, interlineating code as necessary. This is more useful on greenfield and personal stuff, but there is a high probability that I too have undiagnosed ADHD and the use of org-mode helps to organize and clarify my thoughts before I commit them to code. It is a hallmark of the programmer personality to jump into coding without thinking things through beforehand -- or really, sometimes, at all. The management consultant Tim Bryce wrote about a programming class he took in the 70s:
"When given an assignment, the other two members of my team started coding immediately, then keyed up the source code (Yes, we were using punchcards back then), and kept compiling and debugging until they finally got a clean compile. While this was going on, I pulled out my trusty old template, worked out the logic, wrote the code, keyed it up and usually got a clean compile on my first shot. Inevitably, I beat my teammates everytime."
The use of org-mode and LP helps with these organization and planning issues... well, at least it does for me.
The article really makes me think either I or the author have a terrible misunderstanding about what notebooks are or how to use them.
I sat there reading through this thing, staring at my, spiral bound, paper notebook being utterly confused at what in the hell he was talking about.
I took a random comment, here, saying it for me to realize what was going on.
Most code in jupyter notebook is bad, but that's fine, because nobody should waste time doing that unless they must reproduce the results or they are sharing the notebook with others.
Notable example of high quality notebook: https://youtu.be/zduSFxRajkE
Add a "why I might be wrong" section, or "why this might not apply to you". Just show me that you've thought from more than one side of the issue and I'll be more receptive to your side.
I'm not even a programmer, I'm a data scientist 1 year out of college, and I still know that notebooks should never ever be deployed to production!
The right way to view notebooks is as.. notebooks. Text and visualizations that can run online code. They are for presenting!
You would need a tremendous amount of incompetence and a lack of any code reviews or tech leadership at all to end up deploying a notebook to production.
having said that when i see Jupyter notebook I close the tab and do not continue
theres just a sense of dread that I can't explain...perhaps its PTSD from data structure/algorithm comp sci courses that i never completed.
But I often did.
Frankly, notebooks are pretty terrible for exploratory data analysis, especially in python. The main issue is that they absolutely kill interactivity. Straight up CLI based ipython is far more interactive, and often better for exploratory data analysis. In a lot of ways, pretty much any other interface is better than code blocks that aren't independent but can be run independently.
Part of it is the interface (i.e. non-linear state, code blocks, etc), which is great for teaching, but bad for exploration. However, i.m.o. even more of it is the fact that it's browser based. E.g. folks often think it's best to use matplotlib in a notebook. However, using a notebook means you get very little interactivity out of a highly interactive library and remarkably poor performance out of something that's actually _very_ interactive and quite performant on standard Tk/Qt/etc backends.
If you ditch browser based stuff, you're usually far better off. This part is mostly just me griping, but I don't understand why we've stepped back 20 years in basic usability, interactivity, and performance by making everything use a web browser (and then making that not even remotely cross platform).
I've taught and worked with some of the biggest companies in the world. They were using Jupyter and sound EDA to great effect.
It's not that they're not a good tool, it's that they're the wrong tool for a lot of what they're used for. They're an excellent teaching and documentation tool. However, they're remarkably bad for actually doing any type of analysis or exploration. They're a great way of presenting your analysis after you complete it, but they're bad for actually doing that analysis. Write analysis code and data exploration code using the same tools you would for production code. There's nothing wrong with an edit + run + explore loop. Do that with a standard REPL like ipython + your preferred editor.
But there's a lot wrong with trying to use a notebook to write and execute code, as it: 1) encourages writing code that won't actually run later (dependent on non-linear code execution and unsaved changes made to earlier cells), and 2) is a poor environment for writing code compared to more full featured editors or IDEs, and 3) discourages interactivity and data exploration due to being browser-based (poor performance compared to running directly limits interactivity during data exploration).
Your data analysis work needs to actually run independently later. Notebooks rarely do unless they're re-made from scratch. At that point, you're better off working in a more full-featured environment for the exploration phase and then using the notebook as later documentation / communication.
The second thing is the presentation of the plots: it's really handy to have multiple interactive plots laid out right next to the code block that produced them.
There's nothing that requires that this is done in a web interface, but no-one seems interested in making a non-web version (IIRC there was a QT version in the very very early days of ipython notebook but it's long dead). One big thing that any replacement will need is the easy ability for the code to actually be running on a remote machine, something that's trivial for a web interface but requires active effort for a native client (and makes a lot of interactive chart work more difficult: certainly there's currently no other matplotlib backend which can do it).
I'm fairly sure a better interface than jupyter is possible for data exploration, but I haven't found one yet since there's not really anything else that gives you the two features above.
(one thing people often miss about notebooks and the kind of work that go into them is that the code itself isn't actually really hard code. It's 100% about interacting with the data and the algorithms running on it. Working in one isn't much like software engineering)
And that's why I use elpy to execute codeblocks marked by comments in IPython. VSCode supports this workflow out of the box (but you don't get the matplotlib windows out of the box ;)) - although it is very, very liberal in using screenspace...
> One big thing that any replacement will need is the easy ability for the code to actually be running on a remote machine
VSCode remote (proprietary sadly). Also tramp + SSH-X-forwarding likely beats whatever ergonomics jupyterlab offers.
Jumping into the middle of a script and editing is often a bit dangerous. You usually wind up with something that can't be reproduced, as it depends on modifying state in a way that wasn't recorded. You do often need some sort of caching during exploration, for sure (it's not really feasible to wait an hour every time you want to tweak a visualization), but it's often better to break pipelines up a bit and do disk I/O rather than leaning on cells in a notebook.
I do agree on the remote client part, though. Browser-based interactivity often is a good solution for a dev instance + a local thin client. It's slower than native for a local solution, though.
To save everyone time, I think the fundamental disagreement comes down to this. A vast chunk of the audience of notebooks is people whose brain has not been drilled on the daily about best practices and maintainable code and often don't get judged on that criteria or affected by the consequences of this. If you believe that these people are irredeemable as engineers, or more charitably that the incentives driving this cannot be fixed, then you probably believe in taking away good tools from these people because they'll misuse it, and then you'll agree with these articles. If you don't, then you won't.
> I am using code autoreload and write code in modules >> Nice! I've done this for a while as well. This is a good use-case. However, this approach does not address some of the issues, e.g. data analysis scripts should be checked by another team member.
How come? When you write your code in a module, it gets committed and PR'd and linted and tested and reviewed just like any other code, which is the whole point of this approach. Which I think is the most reasonable and mature one.
The notebook then is mostly reserved for a very thin interaction layer optimized for play, on top of your vast iceberg of reviewed & tested & maintainable code. Including lots of prewritten convenience routines for plotting and munging to minimize business logic in the notebooks further, and not have paragraphs of matplotlib crud in hidden cells. You construct what's almost like a shell + DSL that's optimized for the domain and datasets you care about, and you can focus on investigating. If there's more significant code, it soon goes in a module again, without interrupting your ongoing flow and thought process.
I messed around with paperspace for a while, but I could not get detectron2 working in it so I went back to google colab. And my ML projects, that I have ambitions to become production apps, are still toys so for now the quick and dirty notebook will have to do. But yes, I prefer to be in my element in a traditional IDE, pushing to git, tabbing between files. I certainly have > 10 years of experience doing that, but not really for ML projects if that makes sense.
I don't go to McDonald's if I want a salad.
Obsessing over silly details like these can be deceptively tempting procrastination.
So well... Writing in org-mode makes me waste a bit of time but produce much clearer designs because I do not design in code but in natural language and if the narrative is suboptimal is immediately clear, also it proper document the code non with comments but with complete human language. The downside it's that's very time consuming and I do loose some coding helpers that can't work much in babel blocks instead of full source files.
So well, for me such interaction is the most powerful UI we have, the best human-computer interface so far, the default go-to solution for any day-to-day work. A project could be an independent entity but still the day-to-day work born and evolve like that, so I'm not agreeing much, notebooks are not quick&dirty stuff but a tool to reason and reasoning often take much more time than directly write code, if you are disciplined. On contrary might disperse you if you are in "silicon valley mode". This is probably than main point not of the tool, but of the human using it: how many are constantly in Silicon Valley mode vs in Donald Knuth mode?
But for some reason, no matter what nonsensical environment you're being forced to operate in, there's always a Jupyter web interface with access to exactly what you need that none of the auditors, accountants or systems administrators seem to care about. It may not be good, but it works, and won't be bricked by some inexplicable change to IAM roles or VPS settings applied by some automated policy tool you didn't even know exists.
For a notebook/repl environment, you can create any number of intermediate steps, rerun the previous step with minor modifications and check if the results are better, rinse and repeat. For jupyter notebook specifically, you can visualize data and add markdown inline which are very useful.
You won't understand it unless you are already familiar with the workflow.
I wasn't too active in ops stuff at that job, but as a dev it was so awesome for them to hand me off a notebook that was basically an in-progress investigation, including the entire path of exploration they took to narrow it down and dead ends along the way. You can't ask for a better bugfix ticket.
Now I guess a lot of that depends BEAM particulars, and doesn't apply to jupyter or whatever. But it's at least an illustration that notebooks can uncover novel workflows for certain tasks.