Sorry, I'm very new to this, how do notebooks/shared notebooks play with version control?
You are right that versioning is still an issue and we largely punt on it by using the docker container (with layer commits on each notebook teardown) as the versioning mechanism. Maybe not the best solution but it does have it's advantages.
I agree with that problem, yet somehow like the idea of notebooks. Maybe the real tragedy here is that jupyter notebooks are saved as json and not as a valid program with comments, that can be run "as is" from the command line.
You like the tight feedback loop of REPLs, as do I. You don't need the clunky machinery of jupyter to effect REPLs with emacs and ipython.
> real tragedy here is that jupyter notebooks are not [saved] as a > valid program with comments
Here's a simple way of doing that: write your code as valid programs with comments.
That's what I do! Then I have a script that converts my python program to shitty json that my colleague--who can only conceive to work inside a notebook--can run it. Finally, another script translates back the json to a readable code; and more importantly, to something that can meaningfully be put into git.
I would love if the jupyter interface allowed to save the notebook directly into a program with comments. Then all this silly sorcery would not be necessary.
But that's besides the point. The jupyter ecosystem, like other widespread and unwieldy formats such as Microsoft Word and PDF, poses a brutal obstruction to Unix workflows.
Yes, my script simply calls the nbconvert library, with some trickery to ensure that the result is idempotent. But I would like not to need this script, that instead the jupyter interface worked with valid python files directly (maybe after enabling some option).
> poses a brutal obstruction to Unix workflows.
It's not as much the notebook itself, but the file format chosen by default by the notebook. If it was a human-editable textfile there would be no problem, and there is no practical obstruction for that (other that young programmers today cannot conceive a different "serialization" format than json).
Data is the second problem - you usually can't run a notebook without it. If the notebook transforms data over time, it can be a real issue.
Here's a pretty complete run down of the first problem, the .ipynb itself: https://nextjournal.com/schmudde/how-to-version-control-jupy...
An integrated solution that versions data, the notebook, and the computational environment (collaborators will make changes over time) is the ideal collaborative platform.