Hacker News Reader

Comment by yomritoyj | Hacker News Reader

I love how notebooks allow the mixing of code and output and support incremental development by letting you choose which cells to execute. But I find the semantics horrible. Each time you execute a cell you do so in an environment that depends on your entire history and cannot be figured out by simply reading the notebook. I wish for a environment which would have the same semantics as a script but which would snapshot the environment at the entry to each cell so that when a cell is modified execution does not have to resume at the beginning. Even better if downstream data dependencies are tracked so that after modifying and reexecuting a cell we know which downstream results have become stale. Does such an environment exist?

View on HN

I think this persistent state is one of the main advantages of the notebook environment, or the Matlab workspace, which I guess it was inspired by. It allows you to quickly try alternative values for certain variables without having to re-calculate everything. Saving snapshots would not be feasible if the project contains large amounts of data. If you want to reset everything, just "run all" from the beginning, or use a conventional IDE with a debugger.

No, not Matlab but Mathematica: "We were inspired originally by the excellent implementation in Mathematica" [1].

[1] http://ipython.org/ipython-doc/dev/whatsnew/version0.12.html...

And that came from Emacs and old Lisp environments---and perhaps something yet earlier?

As late as 2000, this was the single biggest advantage and single biggest impediment to new programmers in MIT's 6.001 lab: a bunch of nonvisible state, mutated by every C-x C-e. The student has tweaked two dozen points trying to fix a small program, and re-evaluated definitions after many of them, but maybe not all. The most straightforward help from a teacher is to get the buffer into a form such that M-x eval-region paves over all that, sets a known environment of top level definitions, and---more than half the time---the student's code now works.

I have similar concerns about much of Victor's work, for the same reason. Managing a mental model of complex state is a n important skill for programming, but it's best learned incrementally over long experience with more complex programs. These very interactive environments front load the need for that skill without giving any obvious structure for helping the student learn.

Contrast Excel and HyperCard, which have no invisible state: you can click and see everything.

But you cannot recalculate if your calculation has trashed your inputs.And if it hasn't then the snapshot does not impose a cost. If you are willing to forego the opportunity to replay to save memory, just put the producer and consumer in the same cell.

It is essentially already that if you are careful with the variables.

Currently, you can easily control which things are saved and which aren't. So it is the best of both worlds.

That's why I've just resorted to using 'Restart & Run All' as my default way to run a notebook I've finished coding up.

What you describe looks like reactive programming and lazy evaluation and Mathematica has support for this using the notion of Dynamic. This is how actually Manipulate is implemented.

tonicdev seems to do it, though I have never used it: http://blog.tonicdev.com/2015/09/10/time-traveling-in-node.j...

note: it does undo, but not recalculation on demand ala excel.

I imagine with a little introspection, a notebook can figure out dependencies between cells and cascade as needed.

I guess the challenge is to do it efficiently in the presence of pointers, implicit or explicit.

There's great work on this called differential dataflow. You have to trace at a finer grain than cells, but can then avoid lots of recalculation.

The real simple thing would be to paint a border or just a sideline in cells that have already been executed.

Just that would make everything much, much better.

Thanks, that makes a lot of sense. I've opened an issue: https://github.com/jupyter/jupyterlab/issues/405.

Reply on news.ycombinator.com