Seymour: Live Programming for the Classroom
harc.github.io
harc.github.io
Introspecting into how I prototype new ideas and algorithms, the kinds of intricate tracing and exploration that are automatically done by Seymour are things I often do "manually", using a cobbled-together set of tools (one of which made it into Mathematica [1]).
Sometimes, because its fairly easy to do in the language, I quickly code up once-off tools to help me verify solutions. For example, I recently finished a new type inference system for neural network layers that allows very flexible 'backwards' and 'sideways' inference of tensor rank and dimensions. Surveying its behavior on all the possible inputs on the various kinds of layers is not feasible via manually typing REPL examples or unit tests. So I ran the inferencer against all possible cases of a certain form, and by visualizing the results as little squares (with various other info behind tooltips and mouseovers), I checked each result very rapidly [2] and soon found cases I hadn't thought of.
One thing I wonder from my experience of doing this kind of thing (I've written various visual debugging and logging frameworks, and code visualizers of all kinds), is whether a prescribed framework and set of workflows is actually preferable to flexible tooling that you can inject in precisely the way you want.
I've often abandoned my visual tools because they are too hard to customize into the exact thing I need to answer a specific, scientific question that comes up during debugging or implementation. Also, when you are already overloaded with thoughts and ideas about what might be going wrong, you kind of want to be able to ask and answer extremely precise questions, rather than swamp yourself in visual data that you have to interpret in light of your hypothesis. Clever presentation can only help the overload so much.
I wonder if what is needed is a kind of "side program" that one could use to annotate the main program in very precise ways and ask questions of its execution, or to produce maps or timelines of aspects of its behavior. But not via interactions with a GUI, although those are useful too, but as actual code that lives alongside the real code, using the same language. Something a bit like a souped-up, visual dtrace, perhaps. I don't know what is already out there like that, but I would love to see work in that direction.
[1] http://reference.wolfram.com/language/ref/Echo.html
It's not really shown well in this documentation (I didn't write the docs), but Echo is very easy to insert because you can just write "Echo @" as a prefix, anywhere in ordinary code, and it basically a no-op. I have a macro system that makes it even easier: the single quote character (which normally means Derivative) can be put next to any expression and does the equivalent of an Echo. Being able to inject and remove these little probes anywhere in the codebase with a single keystroke makes the good old "print debugging" actually pretty effective in a lot of situations. And various forms of cleverness make it more useful, like putting it next to an assignment will also print the variable being assigned.
The first two lines of each grid represent the inputs to a CatenateLayer, the last line is the output. Red indicates an invalid shape. Gray is "unknown". The other colors are fixed dimensions like 2 or 3, or dynamic dimensions (used for var-length sequence inputs). This display is from a buggy version of the inference system, don't try to check it!
I am currently learning SAS, an old language. I find integrating it in an environment that allows literate programming (and live programming) extremely useful. I chose Rstudio for the many visualizations available. Now I am starting to love SAS -- even the "old school" things like "cards", which allow embedding data within the document. This is extremely important for reproducible research.
But there comes a time when a precise question must be answered, and exploration must be stopped to concentrate on what is important, and cast away the details. That is when learning solidifies into something tangible.
I believe coding is initially a 2 step process, and that the 3rd part of the lifecycle of any program (maintenance!) can benefit from welding these first 2 steps together in a notebook.
A notebook allows to document the development process and refresh your memory whenever you need to look at some old code and understand why it was implemented this way.
For me, the 2 steps + lifecycle process is:
1) exploration, cobbling together a first implementation, and using Echo/print or their equivalent to make sure it works. Good old print debugging is extremely effective because it allows you to check if your algorithm follows your logic
2) refining the solution, reimplementation. Discard the Echo/prints, add a debug flag, and a test data set to make sure you get consistent outputs. Sometimes, it is even time to switch to another language if performance is an issue. Minimal documentation of the code is kept.
3) maintenance, extension. If 1 and 2 were done in a notebook, whenever documentation is not sufficient, it is extremely easy to go back to the first versions of the code, and do some tweaks.
Mathematica is a wonderful tool to do that, but Rstudio is just more useful these days to mix various pieces and languages. I find myself using Rstudio more and more for everything these day -- even for shell scripting and SQL.
By the way, I love your example [2] -- the density of information is high. I just wish there was a way to integrate the Mathematica engine within Rstudio, with a rending rendering of the output.
Do you know a way to submit a piece of code to the Mathematica kernel, then get Mathematica output properly formatted say as HTML? (and of course, automatically, in a way that can be scripted)
This is a bit like how ODS work in SAS. If it takes several steps, that's fine. I just want to create something that I feed Mathematica code to, and that returns to me the Out cells, not just as ascii text but properly rendered to integrate within Rstudio. Extra bonus point if the output is not just a html rendering, but also contains javascript to allow Manipulate.
This is a great idea. I find it ironic that so much of programming consists of 'simulating the computer' in one's head while sitting in front of a computer.
I would love to be able to visualize many things that a compiler does internally, for instance, to see how generic code looks when specialized in a certain context.
$$('.controls').forEach(controls => controls.style.display = 'inline')
It didn't work anyway: hovering over the videos toggles the visibility of controls, which fires the mouseover event again, which toggles the controls off, which leads to really annoying flickering. Of course nobody thought to sniff for that.