PANE: Programming with visible data | Hacker News Reader

PANE: Programming with visible data

joshuahhh.com

119 pointsmpweiher24 comments

Data-centric programming is a great insight for making development user-friendly. Spreadsheets are the second-most successful end-user-development tool, largely because it provides immediate results for any function evaluation.

I have the hypothesis that any programming tool geared towards non-programmers will need to be embedded as an extension for an app with a specific purpose (i.e. not a development environment). Users will understand better how to create an automated process if they can work on real data while writing the functions.

Avoiding the blank page effect is essential for such a tool, as non-developers wouldn't know what to put in a program if they start from scratch - they'll need some example data first, on which they can build abstractions that automatize steps for processing that data. Again, spreadsheets largely work this way.

Outliner apps would be a great use case for a programmable app for end users. We have seen examples of somewhat programmable outliners such as the Leo Editor, notion.so and the like; though they lack a good low-code/no-code approach.

Many developers know how to use Emacs outline mode for keeping track of their data and build automations over its hierarchical structure; a user-friendly, easy to learn tool that worked the same way would be a hit. It should made programming functions over the data tree as easy as functions on a spreadsheet, and that's the missing part from current tools.

I was hoping for someone to ask what is the first most successful EUD tool, but there's been no luck :-P

Let me indulge you. What is the most successful EUD tool?

I believe that to be Copy & Paste. It's so humble and well ingrained in our experience that we no longer think of it as code or as a tool, yet it's what ties together most data flows in our digital environments, at least for end users who have no access to APIs. Before the Object-Verb syntax of copy-paste was invented (i.e. "select object+copy+select destination+paste"), there were only Verb+Object commands ("move+select object+select destination"), and it sucked.

Cut-Copy-Paste has been for decades the single tool that allowed end users to create their own workflows, combining several applications to carry out the successive steps of a single task not put together by a developer. Copy/paste is THE universal tool that is almost guaranteed to work in transferring information between arbitrary tools, to the point that it feels broken or artificially restricted when it's not available.

Nowadays it gets competition from intents (the "share" button) in mobile OS and the web, but copy/paste is still more flexible. And it's somewhat similar to the role of pipes in the shell, but those are our of reach for end users.

I love the typography and the quality of the presentation layer of this. I love how the text ("docs/comments") seems interspersed with the code. Kinda makes it feel like a paper about an algorithm, with diagrams that are live — but it actually is an implementation of the algorithm at the same time. Reminds me of Jupyter Notebooks in this regard. I hope embedded math is/will be possible, and also "inline comments" (though maybe making smaller functions when a detailed comment is needed might be just enough).

I love how the author explicitly mentions a lot of related work (esp. the new one like Luna, Apparatus, but also the classics, LabVIEW etc.), and tries to explain important differences.

One thing I feel I miss is that the same input nodes are not merged (with forking lines), but they're duplicated instead. Seems to make it harder to understand/notice where the data is coming from, that some of it has common origin. In other words: "merging" seems to be visible, but "forking" seems not so. As an idea: maybe a set of vertical lines representing "connection/input planes" (like "ground plane" in a breadboard) could be shown on the left (for passive but somewhat messy viewing), and then clicking on a node could make one vertical line thicker, highlighting all the flows that start from this input (for cleaner but active viewing)?

I haven't tried the demo, so I'm curious about a few more things at the moment:

- how well does it handle slow functions? (e.g. does it hang, or does it allow working during computation and slowly propagates the results through the graph) how well does it handle bigger data sets? (lists with hundreds, thousands, millions of elements...) complex composite data structures? (trees, graphs, ...)

- does it allow "export/compile to JS" for easy running of the code built with it in "production"? also does it maybe allow "import/decompile from JS"?

If you need "export to JS" in a similar app, you might like Mesh Spreadsheets: http://www.mesh-spreadsheet.com

Slow functions and trees/graphs are currently a problem with Mesh as well, but timeouts may take care of the former, and we're thinking about the latter: https://twitter.com/MeshSpreadsheet/status/10794804493138206...

I had not seen Mesh-Spreadsheets before, thanks! I was using Excel with F#, however, the portability and ubiquity of this is very enticing.

Nice! The one big problem with most visual programming paradigms is that they’re not abstraction resistant. Pane counteracts this through aggressive concretization, not by suppressing your expressively, but by showing all the concrete data flowing through it. The usage of data here is multiple folds:

- it serves as static, TDD-like documentation.

- it serves as reactive cells you can manipulate. Note that you can be wishy washy about the reactivity’s semantics here because it’s only used for example and quick-and-dirty explorations, not for runtime correctness of the program. Pragmatically speaking, this is crucial. Some of the future directions mentioned at the end might conflict with this and make the environment less appealing.

- it serves as a very convenient mechanism for defeating the double-edged sword that is program abstraction.

The pragmatism of this project gives me more hope than most of the cited references. Actually, I believe that the more you dive into these paradigms, the more pragmatic you need to stay in order to achieve success. This is intrinsically related to the fact that dimensionality reduction (part of un-abstracting a real-world software program) can only be 80% correct. Visual paradigms that enforce you into a black-and-white way of doing something don’t scale.

Kudo to the author for resisting the temptation of Exploring some of the overly puristic directions! The polish definitely helps too. Some of the perf problems are beginning to show in the latter examples though.

Also, in terms of editing efficiency, this paradigm might work better on tablets, especially through some intuitive multitouch gestures. But then your audience decreases dramatically.

Edit: oh wait, this is the Joshua from Dynamicland! Small world...

> Pane counteracts this through aggressive concretization, not by suppressing your expressively, but by showing all the concrete data flowing through it.

> you can be wishy washy about the reactivity’s semantics here because it’s only used for example and quick-and-dirty explorations, not for runtime correctness of the program

Those are very good insights about development environments.

The thing is, developers have largely benefited from those same advantages for decades. This is essentially how they use REPL and visual debuggers / panel inspectors, by constructing program segments without the requirement to put them all the time into a single correct main program. The sad thing is, until recently, those exploratory environments have been strongly tied to syntax-heavy formal programming languages. Only recently we have started to see practical interactive programming in more visual, "syntax light" tools.

The novelty thus in PANE would be making those exploratory sessions somewhat persistent by using a visual notation. Jupyter notebooks largely achieve the same goal, by allowing developers to save the status of a command-line environment (the sequence of commands typed) for later reuse.

The video is very worth a watch all the way to the end. I work with Alteryx for data processing because I prefer to hire business analysts vs programmers, but the tool is expensive and a bit clunky in few areas (mostly because it’s desktop first, not web first). This concept looks promising, I have few questions: 1) how do you plan to work with bigger datasets (say 100 columns and 2 million records)? 2) in your point .3 in “connecting to the world” have you thought about where the computation can happen? For instance, if data is in a database, do you envision pane running funcions directly in the db through sql?

So much great stuff here! Rough around the edges, but it's a research prototype. Thoughts:

- This reminds me of Subtext ... oh, they cited that as an influence! I love it when people learn the state of the art and take previous work and build on it. Surprisingly uncommon.

- Paraphrased: static types are an indirect way to look at a function's behavior, data is a direct way. I never thought of it that way!

- Is that a single quote used as an icon to indicate that it's a string? Neat! (I use icons in Strukt to indicate data types, but I went the other way, and no-icon means string, with a "#" icon indicating a number.)

- This is the second time this week I've seen a system where you could connect nodes by either dragging an edge, or typing a name. Is that the best of both worlds? Are we converging?

- I love that names are optional, and not restricted to C-style identifiers. This is a perfect example of something you can't easily do in plain text: without an out-of-band place to put identifiers, you need to deal with escaping, or character ranges, or ...

That said, while I love the concept, I disagree with them on some basic points. One that wasn't explicitly mentioned:

- It's still a highly-specialized mouse-driven environment, and I'm still not convinced there's any future in those. It makes for a nifty demo, but plain text really took off because everybody has a text editor which can edit any text, and learning to edit one format helps you learn how to edit other formats in the same editor. (Can you tell I'm an Emacs junkie?) What I want is a generic keyboard-driven GUI graph editor. It should be able to be hooked up to live visualizations, via other processes, too. It just needs to be able to edit any kinds of graphs. I don't want my investment in learning this editor to be limited to one specialized type of (mouse-y) programming.

Somebody needs to make a great graph-based editor so all these people experimenting in this space don't need to keep re-inventing the UI wheel. (Note to self: me?)

> This is the second time this week I've seen a system where you could connect nodes by either dragging an edge, or typing a name. Is that the best of both worlds? Are we converging?

Working on anonymous objects might be the single killer advantage of GUIs over textual environments for defining automated processes; coming up with good names for variables is hard, and GUIs limit the need to do that to only the essential objects.

Conversely, retrieving objects by name is an essential advantage of textual entry typical of classic programming languages. It allows connecting your current expression to far-away objects in the system, which is hard or cumbersome to do by navigating a graphical interface.

So yes, that is probably the best of both worlds. It's also what spreadsheets do, and they're considered a very successful development environment for end users; so, you might be onto something there...

> What I want is a generic keyboard-driven GUI graph editor

Have you seen Jef Raskin's Canon Cat, and Humane Environment (a.k.a. Archy)? There's a lot of good design on how to build a generic keyboard-based GUI.

I also have some ideas for how such a general, keyboard-driven graph editor could work, largely inspired by Raskin's Humane Interface... One of those would be using comic strips to represent state transitions, as in the example at slide 23 (minute 22 in the video).

P.S. Mockup tools for rapid prototyping are typically heavy mouse-driven, and they're very good for exploring user interactions. So, there may be a niche case for preferring direct manipulation for exploratory work when creating new automations.

I've seen some videos of Canon Cat, but I must be missing something because I'm not aware of it being any sort of GUI. It looked to me like a text editor with "no save" (always automatically synced to disk), and keyboard support for Emacs-style isearch (very cool).

For ideas on how to map keys to nodes/edges, there's other places I'd look. For example, lots of video games let you easily move along edges of a graph using just 4 (or 8) direction arrows.

"State transitions" are one very specific kind of graph, and my thought is a generic editor. Think lower level. State transitions as comic frames could be a major-mode that the user installs, if they know they'll need to edit a lot of transitions, but it should also support "I know nothing about this except it has nodes/edges", like fundamental-mode is for text.

Canon Cat did not have graphical capabilities at the time, but its successor the Archy prototype had them. Instead of being mouse-centric like the Mac/Lisa, it was intended to handle graphical objects mostly or entirely by keyboard (like Emacs, but without modes), and Raskin had plans to make it work like an infinitely zoomable browser. The think with Archy is that it contained all the power of a command line for chaining commands, but it also worked on documents, all with a very well thought out interaction model that's easy to learn (in contrast with Emacs, which offers the same power but it's a nightmare to learn).

Yeah, a low-level "graph editor" extensible library that could be plugged-in anywhere to provide a common interaction, that would be great. Things like tree widgets provide some of that, but of course they only support a specific subset of graphs.

Reminds me of Pure Data (PD)

https://puredata.info/

And Luna: https://www.luna-lang.org/

Grasshopper 3d (https://www.grasshopper3d.com) is a pretty intuitive tool based around similar ideas but for the creation of 3D surfaces and meshes.

My two main insights from using it is that 1. visible intermediate data matters a lot, as PANE points out, and 2. Grasshopper figured out a pretty clever and powerful system of trees and branches for grouping data together.

Ever since using it I've been thinking that the world needs something similar but for data in general. Hope that PANE will be it.

Love that video. Highlight on the two differences basically explained everything!

But I'm wonder if the non-patch(i.e. foregrounding that data) idea would also be helpful outside development. For example, helping scientist to model large and abstract systems(i.e. mathematic models/theories) easier.

Usually this kind of visual programming breaks down with large amount of data/operations. A simple test is trying to program a quick sort in it. I'm not sure how this solves the problem.

He specifically addresses that common criticism of "patch-based" systems. Do we have reason to believe that his 2 major innovations here won't help in the QS case?

Looks great, especially UX! Not so many people try to solve problems in this area.

I have tried to make programming simple with ChangeOver[0] which also shows data flowing through functions, but for data I'm using relational database. In these days I'm rebuilding it from the ground up(mostly UX).

[0] https://news.ycombinator.com/item?id=18543468

There is one more related work: spreadsheets.

This is mentioned in the page:

> Spreadsheets are a clear precedent for any live data project. In particular, I would like to call out Glen Chiacherri’s Flowsheets project, which explores how to give spreadsheets the power of conventional programming, while maintaining the immediacy and casualness of the good old grid. PANE aspires to this same blend of power and immediacy.

This is so. darn. cool!

Reply on news.ycombinator.com