Spatial languages: Writing code in 2D
shukla.io
shukla.io
Text is a serialization of an n-dimensional work space. "N-dimensional" doesn't have a formal definition I'm aware of but the major characteristic of it is that a new dimension can pop up anywhere. If you feel like you want to double-click on that, you can see my earlier comment here: https://news.ycombinator.com/item?id=35096647
A conventional textual language can represent everything mentioned there, whereas 2D layouts struggle almost immediately to represent things that textual layouts represent quite easily.
It can still be a useful "see the world differently" exercise, but it's the opposite of "freeing your mind"... it's stepping into a fairly small subset of functionality, albeit an unconventional one, and seeing what you can still manage to do even so. Rather than "unlocking a new dimension" it's actually throwing away n-2 of them. The experience of using this for any period of time would reveal this, as it would be a constant struggle to represent even very simple algorithms in this format.
I post this mostly because of the "familiarity breeds contempt" effect that programmers seem to get hit with for text. There is a reason we use text and it's not lack of imagination or being stuck in our ways... it is actually an incredibly rich and powerful format that poses a serious challenge for any alternative to overcome because of the astonishing combination of compactness combined with representational power.
Text works and is hard because it's a projection of a mental representation. The whole reason we can work with N-dimensional structures in text is because the dimensionality lives in the programmer's head. There isn't really any other way to represent say a >3 dimensional array other than laboriously flattening it into text and expecting the next guy to rotate the hypercube in his mind. That's neither a good nor bad thing, it's purely a fact of our 3+1 dimensional universe.
A shocking number of problems can be best represented in higher dimensions and there's simply no way to encode that information losslessly. Being simple 3D creatures (I'd argue 2D in fact) the only way to do that is by projecting it down into a lower-dimensional space.
Anyone that says text is 1D has a poor model of code in his or her mind.
It's surprising that there is any art at all if 1D textual representation is the ultimate way of representing concepts of the world around us. Incredible really that folks waste their time define colours for such thing as the sunshine.
Oh sorry, of course, you were talking of code - meaning that algorithm concepts are best represented in text? Again, I doubt that very much. Laziness of humans is probably the bigger reason why we're stuck with an inefficient textual representation of higher dimensional concepts.
[1 2 3; 4 5 6; 7 8 9]
|1 2 3|
|4 5 6|
|7 8 9|
In code we split the code into files. Then within the files we arrange the code with lines and indentation. That's all for humans. The computer eagerly throws those away.
ADDENDUM
> meaning that algorithm concepts are best represented in text? Again, I doubt that very much.
One of the (most?) important things that algorithms encode is time. Because an algorithm is a description of a process. The only good representation would have been to snapshot the evolution of state.
But that's wasteful. So instead we have statements and expressions that we assume are atomic. Then we have identifiers for referencing and finally we have procedures and functions to cluster those statements and the associated call/return pattern or the single jump/goto to economize on writing.
So code is at least 2D as operations are described in a line and the lines are arranged by time. But the lines are not linear, instead they are grouped into labelled units and the proper evolution are functions call and jump instructions.
\ | /
.-'-.
-- ( ) --
`-.-'
/ | \
Much like HN is full of textual description and links to images - this entire forum demonstrates the limitations of a textual visual representation.
Yes it works but it's far more verbose than an image would be. Hence an image is worth a 1000 words. And hence HN is more verbose than it needs to be. So is coding more verbose than it needs to be. That's why DSLs (Domain Specific Language) get invented: to reduce the verbosity.
And so it is with a 2D representation: it reduces the verbosity required to transport ideas and concepts. UML is another invention to reduce the verbosity and increase the understanding of the representation of complex systems. Yes UML can be represented as text (see Mermaid[1]) but the UML image is probably a lot more understandable than the Mermaid representation.
So yes in certain context, a diagram may be more useful to get a point across. But once that shared understanding has taken place, Text are more economical.
A lot of foundational aspects of computing are too abstract in nature to be able to do a good 2D or 3D representation. So what we do is transforming them into streams of symbols and do symbolic manipulations on them. And with symbolic manipulation, you get abstraction.
The main issue is that a lot of people are not good at symbolic representation and manipulation. Because it's abstract and it's entirely a product of the mind.
The power of computers is that they can do the tedious part of symbolic manipulations, repeatedly and without failing. And with peripherals interfacing, they can translate symbols from/to reality.
Exactly what I am implying: using the computer to provide a better abstraction is what is needed.
I agree that text has a certain universality however as you also say, a lot folks don't work with symbols, they work with images (for example) or colours (synesthesia) or shapes perhaps.
Besides move up in dimensions is a continuation of a natural progression in computing: cogs, switchs, punch cards, keyboards, mouse, touch .... VR, AR etc. Currently we're (coders) are somewhere between keyboard and mouse while others (most folks that use smart phones) are at touch. Smart phones users are also programmers who code their device using largely visual concepts (at least I've not seen anyone set their alarm using a terminal on a smart phone!).
Now "Oh but it's not possible to represent those concepts that we work with visually, only text will do" argument doesn't really hold water since it just demonstrates a lack of imagination to find a way to use other representations for those concepts that are essential to programming. Strangely that was possible when moving from switches to keyboards, why shouldn't it be possible even further.
Recently I had the thought that in fact, textual programming (as a paradigm) is now the sole domain of AIs. I've heard so many coders say they have not coded a line of code in months/years, AI is doing all the coding for them. Hence, I think, we humans (those of us here who are humans) should step up to the next domain for coding: 2D and then 3D and explore improved abstractions for the concepts we try to represent in a textual manner (even these concepts are abstractions of the real world).
I did not say that. I said that we do not have good representations of those concepts.
The thing is that programming is rarely the end goal of things. It's merely the help. There's often an actual task or process that existed before we got to the programming itself. The goal is to automate some subset or all of it using computers. And programming is about constructing a model of the task/process, encode the state and the rules that govern the evolution of the state, and then interfacing with the real world to get data (sensors), manipulate them according, and then produce actions (with actuators).
Text is universal because it's symbolic (and with it come abstraction which is recursive in that form). And we managed to get a good representation of time (which is also a key part of the task/process) with vertical alignment. And we can implement extra dimensions that matters to comprehension with modularization and whats not. The cons are that symbol manipulation can be abstract and requires a good deal of training to transfer that knowledge.
The issue with most 2D representation is that they're not flexible enough. Symbols are recursive. Meaning to encode more things, you just needs to repeat symbols. And then you can use symbols to encode group of symbols. Like we have 1 and 0 as the base of everything, then we encoded the alphabet using ASCII, then we encoded programming concepts using tokens (group of letters and other special symbols) and grammar, then we goes one to define higher other of things like variable definition, functions, types annotation, which just more syntax and grammar rules.
> Besides move up in dimensions is a continuation of a natural progression in computing: cogs, switchs, punch cards, keyboards, mouse, touch .... VR, AR
That's an evolution in interfacing. And the one thing about interfacing is that it has to adapt to the physical reality of human interactions. Which is often not great for the actual reality of the tasks/process that needs to happen. In the physical world, it's often physical laws and causal order that does all the work, we are only the perturbation force. We move the steering wheel and the car turns, but that's mostly due to mechanical laws than the actual act of turning the wheel.
With programming, the task is often to recreate all the laws that govern the evolution of a process and then offers an interface for controlling that evolution. 2D and 3D diagrams are very bad at describing ALL the relations between the different components of a system. Let's take the "F = ma" equation: How would you represent the force, mass, and acceleration concepts in order to create like a very simple physic engine? We can create a special interface for that, but that interface would be likely bad for creating a bookkeeping software or a text editors.
Symbolic representation is universal because there's only two things that matters, the symbols and the rules of notation. And the genius of the Turing Machine (and equivalent) is that at the foundation of things, you do not need a lot of symbols and a lot of rules. You can go from one system to the next by creating a translation mechanism. But if you leave the symbolic world, you lose universality for power. That's happen with GUI software and other 2D and 3D representation of things. Which is why you see that professional software have a scripting layer or a plugin mechanism.
I think at the core of our arguments lies this. Turing representation, i.e., the Turing Machine is the fundamental computing engine that can represent any problem and provide a solution to that problem, in whatever n-space dimension the problem finds itself in. It is the binary representation of any problem space.
Yet no one uses it to do the real world problem solving. Why? Because its symbolism is to verbose for even the simplest solution.
Herein lies what I have come to realise: its verbosity that a 2D representation can reduce at the expensive of flexibility. As the textual representation is a trade-off, so is the visual one.
Text happens to be extremely flexible for problem representation and solution but it is less verbose than a Turing Machine and based on the self principles, so we can be assured of its universality. Yet textual representation can still become very verbose and hard to understand (hence version control and diff'ing tools).
A 2D solution can be even less verbose at the price of flexibility - a 2D representation is more bound to the problem space being addressed.
I think the win comes when a recognition arises that points to a visual representation as being less verbose and therefore more understandable because the problem space is confined and well understood that there is no longer a need for texts flexibility. This being akin to a DSL definition for the problem space.
The word "sun" is less verbose than your drawing, but more clear(I would not have recognized it as a sun) and has all the information implied.
((1 2 3)(4 5 6)(6 7 8))
Of course, if you're sure that the entire rest of the industry is just lazy, then by all means feel free to join the set of people who have spent vast amounts of time trying to prove out these obviously better ideas. You've got AI now, I'm sure that will make it easier to prove it out. I imagine it can bang together a visual programming language nowadays in a few prompts whereas earlier generations actually had to rather laboriously put one together. You can get right to the part where you actually get to play with the ideas and try to make it fix programming in probably 1% of the time as the efforts that have periodically made HN's front page.
My standard challenge[1] is to make a visual quicksort algorithm that people will generally agree is nicer to read than a textual representation, where the textual representation may include comments (though so can the visual replacement). It's chosen to still not be fundamentally impossible, but to get past the sorts of "look, I can interate on a list and increment each value" demos they tend to come with.
Visual programming, another ever-recurring supposed replacement for text that is just obviously so much better and programmers are just stupid for sticking to plain and boring text, has a similar problem, where in return for whatever visual organization it makes possible (but does not force, since you can still make stupid visual programs) it is countered by the annoyance of suddenly having so many more constraints on the program, like, having to worry about layout, and how the lines cross each other, and whether some part of the program is literally overlapping another, and how all these concerns tend to grow as O(n^2) on the number of elements if not confined somehow. This one has led to some modest success in certain limited environments, but as a replacement for textual programming it fundamentally flounders on the fact that its additional constraints creates problems significantly faster than it can solve problems.
>What does an algorithm look like?
and aside from traditional flowcharts, this isn't much agreement on an answer.
For visual programming, it tends to devolve to spaghetti:
Notably the mental model of the hardware is different for each case. Other species of languages like Lisps and Golang and SIMD fascinate me because computer hardware fascinates me.
I like the toy 2D textual experiment the author has given us, and I agree with you it would not scale. Since the author calls their toy 'goofy' they probably agree. But I would contest (edit: your point that we are not stuck in our ways.) We are by and large seriously stuck in our ways. Even if a perfect visual/spatial programming language dropped into our laps today, I posit it would take over 50 years to meaningfully supplant C et al..
Hypertools[0] used to display a 'standard program' mapped to '4d' esolang piet might be interesting. A window of 'text breakout of piet segment' relevant to 'standard programming line' would be sorta like mapping set of assembler to corresponding source code line.
piet whylines : https://www.cs.cmu.edu/~NatProg/whyline.html
[0] : hypertools : https://news.ycombinator.com/item?id=31816024
I had difficulty reading large blocks of text when younger partly dyslexia I found out about much later, but mostly just overwhelming to look at. Then I learnt how to think of code as checkpoints, paths and basically a 2d map in a sense.
It helped me get used to huge walls of text. Maybe since we now have AI we can do some fun stuff in this direction since human ergonomics for writing code matter less than making code fun to read so that we can read and validate massive AI code.
Would be a fun experiment now that I think about it, I guess I found what I am doing tomorrow.
Nitpick - the andFlip function is wasting a whole dimension with unnecessary lines, and also modifying its input argument unnecessarily (yuck!), and it takes 3 arguments as an array for no good reason.
Better: andFlip = lambda a,b,c : not c if (a and b) else c
Except this is really just a normal 2 argument ternary, it does nothing interesting with a or b individually, it treats them as a unit, so I’d argue this shouldn’t even be a function.
It is used to compose songs. You can see the demo in the linked page.
I've got my 3D glyph rendering system at glyph3d.dev, and use cases like this help accelerate layout and relationship development immensely. I greatly appreciate your sharing this!
Would you happen to have a trove of similar research or papers I could sponge off of you? Or maybe even a few minutes of your time in some chats / emails to discuss how these tools can be brought to a modern runtime?
(vs. more traditioinal abstraction/logic level planes & normal to plane concepts: apl - hide it all in the language; spreadsheet - hide nothing/any information at any abstraction level is displayable[2][3][4][5]6])
--------------------------------------------------------------------------------
[1] Unicode bidirectinal algorithm : https://www.unicode.org/reports/tr9/
[2] spreadsheet : http://en.wikipedia.org/wiki/Spreadsheet
[3] sc-im - an ncuses spreadsheet program for the terminal: https://news.ycombinator.com/item?id=24318367
[4] Excel as a Turing-complete functional programming environment : https://eusprig.org/wp-content/uploads/2309.00115.pdf
[5] lambda the ultimate excell worksheet function : https://www.microsoft.com/en-us/research/blog/lambda-the-ult...
[6] escher circuits : https://esoteric.codes/blog/escher-circuits-using-vision-to-...
I just started note taking by hand and I find it way more enjoyable than typing. So maybe it’s a medium to force myself to code like in the old days.
I was looking for something like spatial computing and all I could find were esolangs, UML, and maybe punchcards? This gives me an idea on how this could be useful.
One can use intensional calculus concepts to model non-lambda languages[3]. A live 'CoC'[6] / rocq[7] layer to hightlight/note 'correctness' issues (auotmated suggestion of proof of correctness/unresolved ANTLER freevars take on ast editors 'valid' language statement(s)/language block(s))
ast editor with gui nodes (panograph[1]) / code block with "user shape" construction using 3DILG[2]?
3d bar codes could be taken as a 'multi-statement' token.
Piet[4][5] programming language might be more useful than 3d barcode as langauge token though (condensed spreadsheet).
-----------------------------------------------------
[0] : https://www.media.mit.edu/projects/2d-an-exploration-of-draw...
[1] : https://github.com/jeprinz/pantograph/blob/main/README.md
[2] : https://1zb.github.io/3DILG/
[3] : [video] Beyond Lambda-Calculus: Intensional Computation calculus : https://news.ycombinator.com/item?id=49067361
[4] : piet examples : https://www.dangermouse.net/esoteric/piet/samples.html
[5] : esolang Piet : https://esolangs.org/wiki/Piet
[6] : calculus of constructions : https://en.wikipedia.org/wiki/Calculus_of_constructions
[7] : rocq : https://en.wikipedia.org/wiki/Rocq
Would a better example be Python's inline if else construction?
x = 3 if thing else 4What I was looking for was actually "Piet":
The great gift of language is that it allows us to talk about arbitrarily complex things using a stream-like medium. Human speech is fundamentally 1D. The tree like structure of the concepts in language then facilitates the exponential growth in complexity.
Or maybe centuries of human progress just went completely in the wrong direction? Unlikely.
Nothing wrong with being silly, of course.
Or, you stick to oldschool old people. Keep things simple.
I think this is because I reason about programs being in some state ~= being positioned at a certain function with some context I carry around. I'm sure it's not the only way to think about it, but it's a reasonable abstraction over what computers do. It's how we oriented our debuggers and early IDEs on as well, and carried this over to modern tools.
Definitely something I'll be thinking about during the next few days.
As a 2D representation, I've been using flow based programming (FBP)[1] and it's incarnation in Node-RED[2]. The trick here is to understand how to use FBP to create more generalistic programs, i.e., to prove it's Turing Complete.
I think it would be a mistake to just start out with a 2D approach without also taking all the learnings we have made in 1D, i.e. TDD, OOP, KISS, DRY, Functional, Message Passing, Processes etc, and porting those to a 2D representation.
[1]: https://jpaulm.github.io/fbp/
[2]: https://nodered.org/