Maybe visual programming is the answer, maybe not

- Large programs / graphs? I can browse maps and photos of the entire planet from my web browser. ZUIs are well understood now. Also, computers got really big and fast. My graphics card has 8 GB (!) of RAM. Find a VPL that was designed for the era where graphics are cheap.

- Automatic layout is hard? Yes, an optimal solution to graph layout is NP-complete, but so is register allocation, and my compiler still works (and that isn't even its bottleneck). There's plenty of cheap approximations that are 99% as good.

- No formal spec? They sound desperate to come up with excuses why something can't work. How many (textual) programming languages had a formal spec before they got popular? I tried to look up when Ruby got a formal BNF grammar, for comparison, and it seems the answer in 2020 is still "nope, whatever parse.y does (13,000+ lines of C), that's Ruby".

- Difficulty to create editors? This is a legitimate concern, as it's true "there are no general purpose Visual Language editors", but that's an economic issue, not a technical one. Plain text is one of the very few formats where we have general-purpose editors. In most areas of computing, profitability has taken over, and companies refuse to cooperate. There's no one word processing or spreadsheet or database format, either.

- Lack of evidence? Again, they're holding VPLs to a higher standard. When a language like Ruby appeared, nobody asked for proof. Simply creating something that some programmers liked was sufficient.

- Poor representation? That's an issue with all kinds of programs! If I had a nickel for every horribly written program I've had to deal with in my life...

- Lack of portability? This is another symptom of the editor problem, above. Make a standard file format, and actually figure out how to get people to use it, and there's no problem.

These issues are from an article written in the 1980's. The same day as its publication, Apple released the Macintosh IIci, with a 40 or 80 MB hard disk, 1 or 4 MB of RAM, and a 640x480 pixel display. (For all that computing power, the starting price was $6269 -- only slightly less than the $6615 starting price for a Honda Civic that year.) Clearly, visual programming was a non-starter in that day and age. So were 90% of the other things I do on my computer every day. That doesn't mean they're bad ideas.

If you'd have asked me to write down a list of issues with C++ in 1989, I'd have a list that was similarly useless in 2020.

For example, consider all the things that have a physical order and position in a file that is actually irrelevant. Why can't methods (for instance) get displayed as convenient, without worrying about where they are in the file?

Limiting programs to fixed-width, single-font text also seems like a historical artifact. (The Xerox Alto allowed full WYSIWYG text for program code.) Having to draw diagrams with ASCII graphics just seems wrong.

Admittedly, these examples are somewhat trivial, but my point is that program structure is probably a local maximum, not the best possible solution.

Instead of a box-and-lines diagram, I think the stuff that people really want is:

- Structured editing. Valid change operations are obvious (maybe listed in a menu). No invalid states.

- Cross-cutting connections are explicitly displayed (with lines or arrows or etc) instead of being implicit.

- UI does a better job of highlighting just the relevant information and hiding everything else.

- Overall more simplicity. As in, I think some people imagine that their thousand-lines-of-code project could be fully represented as a few boxes and lines with labels. There's some unrealistic expectations tied up in this area. But simplicity is still a good goal.

Those are all good goals and there's different approaches to solve them. A lot of those asks can be solved by keeping the text document interface and adding enough cleverness on top of that (like EVE / LightTable). And there's other visual systems other than boxes-and-lines. People just need to think outside the box. (ha ha, sorry)

Hardest part of looking at existing code base is to "mentally map" the call stacks and overtime, atleast for the way my brain operates, I tend to build a 2d map in my mind. It doesn't have concrete positions for blocks (functions), but there is a sense of structure in the brain that persists. I can come back to the code base and it is evoked automatically.

We need visual tools for examining codebases just like we have tools for examining database schemas. Perhaps its not that simple though.

A related point this work makes is this association people make between “visual” and “flow diagram” coding. Not all “visual” coding is flow-based.

I don't pretend that Scratch's drag-and-drop block paradigm is a viable replacement for text-based programming, but I do think it demonstrates that there might be a text-like system we could create that would free us from keyboards, without reducing our source code to piles of spaghetti.

Also, is declarative programming typically mappable to a visual paradigm without severe compromise? Think of all the applications that produce non-executable documents; don't they factor into the visual programming discussion?

The biggest unmentioned one from a learning perspective though, is that with node-based editors, while you can still make logic errors, it’s difficult to actually make syntax errors. When you get used to it, you might not mind forgetting occasionally to match brackets, remember semicolons, use the right keywords in the right places etc, but as a beginner, all of these things can be quite frustrating and opaque, with often cryptic error messages.

With visual node-graphs though, many of these things are just non-issues, which I think helps beginners gain confidence way faster without bouncing off and giving up in frustration. You can’t even connect pins of the wrong type most of the time, so you get some pretty well communicated type safety to boot.

I’d actually love to see an evolution of a fairly advanced visual programming tool like Blueprint turn into a more general-use language, although preferably with files that are still stored as plain text (so at least play nicely with git etc) rather than the binary black boxes that are currently chewing up my LFS...

I want to build it myself, but I have neither the time nor the expertise.

Put a bag over Java’s ugly face, that kind of thing.

I don’t want to look at IntelliJ’s “staircase of doom” formatting, I want to see one parameter per line when there is more than one parameter.

I don’t want to trace down data “COME FROM”s (local variables that are only used once and have to be tracked down from whence they came).

Give me an alternate layout which directly tracks between the text and the diagram, similar to a structure layout, but showing details within functions/methods as a tree, as well. Show me clearly how values/expressions funnel or pipeline into each other, but filter out some of the noise.

And auto-display fuglyCaps names as kebab-case, while we’re at it :-). Cuz, science.

The inherent problem is the lack of primitives to build abstractions to "move the langauge toward the problem domain". These primitives can be first class functions, objects, macros, etc [1]. Visual programming environments I've seen also fail to provide any such mechanisms as this article supports.

[1] http://www.paulgraham.com/avg.html

Having used my fair share of labview i know such examples are reality. Common mistake of the model is the sentiment that because it's visual everybody can do it! Biggest wtf i experienced was when system engineers wrote half completed "reference-implementations" in labiew, which coders then had to translate to Java for production, to sort out all the edge cases, because these visual tools (and the people that use them) are very bad at expressing edge cases. They might as well have given us graphs drawn on paper because the references were so far from reality you could never run them.

Together with that, when the only tool you have is a hammer, everything looks like a nail. Take https://blueprintsfromhell.tumblr.com/image/162487395171 as an example, it's actually one of the simpler graphs from that page. But if you take a look at it you see it's just doing standard function calls in a sequence, like trunctate(), toText() setText(), what's then the point of making it visual?! For simple signal control loops with only mathematical operations the labview model works really well, but when you start doing things like subString, groupBy or indexOf and don't realize you should change domain, that's when you get the worst spaghetti.

The benefit of a visual representation is that there are different ways to visualize things and you can choose which one fits the problem best. You have sequence, dependency, state, class, call-graphs, signals, lexical parsers, tables, charts, etc. Most visual environments only implement one of these and very soon you are using the wrong tool for the job.

I recently wrote software that allows you to transform data step-by-step visually (https://www.easydatatransform.com). I think the visual, flow-based approach here is a real win, especially for non-programmers.

However I wrote the software in a primarily text-based programming environment (C++ and QtCreator). I think it would have been a nightmare to do in a purely visual programming environment.

Horses for courses.

This is bullshit. A lot of musicians & visual artists are super successful with tools like Max/MSP, PureData, vvvv, or the ton of graph-based shader editors out there. Max/MSP is taught in a lot of place to music students, e.g. in conservatories. I promise that by the time you get them to understand for() and if() they've already made some music in Max, and the better ones have already started toying with sensors and controllers. Disclaimer: I worked 3.something years in a company where the owner routinely shipped software entirely built in Max.

People without any programming abilities are able to use Blender's node editor to make super nice visuals : https://www.youtube.com/watch?v=JhLVzcCl1ug and at the other end of the spectrum, code that runs in airplanes is routinely designed using SCADE (https://www.ansys.com/products/embedded-software/ansys-scade... - see e.g. these slides which talk about its use in the A380: http://www.artist-embedded.org/docs/Events/2008/Autrans/SLID...) and other tools based in the synchronous reactive model.

I have thought of doing somethings like that in the browser with a WYSIWYG editor (document.designMode='on') and web components. The DOM is like the AST of the program and is fed into a runtime. Non-expression tags are basically just comments.

Not for regular programming, but for a type of scientific notebook interface to do exploratory programming. Anyway one of this days.

Personally my favourite VPL was Automator, which had the control flow primitives equivalent to the bash operators | and ;. This made layout and control flow trivial at the expense of expressiveness. However, as the goal of the thing was basically “terminal for the rest of us”, arguably those two operators are what most people end up using at the command line anyway.

Of course Automator failed because of a number of other issues (lack of abstraction and bad package management I think are at the top of the list). But I think tools like that have a good amount of potential.

My general feeling is that a visual programming language excels as domain specific languages where the domain is quite visual.

Ive seen two problems with visual programming languages which are quite severe in my opinion.

1. Merging sucks. Merging visuals are not straight forwards. I’ve seen companies do trunk based development before it is was hip just to get around it.

2. As with all high level languages, without an understanding about the overhead of different constructs, it is easy to create very inefficient solutions. On cloud systems it may be easy to throw more processing power at the problem. On embedded systems, less so.

Solved by the ability to pan and zoom. Node graph tools in professional visual effects environments can grow to >20,000 nodes in a document, especially given that nodes can be nested in groups or cross-included from other documents.

> Need for automatic layout. [...] For example, generating an optimal layout of graphs and trees is NP-Complete [95].

Doesn't need to be absolutely optimal to make the problem go away; making something that helps the user enough that they don't have to think about it is completely tractable.

Many professional node graph tools don't do any automatic layout at all and let the user handle it; which while less than perfect, doesn't stop the tools from being incredibly powerful.

> Lack of formal specification. Currently, there is no formal way to describe a Visual Language.

First, a formal specification is not necessary in order to build something useful.

Second, if you really need a formal specification, then write/invent one. There is nothing about visual systems that makes them less possible to specify than any other software system.

> Tremendous difficulty in building editors and environments. [...] These editors are hard to create [...] the language designer must create a system for display [...] which usually requires low-level graphics programming.

So... do that stuff? What about that work is more difficult than the other parts of writing a compiler; or any other piece of complex software? Maybe this was more of a big deal in 1989, but basically all software today is based on a visual UI, so the complaint that the need for a UI makes visual languages intractable is... pretty ridiculous now. None of this would be harder than, say, writing a simple video game.

> Lack of evidence of their worth. There are not many Visual Languages that would be generally agreed are "successful"

"Nothing good exists yet, so nothing good is possible" doesn't hold water for me.

> Metrics might include learning time, execution speed, retention, etc.

From my experience in visual effects, non-coders can pick up a node graph tool very quickly and get very creative/inventive with nodes— but the moment they have to fall back to writing a script, they struggle or don't even try. It seems abundantly clear to me that node graphs are far easier to learn (but can be just as powerful) as writing with a machine grammar.

I also did an image processing development project in a node graph tool that took about 4 days of experimentation to get to patentable IP. If I had to do the same project in Python or C++, it would have taken weeks or months, if I was even able to solve the problem at all without the fluidity and responsiveness of the node based editor. So even for "serious developers", a good node-based editor can multiply speed by a factor of 10.

> Poor representations. Many visual representations are simply not very good.

Can't argue there— but the solution is to make one that is good. :)

> Lack of Portability of Programs. [...] Graphical languages require special software to view and edit

...Like basically all other file formats in the universe. :)

It is not 1989 anymore, and it's time to fix this stuff.

On the other hand, there's no fundamental reason that visual editors shouldn't be able to load and save source code; it's just a different way of thinking about program structure and user interface. There's already a lot of tools out there that generate dependency graphs within or alongside an IDE; the main shortcoming i see is a lack of nesting. There are few instances in which i want to see the entire structure of a program all at once, any more than I want to read all the source code in a single giant text buffer

One might equally ask if textual languages are so great, why are there so many of them? Couldn't everything be written in a nice capable high-level language like C? And of course it could, but lots of people don't like C's minimalism and lack of guardrails. It seems to me that most new general-purpose languages originate in a desire to have the computer do some of the tedious housekeeping (like casting variables) and frustration with awkward syntax.

I'm old, so when I was learning to write code it was just flat text files and every build necessitated a round of swearing over forgotten semicolons, mismatched brackets, or incorrectly spelled variables. Syntax highlighting, code folding, autocompletion, and bracket matchings which do so much to make development in a modern IDE pleasant and productive were once viewed as undesirable crutches that would lead to poor practices, lack of portability, and inaccurate second-guessing of developers' intention by the IDE, subjecting hapless coders to the tyrannical will of the interface illuminati.

I suspect that behind the aversion to visual programming lies a fear of standardization and automation that will result in widespread developer redundancy as it becomes easier for domain experts to build out their tools without code specialists, somewhat parallel to the development of electronics from hobby clubs where people helped each other with soldering irons, multimeters, and resistor identification charts to today's electronic factories populated only by pick-and-place machines assembling components too small for adult hands to manipulate at dizzying speeds.

The more automation and and ML that finds its way into IDEs, the more standardized the underlying libraries are likely to become. To the extent that programmers are freed from syntax production and housekeeping duties and processes can more easily be read (from visual graphs/charts) with domain knowledge alone, the more likely it is that the most common functions become wholly standardized, in the same way that chips are. 555 timer ICs have been around since 1971 and the most popular derivatives are dual and quad versions. Nobody is writing new and improved 555 timers and there's no real demand for them.

http://www.cs.cmu.edu/~bam/

>Brad A. Myers is a Professor in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University. He was chosen to receive the ACM SIGCHI Lifetime Achievement Award in Research in 2017, for outstanding fundamental and influential research contributions to the study of human-computer interaction. He is an IEEE Fellow, ACM Fellow, member of the CHI Academy, and winner of 15 Best Paper type awards and 5 Most Influential Paper Awards. He is the author or editor of over 500 publications, including the books "Creating User Interfaces by Demonstration" and "Languages for Developing User Interfaces," and he has been on the editorial board of six journals. He has been a consultant on user interface design and implementation to over 90 companies, and regularly teaches courses on user interface design and software. Myers received a PhD in computer science at the University of Toronto where he developed the Peridot user interface tool. He received the MS and BSc degrees from the Massachusetts Institute of Technology during which time he was a research intern at Xerox PARC. From 1980 until 1983, he worked at PERQ Systems Corporation. His research interests include user interfaces, programming environments, programming language design, end-user software engineering (EUSE), API usability, developer experience (DevX or DX), interaction techniques, programming by example, mobile computing, and visual programming. He belongs to ACM, SIGCHI, IEEE, and the IEEE Computer Society.

For example, he wrote this one year later in 1990:

Creating User Interfaces Using Programming by Example, Visual Programming, and Constraints. Brad A Myers. ACM Transactions on Programming Languages and Systems, Vol 12, No. 2, April 1990.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.460...

I really enjoyed this 1999 paper “A Taxonomy of Simulation Software: A work in progress” from Learning Technology Review by Kurt Schmucker at Apple. It covered many of my favorite visual programming languages.

http://donhopkins.com/home/documents/taxonomy.pdf

And it's also worth reading the more modern and comprehensive "Gadget Background Survey" that Chaim Gingold did at HARC, which includes Alan Kay's favorites, Rockey’s Boots and Robot Odyssey, and Chaim's amazing SimCity Reverse Diagrams and lots of great stuff I’d never seen before:

http://chaim.io/download/Gingold%20(2017)%20Gadget%20(1)%20S...

I've also been greatly inspired by the systems described in the classic books “Visual Programming” by Nan C Shu, and “Watch What I Do: Programming by Demonstration” edited by Alan Cypher.

Visual Programming. Nan C Shu:

https://archive.org/details/visualprogrammin00shu_2pf

Watch What I Do: Programming by Demonstration. Allen Cypher,. Daniel Conrad Halbert.

https://archive.org/details/watchwhatido00alle

In short, textual programming is just a subset of visual. People think of the dichotomy as a duality of text vs. visual, it is not.

Text just has features people tend to like: A Fixed line based structure. Isomorphism with the english vocabulary... Who says other visual programs can't have this either?

The issue with a lot of these "visual programs" is that they also represent a subset of Visual Programming similar to how text is just a tiny subset of visual programming. What you are in fact seeing with most "visual programs" is mostly programs represented as graphs.

The real dichotomy people are discussing is not text vs. visual but text vs. graph. Whenever I see something like labView or something labeled as a "visual program" it's always a program represented with a series of nodes and lines, aka a graph.

Hopefully by illustrating the reality of what's going on, people can see beyond representing programs as just text or just graphs.

There are other ways to represent modules and processes.