Ask HN: Why does visual programming suck? | Hacker News Reader

Ask HN: Why does visual programming suck?

212 pointsdvdhsu315 comments

Almost every visual programming tool I've seen on HN looks so cool when I see the gif, but when I start using it, it's invariably a frustrating experience. I think it might be because customizing the last 20% is difficult visually, and you so you have to drop down to code. But once you're in visual programming land, changing abstraction layers is difficult.

When I asked Alan Kay this question, his first response was "well there are various types of visual programming, some of them have taken off". The type of visual programming I'm thinking about here is a WYSIWYG drag and drop interface for building UIs. Imagine having prebuilt views (tables, buttons, dropdowns, etc.) and being able to drag them and lay them out how you want. Then you interact with their API: this table pulls in data from this SQL statement; this button executes this HTTP query; this dropdown publishes this value, which this SQL statement uses via string interpolation. These "widgets" publish their data, which can be used by other widgets (w1.value), and inside SQL queries. The tables can pull in data from SQL queries. You can use string interpolation inside SQL queries, so dropdowns can influence the query being run. So selecting a drop down value would cause the query to change, which would automatically propagate changes to the table.

View on HN

"Text is the most socially useful communication technology. It works well in 1:1, 1:N, and M:N modes. It can be indexed and searched efficiently, even by hand. It can be translated. It can be produced and consumed at variable speeds. It is asynchronous. It can be compared, diffed, clustered, corrected, summarized and filtered algorithmically. It permits multiparty editing. It permits branching conversations, lurking, annotation, quoting, reviewing, summarizing, structured responses, exegesis, even fan fic. The breadth, scale and depth of ways people use text is unmatched by anything. There is no equivalent in any other communication technology for the social, communicative, cognitive and reflective complexity of a library full of books or an internet full of postings. Nothing else comes close." http://graydon2.dreamwidth.org/193447.html

I agree with this statement, wholeheartedly.

I do, however, think programming languages could benefit from more visual augmentation. A program, over time, takes on more complexity than a programmer can hold in his mind, and eventually, the program's design will elude him. A disconnect appears between the text he types in the moment and the original design of the program. The sum total of the program's code no longer resembles the problems at hand.

Here, I think, visual tools are underutilized. Not as code crafting tools, but as tools for reflection. Reflection on dependency trees, deviation in style, orphaned code, the ability to visualize how and where data might flow in unexpected ways, etc. Webpack's stats.json comes to mind for dependency tracing. Statically typed languages obviously have a leg up here.

This is, I think, the idea behind IDEs

Yet which IDEs aren't simply really nice text editors with some refactoring help built in?

Making these visualizations central to the IDE would be exciting.

> Yet which IDEs aren't simply really nice text editors with some refactoring help built in?

All the IDEs I use are more like:

    Yet which IDEs aren't simply really _horrible_ text editors with some refactoring help built in?

Yeah. I feel like the depth of OP's writeup blurs the lines between working with an application framework, API, and IDEs. I was under the impression that visual programming is different from IDEs or frameworks...

If you ever want to horrify yourself, google LabView, a truly horrible piece of software.

I actually liked it in a way.

Not my favourite but absolutely not horrifying to a student.

Now that I think about it though I wonder what huge merges look like.

I have a dream that one day we'll collectively realize that "buying programmers the best tools" means the most powerful, and not the shiniest. Then we'll all be running ultra-wide monitors with room for many different visual aides on the one screen, backed by the CPU horsepower (overclocked and liquid cooled, if necessary) to make it all fast. And, hopefully, we'll see what an IDE can really be.

As long as we're speculating: I kind of dream that maybe we'll see programming environments that take advantage of VR.

Humans are really good at remembering spaces. ("Describe for me your childhood bedroom." or "What did your third grade teacher look like?")

There's already the idea of "memory palaces" [1] suggesting you can take advantage of spatial memory for other purposes.

I wonder, what would it be like to learn or search a codebase by walking through it and looking around?

[1] https://en.wikipedia.org/wiki/Method_of_loci

> Humans are really good at remembering spaces. ("Describe for me your childhood bedroom." or "What did your third grade teacher look like?")

Some humans. I honestly can't answer you those two questions. I'm much better in remembering what I just read than what space I just saw, and I absolutely suck at remembering (not to mention describing) faces. If you mugged me I probably wouldn't be able to describe you to police beyond the basics like gender and height (I probably won't be remembering what you were wearing).

I say this because being good at remembering spaces is like "everyone sketch a picture when trying to convey an idea" elsewhere in the thread - both are examples of what some people do, and it may be surprising to discover that not all humans are good at it.

I have face blindness. You may as well, or at least some degree of aphantasia.

No reason for concern, but there is supposedly an incredibly low occurrence of either.

My dream is the opposite. Constrain programmers to hardware that's 2 generations old. That will force us to make our programs run fast in this environment which is much closer to what actual users will use to running it.

I do understand the implication that lowering the need for always faster hardware for end users will slow down the pace of development of new hardware. But with the end of Moore's law upon us, that's inevitable anyway.

Our computers are plenty fast today, beyond what we could imagine when each of us started programming. Let's cut down on abstractions. Let's profile, optimize.

I wrote some of my fastest code on a netbook. It's also the gnarliest code I ever wrote and I do not wish to touch it again.

You can have it both ways: power the IDE with a futuristic power hungry beefy machine, and run the programs it produces on a throttled sandbox that match the slower target machines.

I fear cheating would be too easy, though —just lift the restrictions.

After being stuck with a really, really bad laptop for the past 4 years, I can't disagree more whole-heartedly. I wouldn't wish that experience on my worst enemy, heh.

Though I wish more time was spent on some tools to make them more memory efficient(I'm looking at you, scala and friends), that's just not how it is today. I just bought a new laptop with 32 gigs of ram, and hopefully that is future-proof for the next few years.

What was the laptop? And what were the 3 things that you found most annoying?

Not OP, but I had a similar experience. Low end development machines suck because they put the developer in a worse position than the end-user, not an equal one. Modern dev tools, plus common side-utilities like slack/spotify/skype, and accessories like Insomnia, tend to use a lot of memory and cpu. It makes the development experience terrible, and that affects your output. Developing and designing is going to be ~90% of your time spent on a dev box, actually running the dang thing won't take long at all.

A better solution would be for testing, particularly UAT, to be done inside resource-constrained VM's, instead.

A bit late reply.. but the laptop was just the cheapest laptop I could buy before starting Uni, roughly 4 years ago. I just got my new laptop a couple of days ago, and the difference is incredible.

The other guy that replied to your comment is basically right. My old laptop had a measly 4GB of RAM, and opening multiple tabs in Chrome for documentation, running an editor, some development tools like REPLs etc, and I'd basically be out of RAM. I would literally listen to music on my phone because spotify consumed too much of my precious RAM, and I had to kill off tabs that I hadn't used for a while like it was Survivor, in order to keep my laptop from start swapping(which means everything grinds to a halt).

I even run a very minimalistic window manager and don't use IDEs like IntelliJ or anything.

I agree because I hate the pain of debugging. I disagree because if something takes that away, then where is the skill in my work?

"This is so bad, it's not even wrong."

https://en.wikipedia.org/wiki/Not_even_wrong

Would you mind enlightening me as to the fallacy you perceive in that post? It seems well-formed to me.

I think visual programming is one of those ideas that intuitively seems like a good idea but never is. Thus, it never really dies, it just gets picked up as a kind of rite-of-passage by new generations of developers. Managers also tend to like the concept as it would allow them to bypass the pesky code monkeys and just feed their drawings to the computer.

For my master's thesis, some 15 years ago, I programmed a system for visually composing and executing Java programs. The UI looked a bit like UML, and it worked nicely for anything resembling "Hello world" complexity (anything more complex literally became hairy to look at). Since then I've twice found myself on projects where we had to work with visual programming tools (caused by skilled salesmen and inept managers). In both cases the only sensible action was to pound the "eject" button until we were catapulted away from the monster. All in all I consider myself vaccinated at this point.

As for my reply to the question of the OP, I'm not sure that it differs much from everyone else. The main reason that it will never catch on, IMHO, is that the text editor is far better for managing complexity. It's easy to search for stuff, and it's easy to move or copy it. Also, every serious text based programming languages offers plenty of ways to hide complexity away, allowing you to focus your thinking at the desired level of abstraction. Visual programming tools -- or at least those that I have met -- instead force you to deal with stuff that doesn't matter (such as aligning boxes and arrows), and views very quickly become cluttered when you start connecting things.

That being said, I think visual tools can be a good fit for tasks whose output is UI. There are applications and frameworks that allow you to drag'n'drop to compose the UI, but the code is still written in a text editor. This can work reasonably well.

My horse carriage is very useful. The horse just feeds itself, if you leave him in a grass field. It rarely gets sick and last many years. The wheels are easily replaced from the store.

The other day I saw a crazy guy with a mechanical contraption that, according to the guy, moved by burning wood! The guy was demoing it at the town. He filled it with wood, and then had to start the fire. The crazy machine got so hot that nobody could get near it. The thing barely moved a few meters and then Kabooooom!! it exploded in a black cloud. What sane person may ever believe that one of these contraptions could be useful?

Am I right? Are you right? we don't know. Research is the only way to know.

Except that there has been a lot of research.

The first visual programming language I can find is from 1966. They have always been tiny, niche things over a period where we've had waves of text-based languages rise and fall.

One of the striking things for me when I talk with visual programming enthusiasts is that they very rarely have looked at the decades of research. And if you try to talk with them about the history, they seem actively averse.

There's a joke that some people have ten years of experience, and some people just have one year of experience ten times over. My experience has been that visual programming is more like the latter, but for 50 years.

There are decades of research. It's not unreasonable to adjust expectations based on that.

Honestly, I feel part of the problem is that this seems to attract people who think that text is bad and more visual = better, while the truth probably is that you need to be really, really careful to arrive at something that doesn't explode in complexity when confronted with all the little details of ordinary programs.

Some good points. But I think your view has become too negative, perhaps due to your experiences. Or at least I see more hope for visual programming. :)

> I think visual programming is one of those ideas that intuitively seems like a good idea but never is. Thus, it never really dies, it just gets picked up as a kind of rite-of-passage by new generations of developers.

Perhaps the intutition is right, but the problem is too general and thus very hard to solve. And I think that, looking at the general trend, it does seem to evolve towards something better with every new generation.

Not the first and not the last idea like that.

We're just not there yet.

> For my master's thesis, some 15 years ago, I programmed a system for visually composing and executing Java programs.

Similar experience here [0]. :) But for me it was a year ago, so I guess I represent a newer generation.

> Since then I've twice found myself on projects where we had to work with visual programming tools

I worked with some too, mostly Unreal Engine 4. Fortunately all for education, not for work. ;)

And your experience is probably a good picture of the state of the art in this area, if it comes to complex and more general-purpose application development.

> The main reason that it will never catch on, IMHO

I'm still a believer. But probably not in my generation either. ;)

> the text editor is far better for managing complexity. It's easy to search for stuff, and it's easy to move or copy it.

True. Compared to a visual editor of today.

For this and other reasons, text coding can't and will not be replaced with visual coding. Visual programming, I think, should be thought of as an extension of text programming.

So I think that visual editors will only start becoming successful and more wide-spread among regular developers when they stop being visual-only. Or, in other words, when they stop eschewing all the good parts of text-based environments. Visual programming should complement text-based programming. Not replace it.

> Visual programming tools -- or at least those that I have met -- instead force you to deal with stuff that doesn't matter (such as aligning boxes and arrows), and views very quickly become cluttered when you start connecting things.

It's almost like coding in text without the editor helping you with indentation and structuring of text. You have to arrange boxes and connections yourself.

This is one thing that illustrates that some visual languages are really a few steps back in evolution compared to text languages.

But there are visual environments (e.g. Scratch and the like), which do help you with structure.

Anyway, I'm tired and it's very late, so I'll end here. Cheers!

[0] (copy-pasted from my previous comment):

(a screenshot from my project) http://djedr.github.io/gfx/masters-thesis/ide-prototype.png

(link to my thesis) http://djedr.github.io/masters_thesis.pdf

Thanks for your feedback. I agree with your point that past failures, however many, does not guarantee future failure. Aviation is a good example from history.

And we should definitely keep researching stuff, even things which seem like a long shot. We'll almost always learn something useful in the process.

The future will tell I guess.. :)

As a believer myself, I think the problem is that visual programming suffers the same problem known as the curse of Artificial Intelligence:

"As soon as a problem in AI is solved, it is no longer considered AI because we know how it works." [1]

Similarly, as soon as a successful visual interactive feature (be it syntax highlighting, trace inspectors for step-by-step debugging, "intellisense" code completion...) gets adopted by IDEs and become mainstream, it is no longer considered "visual" but an integral and inevitable part of classic "textual programming".

[1] http://www.artificial-intelligence.com/comic/7

Except there is a successful visual programming tool and it probably runs a large chunk of software in your car. It's called Matlab Simulink:

https://mathworks.com/help/simulink/examples/anti-windup-con...

Obvious lectures from this:

1) Don't be silly and copy a text based language into visual blocks. Nobody wants to drag&drop the components of a while loop or click a box to enter a variable name. If you have variable names, you probably failed this one already.

2) It's about the data, stupid. There is very very little explicit control flow in Simulink diagrams. Instead it's all about the flow of data.

3) Seriously, nobody wants to click together while loops. Simulink works because it has a very large and very powerful library of blocks that you need just a few of at a time to build very complex functionality.

That's actually very interesting. Never heard of that before.

A few remarks off the the top off my head:

1) Visually, it reminds me of electrical diagrams, perhaps taking it one or two steps up the ladder of abstraction. I can see how it might be an appealing notation for someone trained in electrical engineering, which may explain its usage in cars. It's really interesting how different "cultures" find different solutions to similar problems.

2) Still, and this may be unfair since I only glanced over it for a few minutes, it strikes me as more of a niche thing than as something that I'd describe as a general purpose language. It seems fixed at a certain level of abstraction. Probably just the right level of abstraction for configuring your car firmware, but impractical for authoring a complex piece of desktop software.

Maybe the future will prove me wrong :)

Perhaps creating niche visual DSLs is a more viable approach to developing visual programming environments than to focus on creating a general purpose programming environment.

It'd be interesting to see if there are any existing solutions in other domains.

I have a background as a programmer and recently started using Matlab/simulink. It is good for some things like controllers etc. Can't really say I like it for things I would usually use when programming, like loops and if statements(I know simulink has these but they make no sense).

I also believe simulink has been used mostly for simulation therefore it's name and not programming. However now you can generate code from your model.

I've worked on this and also did cross-compilers for it. It's industry standard. It's not only used in cars, it's used all over industry control automation, airplanes, power plants, ... It's reliable and creates almost bug free code. It allows real-time control loops. It creates highly optimized C code (I wrote a C++ backend). You wouldn't want to do that in C or C++ by hand. Before Simulink my prev. company used Occam, a powerful Smalltalk/Objective C like language with CSP, which ended up being succeeded by Go.

But there are also many more such successful tools being used industrially, esp. from Siemens.

The biggest problem with it are diff tools. There exists a graphical diff in the toolbox, but I haven't seen it being integrated in its workflow yet.

I'm developing driver assistance functions for vehicles and I have experience using simulink, although not anymore, fortunately. It's usage is converging slowly but steadily to zero in the company I work for, although we still use it for controllers, but mostly only legacy "code". Why is it?

- First is the most serious, debugging: this is a deal breaker alone. Once your code will go into production, and once, bugs will be found in it. How do you debug it? A typical variable in simulink looks like this: `Simulink_Sum_Node_4521` (something similar, I don't remember exactly). Yes, that is a variable in the c code generated from simulink graphs. It's index start from 0, but you have quite a lot of it - just count the operations in your normal code, and don't forget that (a+b)/c already contains 2 operations! Not sure if the original programmer was lazy to name them all, but not sure if it is possible to meaningfully name them either. If you want to get (a+b)/c^5, (a+b) might not have a meaning on it's own. So you stuck debugging a 10k+ lines long file containing variables like this.

- Reading is natural: we read from early childhood, we write from early childhood. Did you draw computational graphs as a kid? In school maybe? Yeah, me neither. When I want to tell my colleague a formula, I'm not going to draw him/her a computational graph, I'm going to write it down. Hell, you are reading right now, and not looking at a graph consisting words and arrows that represent relations! Imagine how that would look like...

- Graphs take up much more space on the screen than text. Grab a pen and draw a computational graph of a Fourier transformation! It takes up a whole screen. As a formula, it takes up a tiny fraction of it. Our state machine used to take up about 2m x 2m on the wall behind us.

- Also the other reasons already written in this thread...

Working with simulink convinced me entirely that graphical programming has no future whatsoever.

> That being said, I think visual tools can be a good fit for tasks whose output is UI. There are applications and frameworks that allow you to drag'n'drop to compose the UI, but the code is still written in a text editor. This can work reasonably well.

Even there some people claim it works better for micro scale: widgets are ok but larger scale composition (storyboards, composing dynamic views from re-usable sub-views) are better done in code.

There is a huge difference between direct manipulation of concrete concepts, and graphical manipulation of abstract code. Visual programming works much better with the former than the latter.

I think the reason is that text is already a highly optimized visual way to represent information. It started with cave paintings and evolved to what it is now.

"Please go to the supermarket and get two bottles of beer. If you see Joe, tell him we are having a party in my house at 6 tomorrow."

It took me a few seconds to write that. Imagine I had to paint it.

Exactly! I'm actually bewildered why some people think visual programming is a good idea at all.

I know a guy who does all his projects using Max MSP and loves it.

It's obvious that visual programming is much less expressive than writing code, but it's attractive to people that are afraid of code or are too lazy to learn a new language.

I know a few people who do this, too. They're almost invariably awful about it and completely convinced they're on the "right track" over people actually writing code, too..

I suspect it has to do entirely with affordances.

Programming in text, you have to learn, in advance, what the commands are that you're allowed to type, and remember them. That's a heavy up-front investment.

Visual programming generally makes it much clearer exactly what your range of options are -- the goal is for the learning curve to be far easier.

Could be an interesting feature for IDEs to start studying certain control structures, not just function calls and parameters.

Good autocompletion with intuitively named standard library comes close to this. When coding c# in visual studio I very rarely have to look at documentation, you just guess what the class or function might be called and get the options presented for you.

Zero syntax errors.

This is also one of the beauties of text programming. It allows temporary syntax errors while restructuring things.

I've used many visual tools where every block you laid out had to be properly connected, so in order to refactor it you had to make dummy blocks as input and output and all other kinds of crap. Adding or removing arguments and return values of functions/blocks is guaranteed to give you rsi from excessive mousing.

Have you tried using Drakon[0], I really like to use it to visualiase algorithms. It also supports "syntax erros".

[0]http://drakon-editor.sourceforge.net/

It's one avenue of exploration to (further) commoditize programming.

请你到超市去买两瓶酒。如果你见Joe，告诉他我们明天六点钟参加派对在我的家。

Here, that was quite fast to paint. Would have been even faster with a proper brush instead of a keyboard :p

If that was a painting then surely I would have some remote idea of what it represents. As it is, I (and many others) don't.

Unless it was modern art, of course.

Maybe he's making a joke about how Chinese characters evolved from more pictorial-like elements, and are in a sense abstractions over pictures.

Even harder with pictures, searching and, sharing, and refactoring:

"Joe has been way too busy yesterday. What did you ask him to do in the last week?"

"Yeah, that's too much. Just ask Jessica to do the emails going out next week."

So do you use text files for storage or a database? Is your company's accounting system in Word or Oracle?

Are programs text, like a novel, or is it data? And to find out, would you query Moby Dick? Not really -- at least that's not how it's meant to be consumed. You read it from beginning to end.

Would you read your program from beginning to end or would you query it? You query it, exclusively. I submit that programs are data, not text, and visual programming then becomes like data visualization. Text is holding us back.

Right, code is data. But the interface to write and modify that data is text.

Consider, how long does it take you to draw an infographic representing the percentage of the United States that has a college education? How long does it take to modify it when you realise it is a mistake

Now how long does it take you to represent/change that data with text?

Your example is strange. Changing the text and changing the data would require the same amount of time. And a graph would, in fact, instantly put that change into context.

Right, code is data.

Not now, though. Currently, code is text -- it's not data. That's the problem. And, yes, currently, drawing it instead of typing out the text is a horrible idea. We agree.

But remember, currently, we're failing. Software is a disgrace of an industry, in all honestly. It's mired in undue complexity. It's a Rube Goldberg contraption that we all, somehow, take seriously.

Imagine a better future. Imagine visualizing a complex set of rules and seeing all the places where a change would cause knock-on effects. Imagine being able to manage your code base in a way that doesn't require you to download it all into your head. Hell, imagine a decent IDE. The first step for all of that is to move it from text to data.

I think you are chasing something that does not exist.

While i submit to the idea that programs are "data", and could be represented not just like text, the expressiveness when using a visual representation is small, but the exactness is as high as normal languages.

I've seen large visual-programming programs, and they explode into complexity, and I think there is a reason for that, the expressiveness of a symbolic language using text is so much higher.

Could we increase the expressiveness even further? English have high expressiveness but is inexact.

I don't think it's possible to increase the expressiveness and keep the precision - at least without some sort of intelligent agent that can reject interpretations that makes no sense.

So - when general AI is invented, we can probably soon write programs that are very inexact, highly expressive, but still work.

Until then, we will have to continue wrestle down the computer to obey our if-statements, one by one.

Visual programming does not have to mean 100% not text.

https://joshondesign.com/2014/08/22/typopl

Yes, there are probably lots of useful interaction methods to be found in between text files and the "boxes and arrows" visual programming (circuit design) model.

You talk about the compiler having to be sufficiently smart, but I think you actually just need good IDE support, then you can store the code as text but display a nicer rendering.

E.g. the image literal could actually be Image.loadFromPath("../.../image.png"), but you don't see the code unless you want to; and if you drag and drop a file into your program, the IDE automatically inserts the corresponding code.

> Changing the text and changing the data would require the same amount of time.

Really? really?

Open up powerpoint, openoffice, or whatever you use to make infographics. Also open up vim, atom, or whatever you use to edit text.

Time yourself clicking and dragging to make a graph in one and then typing to write a number in the other.

> Imagine a decent IDE

I'm imagining something where you are able to select visual elements without either clicking and dragging or doing an O(n) walk by hitting alt-tab. But I can't imagine how you acheive that, much less efficiently specify how you connect things up and how you write automated tests.

Some of this is my failure of imagination, but its still on you to provide the sketch of a proposal if you want to convince.

How do you navigate code now? Do you walk through code line by line or do you search or use some kind of navigation that relies on the structure of code?

I agree that a concrete product is probably going to be required to convince people who aren't good at thinking outside of what already exists, but these kinds of comparisons just feel absurd. Maybe a better way to think about this sort of thing is "how can the structure of code be used to simplify and improve working with it"; the term visual programming is just too loaded in negative ways.

Edit: I don't mean to sound insulting when I talk about thinking outside of what exists. It's not always an easy thing to do. But it becomes an issue when you it causes you to let previous attempts narrow your views on what something is.

Visual programming doesn't have to be, say, exactly what's scratch is today. It doesn't even have to be close. The two keys IMO are utilizing the structure of code as well as the inherent visual processing power humans have.

This doesn't mean no text, it means expanding how we think about operating on and viewing code in a way that makes it easier (and thus more productive) to do.

Okay. I'm on board with you there. I suspect that SQL or data-model rendering would be the easiest place to start there--allowing you to auto-generate a visualisation from SQL. If that got widespread enough, then it would become a standard way of representing the data that is currently represented by text. That means you can build interactions on top of that understanding.

> Is your company's accounting system in Word or Oracle?

Accounting uses numbers, which in this context is text. If you replace a spreadsheet with a graph, your accountants will murder you.

So, to be clear, you're saying a spreadsheet is the same thing as Word or vim?

which in this context is text

Ha! Nice try. You program in a text file. You don't get to somehow call that a spreadsheet just to make your argument sound better.

My whole point is that we need to turn that free text into a spreadsheet (from which graphs can be produced) and "your accountants will thank you".

Why is it that only very simple programs can we written effectively with spreadsheets and that setups where complex arrangements of linked spreadsheets and databases are described as horrible to work with by the people that have to work with them.

Is it possible that nearly every professional is right and you are wrong and text is an effective means to represent computation.

I completely understand where you're coming from. I'm saying, "try to imagine how it should be," not, "how it is." Those experiences are horrible because current spreadsheets are for simple functions and nothing more.

Is it possible that nearly every professional is right and you are wrong and text is an effective means to represent computation?

I guess this is where we disagree. I see the software industry as a mess and every professional as a person who was given one part of a Rube Goldberg contraption to be an expert on. The contraption works, to be sure, but it boils an egg in a hundred arcane steps. Given the state of our art, we have no leg to stand on to say what is and is not an effective means to represent computation. Normally a billion professionals would lend clout to your argument, but in the case of our industry, we know that they just staff a million abstruse contraptions. It just means we lack imagination and ownership: we're professionals, yes, but on whatever junk we're handed.

It wasn't always like that. We have seen that Computer Science doesn't really advance, but what is fascinating is that maybe 90% of those pushing for it to advance are of the original, older, generation that brought us that innovation in the first place. Max Planck once said something to the effect of, "Science advances one funeral at a time." I worry that Computer Science retreats one funeral at a time. Slowly we're losing those from the time when our thinking wasn't constrained with current tools; failed efforts were taken as learning points, not barriers; and no one viewed current technology as something the gods handed down to them but as something they created (maybe the day before), so they were constantly evaluating it.

Yes, the future of software development has to be away from free text toward data, and it follows as a logical syllogism. I was about to add that that is my opinion, but it's not so much of an opinion as a conclusion. While it's absolutely fair to wonder what that means, we can't brush it away because we already have the best Rube Goldberg contraption we could ever have.

then why does everyone sketch a picture when trying to convey an idea? text is obviously useful but so is visual information. the complete dismissal of visual programming by text-based programmers is often infuriating. visual programming can provide an immediate indication of structure and dataflow, something that text-based languages struggle with.

in my mind, the best case scenario is some sort of hybrid, where visual and text representations take over on what they're best at, yielding to the other when it makes sense. the way people work on mathematics is a great example of this. drawings, diagrams, graphs, shapes, etc. are used to convey structural information where text and symbols are used to convey more detailed, descriptive information.

They are adding words in the form of thoughts or speech to describe the drawing.

that's not always true. the visual representation of an idea that can also be conveyed in text can oftentimes provide additional information not easily gleaned from the text-based representation. the easiest example of this is the visual representation of the graph of a function. a more advanced one is feynman diagrams. also think of symbology that is so ubiquitous in mathematics. it's much easier to write x_n->x absolutely rather than detail out the text-based definition of absolute convergence. obviously this latter idea is close to writing a function or procedure in code that consolidates an idea to a smaller textual representation, but symbology in mathematics far outpaces that of symbology in text-based programming languages.

There are hybrid languages, like Python... Visual whitespace denotes blocks while visual symbols like "if", "for", and "def" denote the meaning of those blocks.

(and sane programmers use visual layout to convey structure in every language)

I don't think that meaningful whitespace counts as "visual programming." That just means that a whitespace character has meaning, just like a word character or a punctuation character. It's still purely text-based. I think that "visual programming" means that the editing environment to some extent visually represents the intended outcome of the program. WYSIWYG web editing is the obvious extreme example of this. Whitespace in a Python program has no visual correlation to the output of the program.

Ah, I see. But other than WYSIWYG editing I'm having trouble imagining how a language could be general purpose and could still be visually representative of the output of the program rather than the behavior of the program (much like the visual layout and syntax coloration of modern text languages conveys).

Are there other examples of those cases?

Edit: maybe I was presumptive to assume that they had to be "general purpose". Are we talking about more narrow domain-specific languages here?

I think the "visually representative of the output" isn't a suitable way to describe it.

The visual language I always think of is LabVIEW. It is very different than a textual language.

No it's not. A for loop is a block with inputs, a space for sub-blocks, and outputs. To add an iteration limit, a box with a literal number can be attached. This is visually represented of the program's flow graph. Which does make sense when actual signals are concerned, to extend the wire metaphor of analogue equipment. Reason did that too, where instruments had a back pane to connect them to effect panels via actually visualized wire plug-in. Buzz or unreal blueed apply the same idea, as does model-sim. Block diagrams are just a very popular abstraction in signal theoretic topics. So Algol syntax, the languages like C being close to the hardware, uses the same visual metaphor.

Erlang (Prolog) for example doesn't so much. Sure you have parameters, but matching blocks aren't strictly evaluated in sequence.

> This is visually represented of the program's flow graph

LabView is more of a data flow graph than a control flow graph, aside from the boxes that represent deviations from the usual rules (For loops, conditionals, sequences, etc.)

This isn't as much pedantry as it might seem... this is one of the big differences between LV and 'traditional' programming... LabView evaluates its data flow graphs in data-dependency order, which can also mean evaluations are done in parallel. (IIRC, for linear execution speed, wires in LV all have a fixed type and blocks can be compiled to machine code with an LLVM back end.)

Now that I Google it a bit, it seems you're right that my definition of "visual programming" is too strict, although I do still think that most people wouldn't consider Python's whitespace to qualify. It looks like most so-called visual programming environments are effectively visual diagrams of standard control flow mechanisms like loops and conditionals.

The point I was trying to make is that the text in a programming language like Python is like a visual diagram of standard control flow mechanisms. I can see the for loop, what it iterates, and what the body of the iterator is. The language has visual clues. And with syntax highlighting it's even more visual.

But, I get that there is a class of visual languages, often dealing more with interconnected nodes like a directed graph, for which graphics might be better at communicating. But they seem more domain-specific rather than general purpose programming (like most text languages).

> then why does everyone sketch a picture when trying to convey an idea?

Does everyone do that? I think I very rarely do. When I need to convey an idea on a whiteboard, it's still almost completely text, with a few extra symbols like directional arrows that I think are still effectively just symbolic written language.

Literally the only time I can remember sketching a picture to convey an idea is while playing Pictionary. The entire premise of that game is that it is much more difficult to get someone to say a specific word by drawing a picture than by simply writing down the word.

Sometimes they do. In my view the most prominent example is documentation for installing computer hardware and software on Windows computers. The instructions will be a series of pictures, with circles and arrows, and a paragraph on the back of each one (sic). This goes on for page after page. And when the OS is updated and the dialogs all change a bit, the documentation becomes confusing or even obsolete.

In contrast, the instructions for the Linux version of the same thing will be something like:

    sudo apt-get install this
    sudo bing bam boom
    ...

I also hate that, but I always felt it was more a failure of the documentation writer than the GUI. "Click File -> Settings -> Whatever, open the Something tab, and click the "Do It" button under the Stuff subheading" can easily replace half a dozen unnecessary images.

GUI is documentation. If you have to document how to use your GUI you are doing it wrong.

It's not practical in a modern operating system to put every possible function at your fingertips and intuitively discoverable. There are better and worse ways to organize things, but there's just too much stuff to put all off it front and center.

GUIs are still much better than a command line in this respect, since with a GUI it's actually possible to find what you're looking for by randomly exploring, even if it's not always quick or practical.

One issue with GUI's is that you're sometimes stuck telling someone how to use a GUI that you didn't develop, for instance having a user manipulate settings in the device manager, which was created by Microsoft.

One thing that's rapidly improving on the command line side is Search. On my modern Windows computers (8 and 10), I don't search for things by hand in the dialogs any more. I press the Start button and start typing text.

For instance, Start plus "device manager" brings up the device manager, whereas I have no idea how to find it in the menus.

It depends on what kind of idea you're conveying. For many things, yes, text-heavy diagrams are the best way to go. If you're trying to, say, explain what a claw hammer is shaped like, that is much easier to do with a simple picture.

I quite often use UI mockups to show an example of the end result (or ask others to do the same). It doesn't even matter if it's correct, but it's a useful way to start a discussion. I've tried to do this in words but often different people have different interpretations and it simply doesn't work without a picture.

I've also found it very useful when people are asking for things that don't entirely make sense, such as a chart/report that would require 3+ dimensions. By asking them to draw an example, they realize the challenge and we can make progress.

"Picture is worth thousand words", but if you just need ten words, maybe it's not worth the effort to make a picture?

It depends on context. If you want to show how and UI looks, then it's probably faster and better to just draw it, but if you want to show how function transforms a string, then it's unnecessary to draw bunch of stuff.

I consider UI mockups the perfect usage for visual programming environments. Obviously a visual product compliments a visual development interface. However, it's a very narrow realm compared to the rest of the programming universe.

It's the same as saying that drawing a picture is easier for most people than programming a picture, and it doesn't really have to do with the topic of visual programming, but rather visual design.

> I consider UI mockups the perfect usage for visual programming environments.

It can be, though the type of mockup I find more useful for discussion is much lower fidelity than that.

The feedback and discussion around a mockup is related to the fidelity.

Low fidelity, such as whiteboard or large sharpie drawings, will garner feedback that discusses whether this UI even makes sense at all, if there's a different approach, or how feasible it is to present everything this way.

Mid-fidelity, such as digital mockups, will often focus more on the actual layout, text labels and contents.

High-fidelity, such as a working prototype or something done in the actual environment (visual programming environment, or even as working HTML+Javascript+CSS), will often get into a discussion about font sizes and icon choice.

If you build a high-fidelity mockup when you really want to have the low-fidelity discussion you're doing everyone a disservice. Even if your team can see past it and still has the appropriate discussion, it's a waste of time.

For that reason, even when my mockup is basically a small modification to something that exists already, I'll often screenshot it, and then freehand draw (MS Paint style) overtop of it with my modifications, using ugly 2-4px wide lines. I've tried both, but consistently found the feedback is better that way -- even when the target audience is a bunch of developers that should be able to see past it.

Ah, I see what you're saying.

Seems like you've done quite a bit of this type of work, I've the exact opposite. I guess it makes sense that the fewer data points the clients are given, the more they focus on the broader picture.

Would you often start with a napkin drawing-esque mockup, and add detail to it while cooperating with the client, or would you sometimes start with a finished HTML+JS+CSS prototype and change things up as they request?

I'm not a graphic designer, so I generally try not to do things that focus on the design -- which means I tend to start on the whiteboard/sharpie/napkin side of things.

That said, some people just can't picture how things look unless you show them how it looks, and in the past I've worked with clients that would think it's literally a joke (and probably fire me when they found out it wasn't) if I come to them with a sharpie drawing.

I think the only times I've gone to a more high-fidelity version up front is when the client/stakeholders had no idea what they really wanted. A "finalized" mockup can start the discussion (even if it turns out to be entirely wrong) and then be used to get into the low-fidelity whiteboard/sharpie mockups. The important thing is to keep the discussion focused on the right thing, and don't stray into fonts/images/icons if you want to talk about the overall structure and what pages even should exist.

I find it's useful for sketching out high level designs, particularly showing how different components interact, or the flow of data through a system. Visual information in this context trades off specifics for a birds-eye view.

Godel, Escher, Bach covers this in great detail. Symbols are a weird idea in the human brain, and there are multiple pseudo languages throughout the book that when you really sit down and think about it, aren't any less ridiculous than the notation we use for mathematics outside of that book.

My idea-book (is that a crazy person thing?) is probably a lot like what you're talking about. I use words for my main data, different arrows going in between each sub-idea, and then sometimes more words attached to each arrow ("pub/subs", "queries", "happens once/happens always", etc)

I think that people who think visual programming could be a thing ought to sit down with APL for a few weekends, until they have the epiphany of "oh, the code is the same as the notation for what I'm actually doing" and suddenly they realize that notation and code are interlinked, and each are basically useless without the other, in the same sort of way that x--p--p---- is exactly as valid as 2 + 2 = 4, without context.

I get the allure of NOT having to write code, but it shouldn't be so difficult for people to realize that it's a ridiculous fantasy.

Write fibonacci(N) in ANY visual language, and tell me you couldn't have done it easier, faster, and more coherently in python or whatever. It's obvious.

> Write fibonacci(N) in ANY visual language, and tell me you couldn't have done it easier, faster, and more coherently in python or whatever. It's obvious.

  [0]   [1]    [x]
   |_____|(+)___↑ ...

Pictures are a mission critical for my thinking and my communication in software engineering.

I would be hard pressed to discus the operation of any sort of datastructure beyond a linked list without sketching. For example a binary tree and it's traversal routines are (at least to me) much easier to understand with a picture along side the exact state.

They do that when the idea is visual/geometrical or when they use an imaginative visual model to process it, which is one of the things we actually do with text. Text evolved specifically to assist with ideas images(glyphs) could not.

Check out this video. He has a nice explanation. https://www.youtube.com/watch?v=4_SvuUYQ5Fo

Basically, painting/diagram is good for more abstract overview, and text/code is for more specific instructions.

> Check out this video. He has a nice explanation. https://www.youtube.com/watch?v=4_SvuUYQ5Fo

>Basically, painting/diagram is good for more abstract overview, and text/code is for more specific instructions.

very well Said ..

Sketching a picture can help for certain classes of problems, but I think that is far from universal: it often tends to be of less use for logical concepts than structural ones, for example.

Beyond that, though, even when a sketch is a useful tool, I find I seldom use the computer to create it. A whiteboard, notebook, or back of an envelope all seem easier than working with typical graphical input mechanisms that are commonly available. Good touchscreens are a step in the right direction, but still far from perfect and far from universally available, especially on desktops. Aside from any innate problems with graphical programming in general, I think this input problem makes the approach generally unappealing at the moment. It's not an insurmountable issue, but until there are good input mechanisms that are as common as keyboards, I think you will just find that it is hard to reach the critical mass a new representation needs.

I know that input speed is not really a limiting factor in most programming, and maybe shouldn't be the deciding factor, but given how attached to their text editors programmers already are, and how disruptive and frustrating it can be to have to switch, I think it's clear that it is an issue.

> Then why does everyone sketch a picture when trying to convey an idea?

Because pictures are helpful, but neither necessary, nor, in most cases, sufficient. Just look at the use of pictures in science and technology generally (e.g. circuit schematics, Feynman diagrams), and try to imagine understanding what is being presented without reading the words.

I have seen examples where pictures can help a lot (I am thinking of a pictorial demonstration of the Pythagorean theorem, for example), but I am not aware of a constrained pictorial formalism that is both general and expressive enough to do the job of a programming language (directed graphs may be general enough, but are not expressive enough; when extended to fix this, they lose the generality.)

>The complete dismissal of visual programming by text-based programmers is often infuriating.

I, for one, would be thrilled to see an example that actually works. Arguments that it should work don't count for much, otherwise.

There are some hybrids that are pretty useful in their areas of applicability, such as state transition networks, dataflow models and Petri nets (note that these three examples are all annotated directed graphs.)

The problem with having 2 representations of the same piece of information is maintaining them in sync.

If we use text for specific and visual for abstraction, we still need to maintain both and there are no effective tools to do that without creating additional work.

There are solutions to this. A) Let one representation be derived from the other. Easiest by making the derived representation read/view-only. B) Encapsulate the specific/low-level details into a generic black-box, which can then be composed.

Because each method has its pros and cons. It's a difference of generality and specificity.

Consider this list as a ranking: 0 and 1 >> alphabet >> Chinese >> picture.

All 4 methods can be useful in some cases. Chinese has tens of thousands of characters, some people consider the language close to pictures, but real pictures have more than that （infinite variants).

Chinese is harder to parse than alphabet, and picture is harder than Chinese. (Imagine a compiler than can understand arbitrary picture!)

Because that way you can abstract away lot of fine complexity and communicate higher order ideas faster

That still means you need writing to express all the intricacies of programming the details.

Alas hybrid languages where you can link developed blocks visually exists and works (bpel, webratio), but no pure visual language could let you describe the intricacies of taxation in a sane way

If you're referring to icons, good and clear icons take a lot of time to design.

A picture is better than 1000 words. But, which words? or for programming, which code?

Most visual programming efforts focus on representing algorithm logic with diagrams. But I think the difficulty to understand most complex programs (from my own experience) lies in the complexity of data structures, not the code logic. Reasoning about the algorithm is like reading a story, it is kinda intuitive; but for a large data structure (say a big struct with more than 20 fields), it's very hard to keep all those in your memory, you always have to go to its definition to refresh your memory, very frustrating. One of my tricks for reading a large open source project is to first find all the key data structures, draw them and the relationships on paper, then reading the code becomes easier and more pleasant. So maybe if a visual programming UI displays data structures with a visual format always on screen, it can help boost coding productivity.

Well, it depends. If we had to describe the situation in very graphic detail, then images can be more succinct.

> It started with cave paintings and evolved to what it is now.

Actually, our writing system evolved from accounting tokens. Eventually we learned we could save on little figurines by pressing the one into clay a bunch. And at any rate, writing encodes speech and is severely limited to the limitations spoken speech has. Rather than be well-optimized, writing is severely limited by tradition, by the physical limitation of auditory communication, and by the physical limitations of encoding it into paper.

I think this is exactly right, but I would add that when we start using computers to analyze and help write programs, faster, more intuitive ways of programming will emerge.

"Painting" it in this case would be a flow chart thingie with an action leading to a decision tree with two outputs. Not extremely complex, and with a bit of practice and a good environment, it probably takes about the same amount of time as writing.

There might be other cases where this is not true, but I think the major gains with a proper visual environment would be on a higher abstraction level. Visualizing the architecture of a project, tracking dependencies, etc.. Basically tasks where your "takes longer to draw" argument is nullified by the fact that it is the computer that does the drawing.

Imagine a world where people only bought bespoke suits. You go to a tailor and are measured and pick out your fabric and wait a few weeks. Then you pick up your suit. It's perfect. You pay $5000 and take it home.

Most of us don't do that. Most of us make due with off the rack suits with some tailoring for 1/10th the price.

With programming we have an expectation of bespoke design, so anything less, even if it's much easier, seems lame.

A potentially useful reframing of the question is, "What are all the common use cases that could be solved sufficiently with significantly less effort using visual programming?"

More like you please :-)

you win the prize for asking the right question. the answers are like with AI, where once it's done in a certain domain, then that's not AI anymore. that is, with visual programming, once it's done in a particular domain, people don't think it's programming anymore.

the big examples i have in mind are: * spreadsheets -- very visual, everything's in a grid. the relationships are spatial. * electronic music -- people lay out their various effects in a flowchart format and chain them from one to another * video games -- consider a game like RimWorld. you're clicking on all kinds of things and specifying what you want done with them. the behavior of the actors in the environment are modified by your specifications.

So, it's all in various optimized subdomains. As it should be!

And who could forget the most successful visual programming tool of them all: Photoshop. A relatively simple visual programming language for putting together programs that generate images of various kinds.

Most of the things you do are achieved through a visual programming environment specialised for that particular task. Maybe text-based source files in complex directory trees, managed through a structured text editor with UI composition helpers represents one kind of specialised visual programming environment as well.

Sooo... Minecraft is a visual programming tool, then?

Well, in light of the fact that someone built an emulator of the 6502 processor in Minecraft [0], it would be difficult to argue that it's a not visual programming tool! (of some kind at least ;)

[0] https://www.theverge.com/2012/5/21/3032765/minecraft-emulate...

Agreed. There are a lot of good points in this thread about why visual programming falls short for general purpose computing. However, I would argue that it already has a firm foothold in select domains such as presentations (powerpoint).

The way people have implemented visual programming has turned out to be a bit of an oxymoron. Visual and symbolic are entirely different channels of representation & understanding (for the most part). Representing a program using lines of code and throwing some "visual magic" to spread out lines of code across a screen doesn't take advantage of what EITHER visual or symbolic manipulation has to offer.

Visual programming efforts have tried to visualize code and operate at the wrong abstraction. Programming is just the act of creating programs. In a visual programming environment, the author / programmer should use visual interactions to "write the program". This means a few things needs to be considered:

- the representation - how is a computer program represented? how is state represented? - direct manipulation - imagine using a tool like Photoshop having to compile / build to see your changes between every edit. - input devices - mouse / keyboard works for writing lines of code. Touchscreen is a better input device / interface for visual programming but still lacking (no tactile response).

In my opinion, not enough is reinvented when visual programming efforts are tried. I think the entire stack (human input --> program state) needs to be reinvented and reconsidered. Bret Victor has a great section on visual program in his FAQ for one of his talks - http://worrydream.com/DrawingDynamicVisualizationsTalkAddend...

I have been working on xol, a graphic based programming language.

xoL is a graphic based programming language. It represents programming concepts with graphics instead of text. It is product of a long, painstaking design effort, to get an optimal graphical presentation of programs. A good description of it is available in this blog post: http://lignixz.github.io/blog/posts/17/xoL_graphics_based_pr...

A partially working prototype from a previous version is available online. The newer current design has fundamental improvements over that previous prototype. The way some program elements work was redesigned. The control method was also redesigned. It is now aimed primarily at touch based controls. Here is the prototype for the previous version: https://github.com/lignixz/xra9 . The prototype is not optimized for performance so excuse that. It is functional to some extent, if you can figure how to use it. You can add and modify programming elements.

Would welcome any opinions/feedback. Also interested in finding partners/investors/sponsors that may be interested in this project.

It looks kind of similar to a dream project of my own. It's a project that I've promised myself to take up on one day. I started building a prototype in haskell, with front end in Gloss. Tried to get inspiration from functional reactive programming/modelling.

I thought about using something like a prototype called Hydra (1) as a sort of runtime for evaluating graphs. Anyway, I really want to get back to it at some point but other cool things that are slightly more low hanging tends to get in the way. :)

Another nice inspiration for these things is Ecolanguage (2), not really programming related but a diagram language for visualizing economic transactions.

(1) - https://github.com/giorgidze/Hydra (2) - https://www.youtube.com/watch?v=-QI1iuAvTKE

I have been working on a visual programming language for Haskell: https://github.com/rgleichman/glance Currently, it uses Diagrams and Graphviz to visualize Haskell programs. Please email me since I think there is a lot we could discuss.

I'll take a look, but if I never get back to you I would like to share with you this one important concept that has caused most efforts to fail.

Let me start with a quote from Brian Kernighan "Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"

When you make a graphic based programming language the debugging better be twice as good as the programming and this includes when things get really difficult with lots of threads, or when the over arching abstraction breaks at some low level and you are exposed to the underbelly of the CPU. At least C++ lets you gracefully traverse from the very high to the very low which is why it is favored in the end, you are never stuck you can always go lower.

I wonder how Victor's Realtalk project will incorporate his interest into visual programming: https://harc.ycr.org/project/realtalk/

Also, for anyone interested it looks like he just posted on HN for the first time under user 'worrydream'.

It's going to be entirely different I feel. Instead of manipulating graphics on a flat screen, focus will probably be on manipulating physical objects on a table / in a room.

Watch his 2 talks that highlight the thinking and inspiration behind the RealTalk project:

https://www.youtube.com/watch?v=klTjiXjqHrQ https://www.youtube.com/watch?v=agOdP2Bmieg

There is a couple of reasons. First, VP is essentially just putting code snippets into nicely coloured boxes, which adds no value apart from making you have to scroll/zoom more. Additionally the structure of the visual becomes important, opposed to text which can be infinitely appended to, and so once you add enough complexity the whole thing becomes complete spaghetti.

VP also hides away a lot of implementation details from you, and assumes the computer will be able to figure out something better than the programmer. For example, when would the dropdown run the SQL query? on page load? on drop down open? As you scale the app bigger, you may want to run a single SQL query and then chop up the results and hand them out to several dropdowns. If your VP interface doesn't support this then your done for, but with code there is always a way.

There is always Clojure of course, which lets you edit your GUI while the app is running, and upload code changes without having to rebuild, allowing you to tweak stuff as you go. REPL driven development is great as it gives you both the flexibility of written code, as well having close to the interactivity that VP offers.

>> For example, when would the dropdown run the SQL query? on page load? on drop down open?

You seem to be dancing around what I came to comment on, and that is event driven programming. It's OK for simple UIs but it tends to break down as complexity increases.

I've been working on a project that does exactly what you described for the last few years. IMHO vue.js/react really changed the game here, especially for CRUD apps.

For example what I did was to code a server part in golang and use vue.js on the frontend and for the app builder. Using the web-based app builder (also 100% vue) the user can design forms. The forms are stored as serialized JSON in the database and the server renders dynamically vue.js components. It's super fast. And there's a complete freedom to design any form with a 16-column responsive grids and a lot of widgets and styling options. Forms are also nestable (thanks vue!) and completely extendable with custom JS code so it's possible to create a custom widget and then simply import it in other parts of the app.

As I said I think the shadow DOM and frameworks like vue really made the difference. I started without using such frameworks and it was not only tedious, but the resulting app wasn't very interactive and felt like a customized Excel spreadsheet attached on top of a PostgreSQL database. Now it's a completely different story - it's easy to build fully reactive apps that feel like a "normal" cloud app and are actually pleasant to use. Of course instead of spending few months designing and building it, it was created in a few hours.

So yeah... I think that drag-n-drop builders are definitely coming and will probably eliminate the need to write such apps in coming years.

Is it something you are doing as open source - pls share link if so (above comment downvoted, why ?)

I too am curious about what you've built, feel free to email me (on my profile)...I have been working on Solvent (link in profile) and have been exploring Angular or Vue as the UI building component.

They might be talking about their older submission.

https://news.ycombinator.com/item?id=12092527

Yes, that's a very early version of the project from one year ago and before migrating to vue.js. I haven't updated the site or the demos since then, but the new one is coming next week.

Also very curious to see what you've built as I'm building something similar right now. Would you mind sending a link to adam@useindio.com?

That's cool -- you should share a link! I've actually also been working on something just like this, which is why I asked for feedback with this. I'll post it on HN tomorrow via a Show HN so we can compare!

For anybody interested -- email is in profile. Happy to give you a sneak peek!

Awesome, looking forward for your post on ShowHN!

This is cool! Yes, looking forward to tomorrow's post.

I think if drag and drop builders find an audience it will be in a sweet spot between engineers and clients, maybe used by support engineers or customer success to build and modify applications to client specifications. Technical users will always want more control and in my experience clients are happy to pay for someone else to do the work, even with DND (I've worked at two companies that have built DND form/app builders, neither caught on much with clients).

I absolutely agree.

That's why I don't target primarily developers, but the end-users of such apps. Of course knowing Javascript wouldn't hurt, because it opens a lot of more possibilities to further extend the entire app, but my idea was to make it possible even for non-developers to create apps for CRM, invoicing, accounting or inventory management for their needs. So in my case it's not really visual programming, but a drag-n-drop builder, which also has an integrated code editor in case the user needs it: if they know SQL, awesome - write a custom SQL and it is automatically mapped and available in the forms.

This sweet spot that you talk about between ease of use & flexibility is definitely key.

> I think that drag-n-drop builders are definitely coming and will probably eliminate the need to write such apps in coming years.

I'm honestly more concerned with maintenance costs more than building costs. In my experience the cost of the alpha release goes down with visual editing, but the cost of subsequent releases ramps up over time.

> I think that drag-n-drop builders are definitely coming and will probably eliminate the need to write such apps in coming years.

People were saying that 10 years ago, they were saying it 20 years ago, the general attitude was even the inspiration for COBOL (that business people could write their own apps). Every single time it ends up being a disaster when it comes to slightly non-trivial use cases.

It's been tried and failed by companies large and small for decades. So what do you know or do differently that they didn't?

I really enjoy Unreal's Blueprints. I program in a text-based language for work, so it's fun for me to go home and be able to hack on a game without having to spend more hours staring at text on a screen.

A big difference that I've noticed is that in a visual scripting language, the "nodes" in the graph can change their shape/color/layout/etc based on what type of node they are. I love this. With blueprints, if I'm putting together a material and I put in a color node, a color picker shows up. If I put in a TextureSample node, a little picture of the texture shows up. In a text-based language, every variable is just a string of characters - if you want to be able to tell at a glance what type of object a variable contains, you need to adopt some kind of a naming convention (or use an IDE that highlights them).

Debugging in unreal is also really cool. The "code paths" light up when activated, so it's really easy to see exactly which branches of code are and aren't being run - and that's without actually using a debugger. Side note - it would be awesome if the lines of text in my IDE lit up as they were run. Also, debugging games is just incredibly fun and sometimes leads to new mechanics.

I initially thought that organization would be a problem, but it turns out that you can make functions just like any other programming language, and encapsulate blueprints inside of them.

There are also some things that I don't like about Blueprints. Typecasting usually involves an extra node which feels verbose. If/thens are handled with a "Branch" node, which again feels verbose.

I guess it depends on what one considers to be programming. For certain DSLs (audio, control systems, UIs), it seems to be doing OK. For general-purpose work, I think they will always be less desirable than text. Even after thousands of years, words are still our most powerful tool for recording and sharing new ideas.

Alan Kay's mistake is that a lot of his original work was with Papert on educating children. For early learning, having a visual/physical analog adds value. For professional work, when you already have the concepts down, it just gets in the way.

It's like the difference between a kid's picture book and an adult novel.

I feel this comment significantly devalues graphic novels. It's a separate art from the novel, but not an inferior one. Pictures have value for adults too.

Things that are difficult in visual languages:

- Humans are centered around linear communication. Spoken language is essentially linear, with good use of a stack of concepts. This story-telling mode maps better on a linear, textual representation than on a graphical representation. When provided with a graph, it is difficult to find the start and end. Humans think in graphs, but communicate linearly.

- Graphs are rather hard to manipulate. You invariably need a mouse, which is a rather slow instrument. Using keyboard combo's is certainly possible, but is difficult to make intuitive.

- A textual representation can be left in a syntactical incoherent position while editing. For example, I can have non-matching braces. This proves very helpful in quickly editing code. Visual programming paradigms generally require a coherent structure, in order to keep them editable.

- UI builders are extremely valuable, but this is technically not visual programming. UI builders have problems as well. Complex UIs are often parameterised (the fields themselves are variable). By offering a visual context and a textual context, the coding experience can become incoherent and more difficult to learn.

- Programming is very context dependant. A programmer focusses on a specific part, mentally forming a high-dimensional representation of the algorithm at hand and the larger context. Using a visual representation possibly competes with the mental map. Also, it forces upon the reader a scoping.

I disagree with several here: The problem is that we still program in text, and that's deeply sub-optimal.* Once we move to programming in data, visual programming becomes both natural and intuitive, much like data visualization.

In other words, VP itself is a red herring. The leap that needs to first to be made is that from text to data.

*https://www.emaze.com/@AWOCZQLL/Text-is-for-Novels

That's the claim, but where's the evidence? Is there any implementation that can show us that?

It will come. Nothing today. The leap that needs to be made is actually in the data structures themselves but I understand your skepticism until it arrives.

Interesting presentation. One thing that we agree, is that the current state of programming is not final. There have to be better ways for humans to program. I wonder how much people thinks like that. A poll would be interesting.

In reality, programming has been stuck in the text approach for many years. Perhaps some people take that as evidence that there is no more progress to be made. Another factor that may be undermining research in programming, is the AI boom. Perhaps some think that AI will displace programmers in a not too distant time.

I agree with you that programs are data. And would benefit from a more specialized representation. I have been researching the aspect of program logic. For example, one thing that I think is terrible in programming, is deep nesting levels. That makes it complex to follow programs. I have been exploring ways to improve that. In my opinion graphical visualizations can provide ways to improve that.

It doesn't suck if your code involves complex flows of relatively simple data. Programs like Reaktor and Max MSP have been hugely successful for this kind of coding in the music world.

If your data structures are more complex and require more convoluted logic, which is more often the case than not, then the benefits of visual programming diminish and the downsides are much more apparent.

GNU Radio is another example of this succeeding with manipulating flows of data.

But I agree that's an intrinsically different problem than what OP is describing, which is essentially a WYSIWYG editor.

I've seen a research lab in Tokyo use Max MSP for controlling the motion of a humanoid robot (using pneumatic actuators). I believe that whenever there is some flow involved it is easier to model it graphically than textually.

VB6 and (especially) Delphi continue to have many fans. Sadly, there is little effort seemingly being expended in simplifying programming for normal people, so a lot of modern visual environments just haven't taken off, which further reduces incentives for investment in the field, and leads to them being (correctly) perceived as crippled and unsound.

Also high-mean IQ dev outfits are likely to produce tools for people like themselves. "Cognitive empathy" is not going to be a strong point for the framework creators at Google or Facebook. Microsoft seems to have quietly abandoned Lightswitch, which was their last high-profile attempt in this space.

Finally market forces mean that working developers have no incentive to adopt simpler technologies, so there is no pressure on vendors to simplify things either.

One size does not fit all.

UI creation is best handled by a WYSIWYG editor, animations are nice to visualize on a timeline, and math formulas are clearest when written as is.

Picking the wrong model creates a mess. Try to express a sorting algorithm as a flowchart. It would be too difficult to follow. Yet a business process is great to model as a flowchart. A picture is only sometimes worth a thousand words.

Take a look at the papers published from VPRI, the research org founded by Alan Kay. They have built languages for graphics, parsing, stream processing--each optimized for its particular domain. They could have used a general purpose language but it would have resulted in much more code than necessary.

So the question everyone should be asking is: why hasn't a multi-paradigm programming environment taken off?

> UI creation best handled by a WYSIWYG editor, a business process is best modeled as a flowchart

I disagree with those two. UI creation has generally evolved away from WYSIWYG, composing a UI with code is a much better approach and is now nearly the universal way of doing it. For the business process, the flow chart only works as an extremely high level overview, the implementation is always done in something like code.

> composing a UI with code is a much better approach and is now nearly the universal way of doing it

Why is it a better approach? Universal does not always mean better. Javascript was built in 10 days and it's now universal.

> the flow chart only works as an extremely high level overview, the implementation is always done in something like code

I agree here. The core point I am trying to make is that one size doesn't fit all and we have to mix and match approaches based on the problem.

> Why is it a better approach? Universal does not always mean better. Javascript was built in 10 days and it's now universal.

Easier to diff, easier to copy paste, easier to upgrade and easier to extend. In the 90's we had both approaches and it was the code based (Qt, GTK) and declarative (html, xaml) systems that (mostly) won.

If I'm working with win32 or winforms then I love the UI designer, but that's because they come with an awful API, for anything else I'll take the code approach.

For me at least, graphical programming tended to crank code style issues up to 11. If you think linear text code formatting is bad, laying out a flow graph on the screen is worse. If you think naming a function is bad, try drawing an icon to represent that function. (Or each function in a library of ten or twenty related functions... )

Graphical programming also walks away from a huge body of work dedicated to managing, searching, and manipulating text. As good as the graphical tools can be, there's nothing like all the work that's gone into text editors, diff tools, query languages, etc. These are all things that are central parts of my programming workflow, and they are all missing or considerably weaker in a graphical language. (This is not to mention the fact that graphical programming gives up two hands and ten fingers as input devices in favor of one hand pointing and clicking with a mouse.)

I think as appealing as the idea is, whatever benefits there are don't outweigh the considerable costs.

Programming languages (and a lot of other things) have essential and accidental complexity: https://en.wikipedia.org/wiki/No_Silver_Bullet?wprov=sfla1

Progress in programming is in removing the accidental complexity to get to the essential.

In the vast majority of domains, visual programming injects huge amounts of accidental complexity utterly irrelevant to the domain. So it isn't really surprising that despite it's superficially appealing elevator pitch and uniquely pretty demos, it doesn't appeal over the long term.

The domains where it works are those where the complexity is limited, or where the usually-accidental complexity is actually fundamental... for instance, in circuit layout the usually-accidental issues involved with layout actually correspond to something.

Visual programming is firmly established in some nights, and I expect it to stay in them indefinitely.

What really annoys me in text programming is that I have to write constructs again and again, referring to manual for method names constantly. Completions may be smart typewise, but IDEs never assume what I am writing. Let's take a function:

  static string defaultFamily = "Sans";

  bool
  foo(string s, Alignment a, int height)
  {
    Par_

Okay, it began. What can be done with all the available values? Seems that I'm going to create a paragraph and fill it with some arguments. One of autocompletion suggestions:

  bool
  foo(string s, Alignment a, int height)
  {
    Font f(defaultFamily, height);
    if (!f.isValid()) return false;

    Paragraph p;

    p.setAlignment(a);
    p.setText(s);
    p.setFont(f);

    <#rest#>_

    return true;
  }

How is it obvious? Because Paragraph told the completion system that it needs at least font and text values set. Height goes to font, other arguments go to paragraph. Names are selected from the style around. Since we return a boolean and few objects can be invalid after initialization, it assumes that it is a status. p is local, so there is something to be done with it, and another <?> block is created.

Even better, you could just throw items together and get the result:

  // this->points is double[20];

  {
    frame inset 0.5 red stroke;
    add points black stroke;
  ---
  {
    Rect r = this.frame();
    r.inset(0.5, 0.5);

    Painter p;
    p.setColor(Color::red());
    p.addPath(rect);
    p.stroke();

    p.clearPath();
    for (point in points) {
      p.addPoint(point);
    }
    p.setColor(Color::black());
    p.stroke();

But the completion style that we have everywhere just sucks:

  int
  foo(string s, Alignment a, int height)
  {
    ParadoxDatabaseProvider_
    ParadoxDatabase...
    ...
    Paragraph
    ParamsManager

I can't point to good visual programming tool, but our "powerful" text IDEs are simply prehistorik typewriters.

> Because Paragraph told the completion system that it needs at least font and text values set.

If the class needs at least all those arguments it shouldn't have an empty constructor. It should have those as arguments to the constructor and then most good ide today would be able to help you at least a bit better. Have you tried intellij? Boolean return values without context is also an antipattern, how am I supposed to know whether true means success or fail or something else? There is no way any AI would be able to infer that either without annoyingly amount of false positives.

Okay, "paragraph told that non-default font is often used and empty text is not very useful". These are just examples, I'm not pretending this to be final form. How about non-construction?

  foo(string s, string p)
  {
    fin
  ---
    int i = s.find(p);
    if (i < 0) <#?#>;
    _

Also, if it appended { on if-line when I press Tab, that would be great.

>There is no way

Really? I clearly see how to do this without any sort of AI. That's graph path weighting problem based on key words. If it gives you a false positive, just add words (or put more useful hints and >>use case snippets<< if you're library writer).

But I got your point, let's walk on the same rakes for life. There was a time when I was amazed by programming, now after two decades I'm tired of it to death (and yes, it seems that I'm "10x" from the feedback around). Programmers create and solve problems that mere mortal cannot even understand and this cool process is far from "cheap". Everyone talks about no-brain easy programming, but no real steps are done in this way. You know, when you're doing something often and for years, you just get used to it, know it all, and see no need for a change, forgetting the learning curve. You can write all of above snippets with closed eyes. The same happens to the entire profession -- you are professional and that's your value. We doomed to raise professionals again and again. In what areas? C++, SQL, etc... But wait, these are no areas! Areas are: medicine, building, etc. I don't want "cee pulse" programmer, I want a guy who understands my area.

I'm attributing these "no need for this" attitudes to the simple fact that such a system would be first step to programming without paying a programmer who solves non-existing problems. You simply put words and the system analyzes best practices and use cases from selected github projects or templates to produce a result, and you only need 0.1 of your development forces. Professional experience is bound to a programmer, but this thing would allow to change that. The same friction we seen when computers appeared. People hated them simply because their professions ceased to exist.

Lyrics off, software libraries are graphs. And graph is a thing that can be processed, if not in generic, then in specific way. There is algorithm that produces the most wanted result for known specific case.

I don't think it is the paradigm that sucks but the existing implementations...it is a hard design problem to solve. No one has quite figured out the right approach....often the existing implementations make you click and fill out enough fields to make the entire exercise much worse than just coding something up in a text editor.

I believe the solution lies in coming up with a way to compose pre-built components into complete applications...I think the existing efforts involving Angular,React,VueJS,Vaadin..etc are all pointing in the right direction....I have been working on Solvent (www.crudzilla.com) for sometime now and solving the UI building (drag-drop) is something that I am still actively thinking about...one of these days someone will crack the problem :)

> I don't think it is the paradigm that sucks but the existing implementations...it is a hard design problem to solve. No one has quite figured out the right approach....often the existing implementations make you click and fill out enough fields to make the entire exercise much worse than just coding something up in a text editor.

True. That's why a proper visual programming environment should build on existing text-based environments and offer the same capabilities while adding power. Not disregard all progress, conventions, and achievements and start form scratch offering some unique features, but at the same time taking away stuff that's already been figured out.

Agreed, that is more or less the approach I have taken with Solvent...it has a familiar IDE environment with the additional visual layer on top of it.

I think that Delphi nailed Visual Programming. It has a mix of speed and convenience that I do not find in other Visual environments. The fact that Pascal compiles so fast also helps with the instant gratification when prototyping an application.

The downside is perhaps that this very same instant gratification makes you very prone to add business logic in the UI, and it the application sticks around, you eventually want to refactor the whole thing.

Picture says it all: http://thedailywtf.com/articles/Labview-Spaghetti :)

See also: the mighty spreadsheet.

i program in labview daily, and it's always a little disappointing to see this same dumb picture posted all the time in reference to labview and visual programming. NO respectable labview programmer would work like that. you could make the same argument for text-based languages as well, as i have seen some atrociously formatted code in addition to poorly structured code.

one of the main issues with labview is that people simply don't take it seriously as a programming language. if people watched how i work everyday, i work just the same as, if not more efficient than, a good text-based programmer.

I've also done labview and have to agree that the picture is a bit extreme. Even so, extract the bottom left corner of that picture for something a bit cleaner and more representative of code in the wild but it's still completely impossible to see wtf is going on there.

it's always a little disappointing to see this same dumb picture posted all the time in reference to labview

Yup. It's not unlike the -insert whatever language here (C++ being a good choice)-haters ranting but not actually having used it a lot, or at all.

Moreover: depending on the type of UI, I don't think there is any language out there which can get me the results I need as fast, when I need something 'process control'-like, i.e. lot of display of e.g. electrical signals and a bunch of control buttons/textboxes/comboboxes. Then there is just no match for Labview. And I have quite some experience with writing UIs in WinForms/WPF/Qt... (typical example: good luck writing a pannable, zoomable, cursor-able live multisignal 2D graph in any of those - with labview you just feed data into the premade component and you're done) It even goes so far that even though the backend is in C++ or Python, I might take the overhead of writing IPC between backend and Labview frontend over doing the UI in the backend language.

If you hadn't had a chance to check out Labview NXG, I suggest it. 2.0 Beta is out now. Completely new editor experience. (Full disclosure, work on the core NXG team)

Haha, yeah, clearly text is so much better:

view-source:https//www.google.com/

The old "Gold Standard" for WYSIWYG programming has been Apple's Interface Builder. But every iOS dev I know has moved entirely away from IB in favor of programatic construction of the UI (in Swift or Objective-C text). Why? Because they want to have automated tests over everything they can.

The simpler way to say this is: "Complex ideas can only really precisely be expressed in text. To the degree you can express them visually is to the degree that there are expressed as complex text underneath."

Can't one test a layout built with IB? Load it in a unit test for instance?

One modern incarnation of visual programming is called "low code", and it seems pretty successful.

This is a good introduction and discussion about whether this is a fad or it's here to stay: https://medium.com/softwareimprovementgroup/low-code-wave-of...

Looking at the website [0], this is a joke right? Flashy marketing business speak sales site with charts, but what does the product look like?

[0] http://low-code.com

You're kidding, right ? that's a really low effort search :)

Anyway, i gave a link above, with good info, including names of leading companies. Here's one:

https://www.mendix.com/

Also , outsystems and bubble.is, oracle Apex

It doesn't. The electrical world has been using contactor diagrams for complex logic for decades, and with the inclusion of more advanced computer components ladder logic (still the same concepts, symbols and logic)

https://www.allaboutcircuits.com/worksheets/ac-motor-control... is the link I quickly found.

It's typical to see a handful of symbols handle what would take a hundred of lines of code for the level of user interface usability, validation, and core functionality.

Difference here is, we are using humans to fill in the gaps. With visual programming, we either still have to draw the exact wire-diagrams, or exactly define the function of a symbol.

Quite a few PLCs have ladder logic as a programming language as a selling point.

For physical systems it is pretty true.

I think this is a minority perspective, but I generally think coding has a lot more in common with writing than it does with engineering. (I guess Literate Programming would sort of support this idea). Can you imagine trying to write an essay as a flowchart? I think it would just obscure the point, rather than simplify it.

Structurally, a program is also similar to music. A program, if we view the bytecode, is read linearly by default but may have jumps at certain points forwards or backwards, like a coda in musical notation.

People make amazingly complex music in Digital Audio Workstations. Personally I think there is some inspiration to draw from their use/function into the programming experience.

Personally, I think that written language is a more natural way to describe and reason about complex systems and symbols.

It's the same difference between writing mathematical proofs as systems of equations vs. proof by drawing a diagram, or making a visualization.

Proofs can be written using both methods, but if you have a strong command of the written/symbolic vocabulary, it is faster and more succinct.

Put another way: I can describe the landscape of a lush, green Tuscan Countryside--with rolling hills, morning dew, and and crisp feeling of the first rays of sun cutting through the mist--in just a few lines. You get a picture in your head, and it is likely very similar to what was in mine.

Imagine now, that we can't use words. We have to draw pictures. It will take much longer to get the same point across. Language--for certain purposes--is more efficient, and more information dense.

That's my hypothesis/feeling.

I wonder how many people have physical issues with computer screens. My own anecdote is that I get severe eyestrain headaches and neck / wrist fatigue, when I have to do fine manipulation on a computer screen.

I spent a few months using LabVIEW, more than 20 years ago, and it was physically debilitating. I also have a hard time doing CAD. When I'm typing code, I'm not really focused on the screen unless I need to be. Sometimes I close my eyes and type, or look away from the screen.

Another thought about text based languages is that they make it much easier to create and explore new languages, because you don't need to program a full blown graphical editor every time, make it work on every platform, etc. Especially for languages that are supported by volunteers, maintaining graphical tools might not be the best allocation of effort.

Unreal has a VPL and it is a pain to use. A simple piece of code takes up so much desktop real estate that you either have to slowly move around to see it all or have to add more monitors to your setup to see it all. You think spaghetti code is bad imagine actually having a visual representation of it you have to work with. Organization doesn't exist you can go left, up, right, or down. If you ever try to enforce coding standards think about adding x and y coordinations and other location standards. Finally I could only imagine the black magic that goes along with such a thing. Visual studio black magic is already horrible to deal with one little hiccup and you are lucky if all you have to do is create a new project and migrate your files/folders over to it.

For the curious, here is an example of Unreal's Blueprint code:

https://forums.unrealengine.com/attachment.php?attachmentid=...

Visual programming is used in Maya or Houdini for example (I think Houdini is more oriented towards it). While these are 3D software, I don't think there's anything 3D specific about how it works. I've seen a video of a guy showing how to create assets for game and building something that works pretty much as a function out of interconnected nodes (I can't remember the exact webinar I watched but found this other https://vimeo.com/202120820). Now they don't call it programming but as I see it, that's programming and the basic idea could be made to work in other contexts.

Because of the Deutsh limit: https://en.wikipedia.org/wiki/Deutsch_limit

That's like saying it's impossible to design a motherboard because you can't show all the bypass capacitors on the top level schematic. Hierarchical design and layers of abstraction are vital in both hardware and software.

True; but visual programming means you need more 2D space (screen) to see the same amount of logic. Maybe with bigger screens it could work better?

Which is; > "The problem with visual programming is that you can’t have more than 50 visual primitives on the screen at the same time."

I think the truth is somewhere in the middle. Look at complex CAD systems and editors. They focused on the visual side and still provide some kind of command line for scripting the things. Even in pure visual programming systems like PD you can type algebraic expressions in textual form. And in text-based systems you may have lots of 2d tables (Smalltalk), graphs of modules, semantic colors (colorForth) etc.

Imagine GUI editor where the user can type the code on the left side and also can move and edit widgets with cursor on the right side. And editing of both sides is working in live mode. My point is that we need to have the best of both worlds to be really productive.

For web apps, this is possible using Chrome and the style editor. Very effective way of live editing visual styles. Functionality is a different beast though

> I think the truth is somewhere in the middle.

Totally agree. I think the major reason visual programming sucks is exactly because some things which are simple to do using text-based programming become way harder when all you can do is drag graphs around.

On the other hand, some things are easier and more intuitive when represented and manipulated visually using graphs or trees.

I think that a visual programming environment which would be an extension of a text-based programming environment, in that you could do everything that you can do in IDEs everybody is already used to PLUS the cool stuff you can do with visual programming, would have high chances of not sucking.

> Imagine GUI editor where the user can type the code on the left side and also can move and edit widgets with cursor on the right side. And editing of both sides is working in live mode. My point is that we need to have the best of both worlds to be really productive.

I made a hybrid visual and text programming language and prototype development environment for it as a project for my master's thesis [0].

The main idea was very similar to what you are imagining. Textual code on the left side and visual representation on the right [1]. You can edit one and have the changes reflected dynamically in the other.

Actually my idea was even more general and could be extended to parallel-editing multiple representation of the same syntax tree.

I would love have the time to explore this stuff some more.

[0] http://djedr.github.io/masters_thesis.pdf

[1] http://djedr.github.io/gfx/masters-thesis/ide-prototype.png

I agree on this point, the best solution is probably a visual editor that convert code to graphs an visa versa (while really working in code internally). Then users can switch when needed.

Also it will help explore/understand what is required to write and effective base/methods/approaches for visual programming.

If we need code (in form of text) anyway, then why even spend time on learning visual representation of the same concepts? Such learning is pricy and increases cognitive load on developer without meaningful increase in productivity.

Visual programming still may make sense in some narrow niche tasks though.

For event-driven programming like what you describe, Visual Basic worked fine. Too bad that kind of very efficient UI building got thrown out irrationally with the hate for Visual Basic the language.

Obligatory plug: We're bringing it back!

Check out https://anvil.works - it's like VB, only for this century (the web) and using a sensible language (Python, both client and server).

We just took it to PyCon, and the reaction was pretty gratifying :)

I believe general purpose visual programming hasn't become widely popular because currently it's not really needed for general programming. General programming used to be pretty low-level (e.g. i++) and linear (because it's single-threaded in vast majority of cases). Things slowly change though. Now popular functional programming tends to be more high-level and the shift to multi-core CPUs even on mobile devices encourages more and more developers write multi-threaded code.

At the same time there are very successful niche cases for visual programming with Excel as the most prominent. Yes, there are no data flows in Excel (although it can display dependencies with arrows), but you typically can easily identify visual blocks (chains) of calculations on a sheet. I've designed a visual programming tool for data transformation (http://easymorph.com) which can be viewed as a visual programming language[1] because typically the task of data transformation is high-level and non-linear, and visual programming works very well in this case.

[1] http://bi-review.blogspot.ca/2015/06/easymorph-as-visual-fun...

The common pattern of applying the same macro to each of the rows/columns can be seen as a SIMD/vectorized operation with the affected cells as inputs and outputs.

I can tell you where in my opinion visual programming works (or can work if anyone actually bothered), but you probably not gonna like it.

First step is to think about IDE assisted programming as a proto visual programming. Second step is to have a real DSL, not just some general purpose programming language with some extra words sprinkled that sort of look like a DSL, but you can easily escape. Visual programming tools that aims to do the job of general purpose programming language is kinda doomed from the start in my opinion.

Here's a snag - how many DSL languages do you know that have intelligent IDE support? Designing a good DSL language is kinda hard in the first place. I know of many, but they are called XML languages, deal-breaker for many. XProc can work as an underlying language for such a visual programming tool, with a good XML schema enabled IDE or editor such as OxygenXML you won't even work with XML as most see it, it's not really text anymore. Dropdown menus in a representation of a tree, that's what it becomes.

There were at least two attempts to create an actual visual programming tool on the base of XProc, most advanced was this one https://community.emc.com/docs/DOC-4382

Embedded DSLs (using the general purpose programming language as the host) is the way to go for an integrated workflow/experience. I think the same is the case for visual programming languages/tools.

Graydon Hoare (Rust creator) has an interesting post on a related matter "always bet on text" http://graydon.livejournal.com/196162.html

I deeply enjoy how this post uses images to illustrate its points. :)

> The type of visual programming I'm thinking about here is a WYSIWYG drag and drop interface for building UIs.

The main challenge in this particular domain is describing what is supposed to happen to the layout when the size of the window changes, or if there are dependencies among visual elements (e.g. some element only appears when a check box is checked). When laying things out visually you can only ever design one particular instance of a layout. If all your elements are static, this works just fine. But if the layout is in any way dynamic (with window resizing being the most common case) you now have to either describe what you want to have happen when things change, or have the system guess. And there are a lot of options: scaling, cropping, letterboxing, overflowing, "smart" reflow... The possibilities are endless, so describing all of that complexity in general requires a full programming language. This is one the reasons that even CSS can be very frustrating, and people often resort to Javascript to get their UI to do the Right Thing.

Yes there are a lot of options but I don't think that it would be inconceivable to have such options split out as separate components.

I want an hide-overflow behavior to this component, then I just wrap it with a hide-overflow component. Doing that drag-and-drop should work. Then again you would need to design such components in a way that makes them intuitive to use, but that's always the case.

Programming through typing text has already become 'visual programming', just from an unexpected direction: syntax highlighting, compile error checking while you type, code completion, fuzzy searching, argument type popups and so on. The only thing that's missing is better debugging integration (essentially show how the program state changes while you type).

Your comment is definitely an interesting take on the question. I would argue, however, that integrated debugging in the way you describe (along with the various programming text editor features) is what constitutes a live programming environment.

I think it really depends on the task. We are just hijacking the brain machinery to do jobs it is not evolved to deal with. If we can find some highly evolved / optimized brain function which reflects the structure of the new job, the brain can process it much more efficiently.

Text might be good for programming because, most of the programming are sequential, text is sequential, and text processing is highly optimized in the brain because language. But in fact, we also use a bit of visual programming (indentation, paragraphs) in text to reflect part of the program structure that are not sequential.

If the program is completely non-sequential, visual tools which reflects the structure of the program are going to be much better than text. For example, if you are designing a electronic circuit, you draw a circuit diagram. Describing a electronic circuit purely in text is not going to be very helpful.

My pet theory: Code is N-dimensional - every set of braces, every nested function call, increases the execution by a dimension and humans are really only good at visualizing in 2 or 3 dimensions. Specific domains that are constrained to 2-3 dimensions are suitable for visual programming, but it's hard to generalize.

Always bet on text: https://news.ycombinator.com/item?id=8451271

True 3 years ago but 3 years from now? Maybe not.

I think that Visual Programming could be more suitable for building modern applications that may need to make web services to monitor physical devices (for example) or to gather and analyze values coming from sensors,etc... That's exactly what has driven the design or our new Visual Programming language : the need of being able to rapidly and easily build applications that connect and use technologies that were not made for working together (https://www.youtube.com/watch?v=uRWIlDXeLdk and https://www.youtube.com/watch?v=JccLWZ6nPlY). (Full disclosure, CEO of Hackeet)

I'd like to share some thoughts:

- Visual programming tools usually only work at one abstraction level (your first point)

- Often centered around a gimmick and cannot encode "normal" detail-oriented imperative code efficiently

- Only a fraction of a given program is amenable to flow charts, Visual Basic-style forms, etc. Unfortunately, most visual programming environments are all-encompassing.

- So VPs tend to become blunt, awkward tools that lack power of expression, ergonomic editors, scalability to real usecases, etc.

FWIW, I do believe that a better code editor is possible through non-plaintext programming. (Serialize token trees and ASTs, instead of plaintext.) But such an editor would likely be used in text-editor-mode >50% of the time, just due to the high information content and sheer readability of text.

> I do believe that a better code editor is possible through non-plaintext programming. (Serialize token trees and ASTs, instead of plaintext.)

Funnily enough, this is itself another of those mirages, like visual programming, that have been chased for years and failed to gain traction (outside specialised applications).

For my money, the reason for this is that a human editing code needs to write something invalid - on your way from Valid Program A to Valid Program B, you will temporarily write Invalid Jumble Of Bytes X. If your editor tries to prevent you writing invalid jumbles of bytes, you will be fighting it constantly.

The only languages with widely-used AST-based editing is the Lisp family (with paredit). They get away with this because:

1. Lisp 'syntax' is so low-level that it doesn't constrain your (invalid) intermediate states much. (ie you can still write a (let) or (cond) with the wrong number of arguments while you're thinking).

2. Paredit modes always have an "escape hatch" for editing text directly (eg you can usually highlight and delete an unbalanced parenthesis). You don't need it often (see #1) - but when you need it, you really need it.

I've been thinking along the same lines. The "fighting your editor" problem cannot be ignored, and it's common in VPs. Real programming code needs "jank". We need to be able to move between different states:

Bags of characters <-> Unstructured trees of tokens <-> ASTs

(BTW paredit is super cool and I'd like to see more of its kind!)

Thing is, the architecture you just described is a modern IDE with refactoring etc.

This is a well-trodden road. It starts with "wouldn't it be awesome if we could manipulate everything as ASTs", then usability intervenes and we fall back to "well, we need to be able to selectively edit as text", which means you need to be able to convert everything to bags of characters and back. And now you've built that conversion, you might as well represent the "source of truth" as bags of characters like everyone else does.

I agree with what you're saying, except the last part. Using ASTs as the source of truth (embedded in a source control forest) has benefits that are worth the difficulty.

What's the benefit? If your tooling already requires lossless round-trips to and from text, why invent a funky storage format that doesn't interoperate and can't be fixed with a text editor when it all blows up? You already have a perfect serialisation of the AST, in a format every other tool understands - that is, source code.

(And heaven forbid you should want to make a checkpoint commit that doesn't parse...)

Some potential use cases:

- Using Merkle trees, we can assign every node a hash-based ID. So now we can refer to other nodes by ID. This lets us store a graph whose vertices are all the tree nodes.

- With the graph, we can now reference bindings not by string literal, but by ID. This eliminates shadowing problems and missing imports.

- There is now one source of truth for the names of variables and functions. As a result, in source control, a commit that renames something is a one-line change.

Visual languages never found a good solution to most of software development tasks.

For example:

1) How do I view diff between old and new version?

2) How do I search for an element in my visual code?

3) How do I email example of visual code to my colleague?

4) How do I message an example of visual code?

5) How do I blog about visual code?

Nobody here has mentioned language workbench tools, the best of which is probably Jetbrains MPS: https://www.jetbrains.com/mps/

These language workbench tools make it easy to switch between graphical and textual representations. Also you get intelligent IDE support and typechecking for free.

The primary alternatives to MPS are Xtext and Microsoft's language workbench (formerly known as Intentional Software, Microsoft just bought them).

I did some work with webMethods Flow in a previous job. I hated it. Aside from the fact that it was slow from a point-and-click point of view, the interface was generally slow, and navigation and search were generally tiresome. To my memory it was not possible to have related flows side-by-side for reference, and much of it was abstracted to a high enough level simply due to language features that it was difficult to identify patterns and idioms within a set of logic at a glance. This also meant that any opportunities for refactoring or code reuse were few not only because it was difficult to do, but also because they were difficult to see.

Much of this can probably be improved in the IDE, and it is possible that there were issues simply because we may have been abusing it (there other platforms/languages that were being used beyond their original design).

What little I've seen of LabVIEW at my current job suggests that it has a more usable interface, however having written DLLs to be used by LabVIEW, it still appears to be quite restrictive at a language design level which I expect would still limit its usefulness (I hear regular talk of the use of singletons, users appear to be required to translate a C API to LabVIEW manually, useful C language features like structs and function pointers don't appear to have any support, the ability to maintain a reference to a context pointer gives rise to groaning developers).

https://en.wikipedia.org/wiki/WebMethods_Flow

I think it's because some visual mediums are better at representing certain forms of information than others, and visual programming tools attempt to represent a form of information (programming logic) using a visual medium (images) that isn't optimal for that form of information. Analogies can be made for things like calendars (which are the preferred visual medium for visualizing schedules), sheet music (which is the preferred visual medium for visualizing music - though guitar tabs present an interesting alternative to think about), and movie scripts (for which text is the preferred medium).

In the same way that a movie script is best represented by text, my belief is that text is superior to images for representing programming logic, and images are superior to text for representing UI elements. I think most people at least feel the same about representing UI elements, which is why visual programming tools are appealing.

This leads to the conclusion that the best tools for building UIs would represent programming logic in text, and represent UI elements using images. In my experience, this is true; tools like Unity and Flash are, in my opinion, better for UI development than HTML/CSS (text). This also leads to the conclusion that unless someone invents a better way of representing programming logic than using text - like the equivalent of inventing a calendar to display a schedule, if we had never invented a calendar before - visual-only programming is inherently limited.

Tools that keep UI in visual-land and programming logic in text-land - and don't try to mix the two - work great.

Part of the problem might just be that you are ultimately comparing it to using text for programming and that text just works extremely well for the vast majority of programming.

Think ancient Egyptian hieroglyphs or Chineese pictograms. They seem like a good idea until you need to describe specific nuances. There is a reason alphabet won.

That's how Interface Builder on macOS/iOS works, especially in concert with Key Value Coding and Key Value Binding. It's cool stuff, check it out.

As a fellow mobile developer, I highly prefer Android's non-visual option with human-readable and -editable XML. I always find myself editing layouts textually and then just using the visual editor to preview how things look.

Interface Builder was great for me at first, when learning iOS-specific controls and how everything worked, but as I've gotten better I've found it's always more of a hindrance. Things really start to break down when you need to move fully laid-out controls to a new container, for example, whereas Android's XML layouts let me do that by deleting/changing a few lines of text.

One thing that nobody seems to notice is that big thing in front of you with the tons of buttons: the keyboard. The main input device we use to enter stuff in our computers are based on typewriters, so everything more advanced than clicking widgets with the mouse revolves around typing stuff. Most visual languages feel like you are fighting with their code editors because the editors are not following the typewriter model that the main input device of our computers use.

Perhaps this will change in the future with other input devices, touch devices like tables could provide for a different environment - currently programming with a tablet feels slugggish but this is because everything about programming revolves around typing. Touch is different. But it needs to be designed from the ground up for the interface you work with.

Whatever that is, i don't think text is the only way to go. I believe our keyboards have shaped the way we think about programming - and interacting with computers in general - and if we are to explore other ways to program, we need to also explore other ways to communicate with the computer.

Way back in a previous life I made a living building a visual programming tool and language to specify simulations. After many years of work it became clear that visual programming works very well in industries that already have a well established visual paradigm. We had great success with simulating hydraulic systems by letting folks specify them with standard piping and instrumentation diagrams. The cooling system simulation for the space station simulator ended up being built that way with our system, as were many simulations of nuclear power plant cooling systems. However, after you do P&i diagrams and electronic schematics you very quickly run out of well established visual paradigms. Uml is just not useful in the way that a piping diagram is.

As to text, a colleague once described the best feature of a text specification of code was "visual rhyme". You know, when you can look at a block of code and feel how it ebbs and flows. That's from the old days, I don't know if functional code rhymes like Fortean did.

SQL Server Integration Services (SSIS) does a good job of this I think. SSIS is made for manipulating data flows. The flow of data happens to be very well represented in many cases by visual network flow diagrams. Most of the common patterns of data flow are implemented as drag and drop widgets.

That said, as necessary, you can do pretty much anything in that environment as code if you want. The best thing about the language, though, is that /if/ you go down the path of coding something out instead of using a common pattern, it forces you to ask "Why am I doing something out of the norm here?" Sometimes you have a valid reason. Sometimes you don't and that question course corrects you back into a maintainable architecture. It's a very niche language (even mostly vendor locked down), but I think it's a great example of a visual programming environment that's mostly great to work in.

Check out node-red[1], a visual language for creating node apps. It's a good demonstration of the potential of visual-based languages (but it's also far from that potential).

[1] https://www.youtube.com/watch?v=f5o4tIz2Zzc

WYSIWYG fails for two reasons:

- Programming errors most of the time break everything. You probably know how bitchy Word can be. Errors will be frequent und finding them when you don't really know what happens under the hood will be very difficult.

- WYSIWYG does not abstract very well. The big power of programming however is, that you can build your own tools from a minimal set of building blocks. Visual representation for self-build tools and routines will not be pretty out of the box.

---

Those visual languages that do have a big userbase do the following:

- no WYSIWYG, just "nodes". The visual parts of the nodes:

- - inputs and outputs are often connected with "wires"

- - simple control UI like a knob or a button can be shown directly

- they allow wrapping a bunch of wired mess in to a box with simple in- and outputs again

- some allow to jump back in to code where necessary

- in general nice for small art-projects

- difficult or at least in-flexible for complex stuff

- often a lot more copy+pasting

---

>Then you interact with their API: this table pulls in data from this SQL statement; this button executes this HTTP query; this dropdown publishes this value, which this SQL statement uses via string interpolation. These "widgets" publish their data, which can be used by other widgets (w1.value), and inside SQL queries. The tables can pull in data from SQL queries. You can use string interpolation inside SQL queries, so dropdowns can influence the query being run. So selecting a drop down value would cause the query to change, which would automatically propagate changes to the table.

This is very similar to Xcode Cocoa/CoreData bindings, but these are too restricted to create anything nontrivial. IME I always had to fallback to delegate and data source methods in source code.

> Why does visual programming suck?

It doesn't? Shameless plug, I build a visual programming tool for UI design (https://www.ux-app.com/static/learn-actions-and-events.html).

It's great for a huge subset of people for whom coding is tangential to their job. Product managers, designers etc.

It's not ideal for a programmer because it's not as dense as the textual representation and once you've developed that muscle memory, typing your code is a lot quicker, but increasingly, there are a lot more people coding now who are not career programmers. For these people visual programming is great since it's less intimidating and reduces the possibility of syntax errors.

The type of visual programming you're suggesting seems to exist in an extreme form as the apps we use to read and write data, like Facebook, Trello, etc. On the other end of the spectrum we have the very technical savvy that want more direct control of what's going on. There's a lot of middle ground that's been covered, from both ends of the stack. I don't think the idea of visualizing things to be more material is flawed, but surely you could overdo it or apply the wrong model at the wrong level of abstraction.

All of our programming is visual.

We rely on a set of visual units of some manageable size (an alphabet of characters), and then we compose them sequentially into strings (visual units of a more general alphabet)

Stringing units together makes sense based on the way we experience a limited bandwidth of signal over time.

While listening, signal is presented one unit of the sequence at a time. When reading, we are presented with a whole page of characters at a time. The surrounding area is displayed, and we can move the focus around at will.

This leads to the development of the visualization models/techniques that we progress from simple features like spaces between words; parentheses/curly braces to group words; indentation of blocks to group/subordinate them, forming trees.

The idea of the unit/object/thing, composed of unit/object/things is thematic.

Files are units of text that are read/written one at a time. Eventually you stop directly typing, and you enter a different level of abstraction, where the units are displayed and manipulated with a different set of controls.

I think this is the point where the flowchart (directed graph) model fits in with the text focused model.

In Sublime, I like having the directory outline view and the zoomed-out scrollbar view with whatever single page of text I'm viewing. I think it'd be cool if these three were integrated, so that I open and close the files in a flowchart in the plane, where edges/arrows represent organization and dependencies. I'd like to import a module by dragging an arrow from one node to another

But, isn't text just abstract signs and symbols? Why can't we, programmers, make new or better symbols? Why can't the language be augmented to incorporate concepts of computing?

I never desired visual programming until recently. I never had the need for it. Plus, I always found the existing visual languages and tools to be clunky.

But, lately, I often find myself with just my tablet and stylist. And, during these times, I want to 'sketch' out programs. I have some ideas on how this would work in practice. There would have to be detection of gestures, and instant feedback that provides visualization of the changes.

Your code, as text, serializes structures that change in all sorts of beautiful ways over time. It's hard to represent that in two dimensions. We can write about it though, in a language the computer can interpret/deserialize.

Building a good visual programming environment I'd imagine would require either having a way to visualize and move around in higher dimensions, or dumbing down and limiting what can be done to fit neatly in two dimensions.

I like Reaktor Blocks a lot, by native instruments. They took the second tactic, and the result is beautiful (but limited). For what you use it for, it works great.

Programming is inherently difficult. We can do a lot in tooling and increasingly higher level languages to help make a lot of the details a lot easier, but a the core, it remains abstract symbolic manipulation.

It seems to me (and I have by no means done an extensive survey of the field, so YMMV) that efforts in visual programming suffers from mis-diagnosing that the difficult part of programming is the act of writing the code (indeed, the empty editor is intimidating and code in any language looks impenetrable to outsiders), rather than understanding what code to write in the first place.

VP isn't a bad idea by itself, but its implementations usually have the same problems:

1. It's hard to make a scheme in editor without spaghettifying it. (Partially mitigated by blocks incapsulating parts of scheme in themself, some discipline is required)

2. Usually VP makes it hard to work with complex data structures. The only solution is to basically write the same code as text losing all VP in the process.

http://hiasm.com/ : Very usable constructor for small programs, the only problem with it is that website is in russian.

Because language and abstraction goes hand in hand. The great leap 50,000 to 100,000 years ago did something to our brains that brought both our use of language and capability of our mind to another level.

Because it's not optimal for programming. It's however great for scripting.

If you are a good enough programmer you want the power of ex regular expressions and you can solve almost any problem out there.

If you aren't you aren't much better off as you still need to learn how to program.

As a scripting language however it can be great because you can be much more opinionated about what should be prioritized.

What would be much more interesting and useful IMO was if normal languages created a visual environment akin to the visual programming languages we see sometimes.

I don't know much about how other people think but I'm fairly certain that I favor auditory mode. When I write code I generally am in auditory mode and then expressing logic becomes fairly easy with text. Further, I believe abstractions like classes and objects, etc. tend to favor text/auditory expression.

Visual programming to me becomes very scattered and fragmented. Bits and pieces of logic seem to end up all over the place and become hard to keep a cohesive functionality together.

tooling tends to suck, input ends up being slower once you know how to do it. (Argument to be made that visual is faster on a tablet with no keyboard. Okay).

You still have to type to name things.

Density of stuff on the screen is lower.

Not sure about readability (you can find readability advantages for any format if you ask the right person). I would bet math expressions with many terms are clearer in a visual format like tex.

Visual elides the connection between functions and files/modules, potentially making it hard to scale projects.

This was an active area of study in the 80s, with many journal publications. The goal was to make programming intuitive.

Then one day someone published an article explaining that we need words to represent variables, doing it visually runs out of abstractions too quickly to do anything useful.

There were no journal publications after.

It's been about 14 years since I last looked at these publications and my memory is foggy. The point is that any useful program is too complicated to represent with pictures.

In dataflow programming, and some variations of functional programming, variables are pretty rare.

But I'd be interested in reading the study you refer, do you know how to find it?

In dataflow programming, variables are unnecessary.

We call visual programming that doesn't suck a 'graphical user interface'.

Look at a daw like Logic or Ableton for example -- it's essentially visual programming.

Filter Forge founder here, now working on a turing-complete functional visual programming language inspired mostly by Haskell: https://imgur.com/a/HkSgJ

Here's what I currently think on visual programming languages:

1. I see no place for imperative and non-pure functional languages in visual programming.

2. Pure functional languages can absolutely be implemented with visual syntax. This is just another form of representing a program. Instead of taking text as an input, the compiler / interpreter takes the node-based representation, which may or may not directly correspond to the AST. Here's an example of the same toy program in Haskell, Racket and our visual language: https://i.imgur.com/dsNJYyx.png

3. Visual languages shine on higher levels of abstraction. They work great in Filter Forge and similar systems, e.g. material / shader editors in 3D packages or game engines. However, I've never seen a visual language that remains usable and understandable on a lower level of abstraction.

4. Our civilization has settled on text as the lowest common denominator format of information exchange. We have accumulated an enormous amount of tools, infrastructure and software ecosystems built around text. Git, Slack, WWW, Copy / Paste, Ctrl/CMD + F, diff tools, IDEs, regexps, Unix pipes etc etc etc. A visual language will have to reinvent all these wheels.

5. Visual programs have no equivalent of line numbers for addressability. You can unambiguously refer to a location in a text program with just two numbers, line & column, but how do you do that in a visual program? "Uh, it's the red rectangular block with the blue output, to the left of the square green one. The connection you need is the second from the top."?

6. I see no way of quickly commenting out / deactivating a section of a visual program.

7. When you write a text program, you are compelled to give descriptive names to things like functions and variables. A visual program doesn't force you to do that. For example, in our language you connect function outputs to function inputs with graphical arrows, and named binds aren't necessary. In Haskell, however, if you don't want your program to look as a huge chain of function applications, you bind results of certain function to named 'variables', and the names tend to be human-friendly (well, at least in monomorphic functions). To sum up, visual languages lack built-in annotatability and don't compel programmers to annotate programs. Annotation feels extraneous, tacked-on.

8. I/O is hard. Sure, it can be done the Haskell way, but without some kind of monadic sugar (somehow adapted to a visual language) it would be absolutely unpalatable. Also, this would only work on visual languages that are capable of expressing a monad.

Filter Forge is great. I was thinking on doing something similar and will definitely take inspiration on FF. What will be the use of your next turing complete VPL ?

It's intended for a Filter Forge sequel. The main pillars will be Turing-completeness and GPU support (implemented via a separate node-based DSL).

Great ! That's exactly what I want to do, a FF for GLSL :D

Wow, I really like it!

Although, I think your ui is too verbose. Main indicator, is, well, the size of "blocks of data", where text takes 4 times less space than visual representation. But there is also an information overload: text representation of `map` call has no indication of what kind of arguments it expects, it is implied by order of args and basically is just defined by language. I guess, visual analog of this code should follow the same trope: it should strip type annotations (which probably should be available VP IDE with a shortcut), and imply meaning of args (maybe with colors?)

Agreed on verbosity. That's one of the reasons why I think that visual languages don't work well on lower levels of abstraction.

(BTW, texts in the Map call are argument names, and colors indicate their type.)

Google's app maker talks a good game. Haven't been able to try it yet. Supposedly, you are able to script pretty freely on both the front and back end for situations where the canned functionality isn't suitable. https://developers.google.com/appmaker/

Every time a VPL (Visual Programming Language) has gone against a TPL (text programming language) , VPL not only has come out a winner it has massacred its opponent.

There are 3 areas that I am aware of that VPLs have appeared a) modular audio synthesis b) 3d graphics c) game programming

(a) has been the first with origins back to 1995 , there are like a ton of programming apis for building modular synthesizers , there even programming languages that specialize in this that have existed for decades now like CSOUND and Supercolider but Reaktor and Max/MSP have literally mopolised the market. Reaktor itself has been such a massive success that made Native Instrument as the dominant company on audio software.

(b) 3d graphics is no secret that is probably the richest software industry with the generation of the well knows CGI for films and games its a goldmine for developers and has been dominated by VPLs. You are not aware of this most likely because we never refer to them as VPL but rather as procedural material, textures, animation etc. Not using a VPL for todays standards is considered insane even if you try to do it via a very lovable language like Python. Text languages like POV RAY that specialize in 3d graphics have died a miserable death. 3d is where TPLs go to die and are used strictly for generating the software alone and thats about it.

(c) computer games there VPLs are something new but Unreal has definitely changed that with Blueprints. Probably the best and most powerful VPL. If you have played any kind of AAA game chances are that the game you played uses Unreal which in turn makes heavy usage of Blueprints and C++ code. The rule of thumb among Unreal developers is "Do it with Blueprint and if you cannot avoid it do it with C++". Unity has smelled the roses and has implement its own VPL as well very similar to Blueprints.

So if VPLs are so amazing why we do not see more of them ? They are actually very hard to make. If an implementation of lisp , a basic one, needs a few hundreds lines of code a VPL needs thousands and that imposes a high barrier for language designers. Hence why we have thousands of TPL and only a handful of VPLs.

The majority of them are crappy obviously because that is how quality works , you have to have high quantity for high quality.

But a high quality VPL will kick high quality TPL ass any day and this is why VPLs are massively more popular than TPLs for the same task.

One that does not work as well as it looks and it depends on drag and drop is a crappy VPL . Yes a crappy VPL will suck, not as much as a crappy TPL (see C++ , Javascript etc) but it will suck none the less.

you can also read my quora answer here for more details why VPLs dominate so easily and why they will replace TPLs in the not so distant future to a very large extend

https://www.quora.com/Why-did-graphical-programming-language...

You're cherrypicking examples and ignoring counter examples. Yes VPL's in specific niche areas have been useful and largely beaten there competitors. But I know of no general purpose VPL that had succeeded in beating a text only competitor. Text is superior for general purpose languages because it sits in the sweet spot for information density vs ease of use.

Yes and Yes. VPLs are indeed specific purpose and its because of information density.

Contrary to what you may believe a VPL is not the evolution of TPL , its actually the evolution of GUI. Reaktor for example is an evolution of VSTs , synthesiser and audio effect plugins that are GUIs. VPLs in 3d are an evolution of the gui properties for material , textures etc (they are not even referred to as VPL). Even in Unreal even though Blueprint has replaced Unrealscript , its an extension of the editor GUI.

VPLs are successful specifically because the information density is so large that a linear GUI is not enough.

The success of TPL is legacy, its just has a lot of libraries. Mainly because its easier to make and has been around a lot longer.

However to be sincere here when we talk about TPLs we also cherrypick , we do not think of Assembly, Pascal, Cobol and Fortran but rather C/C++ , Java, C#, ruby, Python and Javascript. The truth is that 99.9% of TPLs have failed to gain approval of not just the users but also the coders too.

But I am not denying the fact that a TPL will target mostly pro coders than users and that VPL will target mostly users than coders. They are just different kinds of beasts making the comparison more than challenging.

Poor form of the people who downvoted parent.

I don't agree with parent but they are contributing to the conversation.

If you disagree why don't you provide some counter-points instead.

Here, let me start.

> Every time a VPL (Visual Programming Language) has gone against a TPL (text programming language) , VPL not only has come out a winner it has massacred its opponent.

That's an unsubstantiated claim. Point to some sources please.

> There are 3 areas that I am aware of that VPLs have appeared a) modular audio synthesis b) 3d graphics c) game programming

These are not the only ones [1], and VPL first appeared earlier than 1995. For example, the first version of LabView was released in 1986 [2] and certainly LabView was not the first one either.

> 3d is where TPLs go to die and are used strictly for generating the software alone and thats about it.

Even if this is true in the case of VFX (which I don't know that it is but I don't know that it's not either), it certainly is not true in video game development, which is very much an area of 3d graphics.

> computer games there VPLs are something new but Unreal has definitely changed that with Blueprints

Blueprints is awesome but you could not TODAY make an arbitrary game implemented using ONLY blueprints and have the amount of control and freedom you have if you use Unreal or another game engine with C++ or another TPL. Thus falls your initial claim, which (quoted above also) was

> Every time a VPL (Visual Programming Language) has gone against a TPL (text programming language) , VPL not only has come out a winner it has massacred its opponent.

And I bet that others can point to a wide range of areas where an existing VPL will come up short.

Some people love LabView. I certainly don't. Some prefer it and for them it's a thousand times better than using a TPL but for me 99% of the time I would go with a TPL over LabView.

[1]: https://en.wikipedia.org/wiki/Visual_programming_language

[2]: https://en.wikipedia.org/wiki/LabVIEW

I did not downvote the parent , mainly because I rarely downvote posts, must be a facebook habbit :D

"That's an unsubstantiated claim. Point to some sources please."

Reaktor and Max/MSP vs CSOUND and Supercollider , Blender python vs Material/Texture/Animation/Compositor Nodes, Blueprints vs Unrealscripts are all examples I mentioned already. Why they are not enough ?

"These are not the only ones [1], and VPL first appeared earlier than 1995. For example, the first version of LabView was released in 1986 [2] and certainly LabView was not the first one either."

This is why I said "that I am aware of" I am not an expert on all things just a few areas.

"Even if this is true in the case of VFX (which I don't know that it is but I don't know that it's not either), it certainly is not true in video game development, which is very much an area of 3d graphics."

I already mentioned Unreal and Blueprints. But yes what I meant was on the fact of actually a VPL competing with a TPL. Obviously when no VPL exists you have no competition. In the case of 3d apps efforts have been placed into using scripting languages but the users have picked VPLs instead as much easier to use and learn.

"Blueprints is awesome but you could not TODAY make an arbitrary game implemented using ONLY blueprints and have the amount of control and freedom you have if you use Unreal or another game engine with C++ or another TPL. Thus falls your initial claim, which (quoted above also) was"

What you say its true but its not because VPL is inferior its because its not fully supported. Unreal started as a C++ game engine so of course the support will be more extensive than for Blueprints . Blueprint have been around for few years and its already growing faster than the C++ api. It was only lately that Epic added the ability to compile a Blueprint to C++ code. C++ API also make it easy to wrap any C++ code and expose it to Blueprints. So if you want to make your entire game in Blueprint its not possible but also easy.

"Some people love LabView. I certainly don't. Some prefer it and for them it's a thousand times better than using a TPL but for me 99% of the time I would go with a TPL over LabView."

VPLs are made for users not coders, its their ease of use / idiot proof mentality that have made them succesful. So yes if you are a coder in most cases you will prefer and should prefer a TPL , especially a popular one.

But again I repeat in the vast majority of case you will be extremely lucky to find a VPL to even consider moving to. VPLs are extremely rare.

If Blueprints have really become the default in Unreal, why are they so barely talked about? Searching for "unreal blueprints" on Google returns less than two full pages. Do you work in the industry?

If I go to the last page of a Google search for unreal blueprints it says that it showed me 120 results. 12 pages.

If I go to the last page of a Google search for C++, it says that it showed me 180 results. 18 pages.

It used to be that Google would tell you an approximation of how many matches they actually had in total for a search of their index but they no longer do anywhere that I am aware of. If there were only two pages of results I might have agreed but when there are 12 pages it will show to me I don't think there is any useful conclusion to be made about the actual total amount of sites that contain these keywords.

However, there is another site that still gives us an approximate count of results. YouTube.

Searching for C++ on YouTube I am told that there are "about 1,010,000 results".

Searching for Unreal Blueprints on YouTube I am told that there are "about 57,800 results".

Searching for UDK Blueprints on YouTube gives "about 18,600 results".

There will be some amount of overlap between the latter two.

I don't know how many of those videos matched only blueprints and was included though.

But let's do another couple of searches.

UDK C++ on YouTube gives "about 11,000 results".

Unreal C++ on YouTube gives "about 60,900 results".

Well, it seems that the words UDK and Unreal are significant when searching on YouTube, since otherwise both of those searches should have given about a million results like the search for just C++ alone did.

So comparing "Unreal C++" and "Unreal Blueprints" on YouTube actually gives a surprisingly high amount of results about blueprints compared to C++ for Unreal, certainly a lot higher than I would have expected. I would have expected at least 25 times as many for C++ but it seems to be almost the same amount of videos for each of those.

Of course one might wonder if even though the words Unreal and UDK might have matched correctly, did the words blueprints and C++? Or do the results include a lot of videos that are about neither blueprints nor C++?

Additionally some amount of videos will have both these words but will not be about the Unreal Development Kit. Instead one video might be something like "Unreal! Secret blueprints confirming conspiracy theory so-and-so confirmed!" You get the idea.

Regarding Google's search, fair enough, but I did actually only get two pages (though I have it configured for more results per page).

YouTube's search is wonky, after a few pages it starts to omit words. I got a video for a blueprint of a building (no "unreal" mentioned anywhere).

Welcome to the failures of Google search engine. Google is great but not so great when you try to make the search a bit smarter.

I am about to "work" in the industry I have a Blender plugin and Unreal editor plugin in the works , none of the use a VPL by the way because simply what I am trying to do is too technical for VPLs that target users and not coders. But of course I will be offering VPL support because users love VPLs.

I never said Blueprints are the default in Unreal , C++ is still king. However if you want proof , you can take a visit at the official Youtube channel of Unreal by Epic and see for yourself the rise of Blueprints. Most tutorials nowdays for making games with Unreal target Blueprints.

Well, you said the rule of thumb was to use Blueprints unless you can't avoid C++, that implies a default.

I do agree that Google's not evidence of anything, I just use it as a first approximation when I have no better sources. That said, the marketing channel of a company trying to push its tech doesn't really satisfy me either.

If that is what you mean by default then yes Blueprints are generally preferred because they are accessible both by Artists and experienced developers. C++ is generally preferred for performance reasons because Blueprints being an interpreted language is around 10 times slower ( these are estimates by Epic the creator of Unreal).

Of course how much Blueprints will be used will depend on the game. There are games made entirely with Blueprints.

Also Epic recommends the use of Blueprints as first choice again because of ease of use. They have invested a great deal to it after all. It even has a debugger with break points , watching values and visualization of execution flow.

Your example are great but it seems theses VPL succeeded because the data manipulated was constant (sound OR texture OR 3D object) in a whole project

in 3d and Audio the data is anything BUT constant. You will have to be more specific than that because I do not understand what you mean. Anything can be turned off and on.

I meant the input and output of nodes. The main thing a VST oriented VPL does is manipulating audio samples so the graph is easier to use and reason about than a generic VPL (domain specific VPL seem to thrive, I wouldn't say the same thing for generic VPLs)

No that would have been very limited. It's very important to turn on and off execution flow and of course branch it. That applies both in graphics and in sound. Why you got that impression ?

Audio VPLs are far more powerful than what you describe.

Example ? (And I didn't say one couldn't branch or stop the execution flow)

Two materials combined to one over a 3d surface where instead of mixing in certain areas you can have one or the other depending on a texture transparency acting as a mask. Or a gradient color customized to act as a bit mask with one and off values in the colors of black and white. Non continuous data in sound can be midi or any form of modulation source like a square wave that can also act as a bit and tons of other methods.

Ok so theses VPL have 2 primitives : 2D texture (a bit mask is a 2D texture) and 3D object / sound samples and MIDI stream. Can you generate an image from sound samples ? Can you create 3D objects particles from MIDI stream ?

I use blender, Blender has multiple VPLs that each specialize in one area. Texture, Material and Compositor. Blender VPLs are fully upgradable with Python code that allows the creation of new nodes . This allowed developers to create new VPLs using animation and modeling nodes. So even though Blender VPLs are not meant to be used as generic languages however there is already the API in place to allow this essentially exposing the entire Python scripting interface as nodes.

Reaktor I do not know if it supports 3d physics particles , but you could do what you ask for from blender. Blender supports midi for controlling animation.

Blueprints on the other hand are a generic VPL and can do anything you describe. It's also the best choice because Unreal excels at real time graphics.

Cool ! Thanks for the info, I have to check this out, looks interesting :)

Thanks, very informative. If you implement a visual language, you also have to implement the means to input your code, i.e. you have to implement not just a language, but also an IDE.

The other thing to note is that hardware had influenced programming languages, which in turn have influenced hardware, in the direction of SISD on CPUs and later SIMD on GPUs. But textual languages are poorly suited, and visual languages well-suited, for representing dataflow. In the 1970s and 1980s, there was active research on dataflow hardware as well as work on (mostly textual) dataflow languages. Now, CPU speeds are no longer improving exponentially as they were in the past, so dataflow should be re-examined.

I've been working on my own visual dataflow language (http://web.onetel.com/~hibou/fmj/FMJ.html). I haven't updated the pages for some time, but big changes are under way.

IfLoop is a visual programming language which is like text programming. It gets both the benefits of the two styles. It is very good for new programmers to learn programming. http://www.tapirgames.com/blog/ifloop-web-demo

Imagine trying to do math without numbers. That's why visual mathematics sucks. Numbers are needed to count. We do it with our fingers when we're little, and we're limited to 10. Then we take off our socks to get to 20, but that's it.

Imagine trying to reason without words. That's why visual programming sucks. Words are the smallest units of abstraction that we can reason with. So when a visual programming language provides us with glyphs, we are already limited in our vocabulary and in what we can reason with. It becomes excruciating.

What we need is not a visual programming language, but more tools to visualize the programs that we have. Ultimately, all visuals need to fall back to literal code a parser can reason with, so there is no escaping code. In fact, there are already layers going down from the higher level languages that we use now. A practical graphical language would be one built on a higher level that adds a layer to the stack, and one which we can use or skip based on our problem at hand.

Also it is because graphics too easily limit abstractions that most graphical languages are best being domain specific. A WYSIWYG editor is for the web, the form designer in Delphi was for application layouts, and Scratch is for games.

My company sells a product that just does integration between things, and the main way to program it is a visual language that does impedance-matching between XML data structures, loops, conditionals, etc. For that it works really well, and is superfast. But I can't imagine using it for general-purpose programming.

Here's an article summarizing the major problems (real and perceived) of visual languages https://hackernoon.com/visual-programming-is-unbelievable-he...

Why is pictionary a fun game to play? Because drawing pictures to express ideas is harder than using a language :)

Funny enough, all examples of successful visual tools in this conversation are domain specific (yes, even UI builders are domain specific). The ones mentioned in the 'it's unusable' category are the ones trying to do everything.

That should answer the OPs question better than any specific answer...

It is unproductive. You waste time managing a 2D representation for little benefit. It's hard to do meaningful diffs and merging needed for revision control.

You can program FPGAs in 2D with schematic entry. It is a huge time sink for anything non-trivial which is why few bother with it these days.

> It's hard to do meaningful diffs and merging needed for revision control.

That's because existing revision control tools are specifically designed for textual languages, therefore unsuited to graphical ones. There's no reason why graphical equivalents can't be implemented.

Your example is still too closely tied to traditional textual methods of programming: "query", "statement", "string interpolation", etc. are not likely terms to be found in whatever visual programming paradigm eventually becomes successful.

A picture might be worth a thousand words, but if I give you a thousand word essay there's no way you'll find any number of pictures that fully express it.

Text is hugely expressive and can be precise and unambiguous in ways visual information struggles with.

Klik & Play had a curious mix of questionnaire and spread sheet programming when making games. I pushed it as far as a 14 year old could. I think the density of information and need to control abstractions are where visual tools break down.

You are describing asp.net forms in visual studio circa 10 years ago. It was not a bad experience to use the UI tools and "code-behind", but then MVC and Ajax came to the scene, and it fell behind I guess.

It has taken off in certain niches. When I was an undergrad all of the physics labs were hella into LabView for developing instrumentation software. Most shader development tools are essentially visual programming.

Text doesn't interfere with a programmer's imagination or ability to visualize the system. Visual programming suppresses imagination and imposes a very limited and impoverished visualization.

On the other hand it can be really tricky to reconstruct an understanding of a non-trivial system which someone else made from the textual code. With visual diagrams overall architecture can be conveyed quickly and relatively concisely.

i think most people that have an opinion on visual programming have not actually ever used a visual programming language in earnest.

i have worked in labview very seriously over the past 4-5 years. at this point, i am essentially an expert in the language. i have done object-oriented programming (OOP), functional programming, and also programmed using an actor framework. all of that is in addition to obviously adopting the dataflow-first paradigm. my projects have not been simple applications. they have been large applications, with many different components and non-trivial user interfaces.

it's amazing how such a simple and small base language can be so powerful due to the dataflow nature. my programs are almost always essentially bug free when i am done. this is due to how labview works, which is rather advanced. as i program labview, i work on VIs. these are small blocks of code that perform a certain task. oftentimes, these are purely dataflow functions. in other words, they take in information and pass information out without modifying things via references. since every VI has dynamic type propogation and compilation as you work on it, you get immediate feedback on wehther or not your types match up and whether your VI runs. once your VI successfully compiles (by the way, you never kick off the compile process as it's automatic), you can run it. so i build up systems out of basic dataflow processors, almost all of which i am able to run freely with inputs to see how they interact. out of those that i can't, mainly due to needing a "mock object" to test, it still makes it easy to reason about to due simply having to understand how the data flows through your code. if something's not available, it's pretty easy to trace it back to where it should have come from. all of this also makes labview easy to debug. data is just flowing through your program. just navigate to where you want to view your data and place a probe on a wire to see what's happening at that point in your program.

all of this is quite advanced. this type of interactive writing of code, running, and debugging doesn't really exist in other languages. then you have the ability to easily create modern looking user interfaces (my interfaces are almost untraceable back to labview based upon their look) and program desktop, real-time OS, and FPGA applications all with the same language.

and i haven't even talked about how labview is concurrent by default. want two while loops running concurrently? just draw two while loops on the screen, and you're done. multithreading is so easy in labview that you don't even have to think about it. it's inherent to the language, and labview takes advantage of multiple cores by default.

many of these features are discussed ad nauseum in reference to other languages: type propagation, REPLs, dataflow, concurrent/multithreaded programming, etc. these are things people throw out in praise of a language, and yet labview has all of them.

finally, i will mention that labview does struggle with some things, but it is not due to the visual programming environment being inadequate in most cases. most of the time it is because the language is manufactured by a company, which has finite resources in what they can do to the language. there are plenty of problems with labview, but they are all solvable. they just need attention from national instruments and the research community at large.

it's just that visual programming is different and necessitates different approaches. people somehow forget that it took them years to understand how to program computers using text. it isn't that text is THE way to program computers, but people certainly seem to think that. i am convinced a hybrid approach is appropriate. i myself am slowly designing a language and environment along these lines.

> i think most people that have an opinion on visual programming have not actually ever used a visual programming language in earnest.

Yes. I have used a dataflow language for 3d modeling - Grasshopper - for 5-6 years. I can't agree enough with you about the joy of interactive writing, running, debugging, etc. Grasshopper also happens to have blocks/nodes where Python/C# code can be inserted, which is the best of both worlds.

As someone who has never used labview - do you have any links to examples of your work, or well-structured labview code?

I don't encounter people, either IRL or online, with significant experience in LabVIEW, or other visual dataflow languages. I wonder if you could give me feedback on my own visual dataflow language, described in http://web.onetel.com/~hibou/fmj/FMJ.html

Though I haven't updated the web pages, the language is still evolving, and not yet ready for release.

Look at this https://scratch.mit.edu/

The idea is similar...

I'm really curious if one could compile this directly into some form of AST...

It doesn't. Try Outsystems: https://www.outsystems.com/learn/

https://developers.google.com/appmaker/

What you're describing is Developer/2000 or more generally RAD which was all the rage in the mid-90s.

At the end of the day, programming is symbol manulation, and nothing beats representing symbols like text.

No mainstream programming language even supports symbols as a data type. They process numbers and strings of characters.

Symbol manipulation is done well only by Lisp and its variants.

Javascript (with ECMAScript 6) supports symbols as a datatype.

1. Try flowgrid.org

2. File an issue describing why it sucks

3. I'll fix it O:)

Ok, realistically, I guess I agree with the top comment

What would the United States Code look like as some painting or interpretive dance?

(may be) google app-maker (+ more AI) - something to watch out for.

Is there a VPL that's been developed without using any text?

Have you ever try Drupal Views?

please try Bubble.is

Honestly, I think most languages have shitty syntaxes (syntaxi?) and that the issue is not about visual programming.

The other problem is that everybody who has a big idea for visual programming has the same, actually shitty, idea.

Drag-n-drop blocks that represent logic.

UML is not a thing anymore because it's useless and wastes time.

Visual programming is great for describing high-level logic, but not low-level semantics.

Try and describe your last trip to the grocery store with just emojis. Maybe you might get creative and come up with a paper for your list, some fruits and stuff like that, but how could you guarantee that the user on the other end of your description gets the exact same idea? You can't.

With words, I can write my comment how I want, and I'm more or less guaranteed that people will read it the way that I intended them to, because it wouldn't make any sense any other way.

I avoid Text as much as possible. I find that Text is good for command line stuff. For building a good looking app, a visual tool mandatory. Of course, if I want a quick and dirty clunking looking thing, Text is ok.

If you want an answer to your question, you yourself should know what you're asking. I honestly don't know if you're asking a rhetorical question, complaining about the reality, or asking for feedback on your idea. You are talking about multiple things at once here:

1. asking why visual programming sucks;

2. Talking about your experience with Alan Kay;

3. talking about your idea of an ideal visual programming method.

Thanks for the feedback! I'm asking about 1. and 3.

1. I have a theory on why VP sucks -- is it right? What do other people think?

3. Yes -- I'm looking for feedback, but in the context of why visual programming is no good. Also because of below:

2. I brought that up because visual programming is big, which is a good point that Alan Kay made. So I provided an example, or idea that I had. Is visual programming bad in this case? Why, or why not?

Luckily we're on an Internet forum where people can take their time to read the question and then have even more time to figure out how to respond, what to respond to, and what to ignore.

It's entirely possible to have complex conversations here where more than a single topic is being discussed at one time. This isn't Stack Exchange.

Reply on news.ycombinator.com