Visual Programming Is Unbelievable (2015)
outsystems.com
outsystems.com
Most PLC programming is done using Ladder Logic [2] and various visual tools tools for creating and editing Ladder Logic. Here's a screenshot of one such environment, RSLogix: https://i.ytimg.com/vi/4yEj065J1q4/maxresdefault.jpg
It's an interesting data point because A) it's a highly technical field where accuracy (and verification) matters B) almost everybody who does PLC programming spends all their time in a visual programming environment.
[1] https://en.wikipedia.org/wiki/Programmable_logic_controller
I understand the appeal of having visual flow and feedback for your code but I see no reason why the building of code should be exclusively visual. I see great improvements that can be made by programatically generating control structures. But, well, the clients pay for it, the companies get wads of cash for sending engineers in the field so I guess it all works out in the end.
As a side note, I suspects that if someone good puts his mind to it they could generate the code out of the layout of the plant and the different control mechanisms involved. With proper constraints, I suspects the system would be sufficiently defined without laying one stupid line (like, an actual line, from point A to point B) of code.
And I agree, the "language" forces you to repeat yourself. I believe that over the years there's been attempts at alternatives but none of them caught on. The only new things you see if faster hardware (PLC), new buses (took forever for ethernet to catch on) and new classes of IO / devices to control.
The controls / industrial automation industry has different set of constraints & values. It's a very conservative industry. Double so if what you're programming is safety and error proofing.
So the industry values tools that consistent, maintainable (by current / future staff). There's many cases where you might be supporting an installation with a mixture of both new and 20 year old hardware.
It is just a different world.
I dabbled in PureData a little, and while it seemed to be ok for simple things, I imagine that for anything even moderately complex, it would quickly become a nightmare.
While visual programming paradigms seem like a nice thing at first glance, they are missing out on whole ecosystems of tools developed over many decades for text-based programming. Things like debugging, version control, grep, search/replace, all the features of advanced text editors like vim or emacs, etc.
It's been long enough that I've forgotten the terminology, but I remember thoroughly enjoying the process of first building a widget, then meticulously arranging all of the little boxes, color-coding, and labeling various sections until it looked visually attractive. Then when I was satisfied, I could take that entire widget and abstract it away as a small box with only the relevant input/output connectors.
I guess it's always just stuck out in my mind as the one time I really dug into a visual language, and the aesthetic aspect of organizing my "code" was very appealing. It's been a decade since I last opened any of those old project files, but I can still picture many of the components I built with my mind's eye. In and of itself, that's got to be worth something.
I imagine that someday we'll see the rise of a programming metaphor that works as a hybrid of code and visual. Even though working with PureData was inspirational, I don't see text-based languages losing relevancy any time soon. However, I think that while text code is great for representing the flow of logic, higher-order organizational constructs (such as class hierarchies) might be better represented by visual relationships. If I give you a paper and pencil and ask you to map out interconnected relationships between a bunch of entities, you're probably going to start drawing boxes and lines-- and end up with something that looks a lot like PureData.
Especially if you consider the potential of future iterations on today's VR/AR concepts, I wouldn't be too surprised if programming languages eventually looked like color-coded interconnected boxes of text code that can be rearranged in space. It wouldn't even be that far-fetched to imagine animations being included to enhance a programmer's ability to visualize the flow of logic. And I don't think that such concepts should be particularly offensive to those who consider themselves to be serious programmers.
You're going to enjoy the Eve language...[1][2] :-)
[1] I helped write a system called Heavy which can translate Pure Data patches into fast C code. https://enzienaudio.com/
[2] Here's a simple example video. https://www.youtube.com/watch?v=_zwI5g3NNYw
Depending on the temperament of the "NATO.0+55 pilots" [2], the code could get pretty ahem "artistic" looking. [3]
[1] http://www.bootsquad.com/old_site/nato/nato00.html
[2] https://en.wikipedia.org/wiki/Nato.0%2B55%2B3d#NATO.0.2B55_p...
[3] https://en.wikipedia.org/wiki/Nato.0%2B55%2B3d#/media/File:N...
But yes, absolutely, for more complex tasks (and my examples would be: DSP routines such as filtering, anti-alias oscillators, etc.), it would be better to use a text language. Both PureData and Max/MSP have external objects (that you write and compile in a text based language) for this purpose... although, even these objects will never be as fully optimized as a plugin that is completely text-language based.
...From here, I definitely feel like there's something "we" missed in how these systems work, and the situations they describe. But there's also a great deal that they've missed - and are stuck in the 80s.
I think the big one is that these systems (PLC visual programming environments) don't let me make my own domain-specific languages, and that's maybe the killer app of modern code.
But there's also some interesting bits - there's something that looks like agent-based programming: There's a series of environments, and you have a function (agent) that modifies an environment. You point it at each the environments, just like a loop, and it does it's thing, like a weird cross between functional and state-based, that works because it's all a form of stigmergy.
This is a new word for me—thanks :-)
I usually think of it as "communication through modification of the environment".
And agent observes their environment, makes a decision about how to modify it, modifies it. The same or a different agent then does the same. Instead of a central authority dictating tasks, each agent knows the complete plan, and compares the current state to the plan to find the next step, then does it.
https://en.wikipedia.org/wiki/Ant_colony_optimization_algori... is how I first learned about the idea, and remains a good introduction.
At the high end of the mission critical benevolent guardian mega robot spectrum, the Maeslantkering [1] is programmed in LabView [2]. I sure hope it's accurately programmed and well debugged!
[1] https://en.wikipedia.org/wiki/Maeslantkering
[2] http://www.ni.com/labview/?cid=Advertising-LabVIEW-Northern_...
[1] https://jshoer.wordpress.com/2007/08/03/why-i-hate-despise-d...
I was similarly furious with LabVIEW before I started working with people who had been using it for decades.
It's the easiest language I've used for making graphical user interfaces.
I'm very impressed by the usability of Grasshopper [1], the visual programming language in the Rhinoceros3D [2] nurbs modeling tool.
It feels like the people who designed it use it themselves regularly, which can't be said for many other visual programming languages. Plus the user interface and component libraries are built on top of and integrated with the wonderfully powerful set of graphics capabilities that Rhino supports.
Grasshopper is very popular for architecture design [2], and even jewelry [4] and 3d printing.
[1] http://www.grasshopper3d.com/
[3] http://www.grasshopper3d.com/video/video/search?q=architectu...
[4] http://www.grasshopper3d.com/video/video/search?q=jewelry
What am I missing?
I worked with someone at TomTom who worked on it in his previous job, and whose name is mentioned in the acknowledgements of that paper. I could ask him for more information about the parts he wrote in LabView, if you like. Here's a presentation about the Maeslantkering's fault tolerant software architecture. [1]
Oh -- mine too, I'm just surprised NI's mega marketing army isn't mentioning that everywhere :-).
Also, no need to bother you colleague for me, but whoa, thanks for the presentation!
Both in real-time and pre-rendered graphics, node graph-style programming languages are very much a thing. Unreal Engine's Blueprints system, Motionbuilder's nodes system, Maya's graph systems...
And they can get pretty huge and complex.
When I write code, for example a layout algorithm for a set of gui elements, I visually see the data in my head (the gui elements), then I run the algorithm and see the elements 'move' into position dependent upon their dock/anchor/margin properties (also taking into account previously docked elements positions, parent element resize delta, etc). This is the visual I need to see on screen! I need to see my real data being manipulated by my algorithms and moving from A to B. I expect with this kind of animation I could easily see when things go wrong naturally, seeing as visual processing happens with no conscious effort.
Instead visual programming thinks I want to see the textual properties of my objects in memory in fancy coloured boxes, which is not the case at all.
It didn't take very long to realise that as a solution SSIS was well suited to some simple tasks where the logical actions on screen matched those taking place on the database.
But as soon as those tasks became even slightly more complex, and this mirror was broken, the whole thing sort of fell apart. Then I was struggling to find ways to defeat the system to make it do what I wanted. It was with this realisation, i.e. that the solution was the problem, that I stopped using it.
But it's not just SSIS that suffered this way, ActiveBatch is another example and I'm sure there are plenty more.
It's not meant for professional programming, it's about empowering the end user with programmable tools.
I've used it for real projects and it actually solves the spaghetti problem. Compare this:[2] with this:[3].
[1] https://en.wikipedia.org/wiki/DRAKON
[2] http://scriptsofanotherdimension.tumblr.com/image/1386972466...
[3] https://upload.wikimedia.org/wikipedia/commons/0/0f/Dutch_cr...
You're right that until now its usage has been limited to domain-specific languages for automating applications. But with the emergence of functional programming in the mainstream and visual tools that follow that paradigm, I predict that soon there will be visual tools that allow quite general programming. (I've already seen working prototypes, and they deliver on most of the promises).
Have you seen the tools by Chris Granger (Light Table, Eve)? Although they are not as spectacular as Victor's, those show a lot of promise to become practical tools.
Chris Granger himself stopped working on LightTable because he didn't believe in the project, and if you use it, the live editing gives minimal gains in productivity. As well as being buggy, slow, distracting and potential dangerous when blindly running half complete IO operations.
As for Eve, well.. his first release was a visual programming tool that inflated 3 line sql statements to take up a whole page with, you guessed it, boxes and arrows. Now he is working on a 'card wiki' idea which effectively tries to hide all the program logic behind pictures and hyperlinks. Neither are good ideas, or once again, scale in any meaningful way.
As soon as you add encapsulation and abstraction, the system can scale; it's no different than hiding code logic behind function declarations and object schemas. If you add easy composition, like modern functional languages have, such systems could be more scalable than traditional imperative languages; code outliners like Leo follow the same structural principle, and they're great for programming.
[1] https://en.wikipedia.org/wiki/Cognitive_dimensions_of_notati...
* Ok, Wikipedia contains articles and not code, but my point is that the structure can handle lots of hierarchical content in multiple categories.
Additionally simply adding 'encapsulation and abstraction' does not equal scale. Live programming for instance requires context and re-running of code on every edit (and we could be talking millions of lines of code that need to execute), re-reading of data files, re-running of prior user input and so on. There's a reason it is only implemented for small systems (read, useless practically).
IMHO programming a large system shouldn't be all that different from writing an encyclopedia, or rather a math textbook; the same combination of informal explanation and formal demonstrations could be the shape that "visual" programs could take. Literate programming is written in English, and it provides much of the same advantages of the encyclopedia or technical book. Systems like IPython / Jupyter work in that direction, and they belong in the family of visual programming (although they use "classic" languages for the formal part, nothing stops them from also including graphical models as part of the code specification).
Programming languages should be made for human consumption, and one of the promises of visual languages is that they use secondary notation to convey the meaning of the program, so that parts that are not visible to the compiler are still relevant.
I believe that programming environments should converge towards a hybrid of programming, using the best representation for each concept - like text (to represent abstract concepts), diagrams (to represent more physical properties - time, data flows, and dependencies between modules), and raw text for the human-readable comments that explain how everything is tied together. If you build a programming environment like that, my comparison is not unfair at all.
>Live programming for instance requires context and re-running of code on every edit (and we could be talking millions of lines of code that need to execute), re-reading of data files, re-running of prior user input and so on. There's a reason it is only implemented for small systems (read, useless practically).
Modern IDEs are capable of doing a lot of that when in debug mode, I see no reason why having a language with a graphical syntax should make any difference - it just places a little more weight on the parser and graphic card.
The problem here is the "impedance mistmach" between humans and computers. Wikipedia is not a good example: it is exclusively for humans. If there is a "program" for the computer, it's minimal: "search this", "when I click here, take me to this page" (you'll notice everything more complex than that, even in Wikipedia, requires its own mini-language). This is the kind of trivial systems which are easily encoded by "visual" programming.
The problem is that real computer systems need to be understood by the computer as well as by humans. In order for something to be understood by a computer, it must be specified in formal terms; English -- natural language, actually -- is terrible for this. Once you're working in a complex system with a formal language, the limitations of these "visual" tools become evident.
Not exclusively. Bots and Wikidata have shown themselves pretty capable of doing complex automated tasks on top of the structured data contained in the encyclopedia. And the wiki platform plus templates is a very general and flexible data structure that makes it easy for humans and software to collaborate.
> In order for something to be understood by a computer, it must be specified in formal terms
This doesn't need to be that way, at least not for 100% of the system. End-user development tools like spreadsheets and Programming By Example can be used to ease the impedance mismatch without requiring the human to specify by hand every single step in a formal language, so that partially ambiguous inputs can be used to specify valid programs.
Only the task of building system software should require the full technical approach; most business applications can benefit from looser approaches - in fact being less strict can be a benefit in the requirements specification phase, just like User-centered design has shown us.
Combine that with logic solvers and deep learning, and in the future we could get some systems that can use very high-level commands to build fairly complex automated behaviors.
As for the rest, I think it's a pipe dream except for the most basic, "build this out of predetermined blocks" apps.
My point is that the structure of a wiki, where each unit of content is a separate building block, can be used to build a visual system in the same way that you can build a textual one; the basic components of storage and APIs are the same, only the thin syntax layer is different. You are no longer dependent on the single stream of raw text following a strict syntax, which is the basis from which compilers generate binary code in traditional languages; you'd be able to build programs by arranging code and data in a network within a semi-structured platform, which is one of the characteristics of visual tools.
You're right that stuff like this has been a pipe dream for at least half a century, since the Mother of All Demos; but in the very recent years I've seen a lot of the required pieces falling into place, so I believe it to be possible.
If we apply the same amount of engineering work and refinements that went form Engelbart's demo to modern desktop systems, we could have a functioning system with this paradigm; the basis are in place for it, and many of us have the vision of the benefits it shall provide.
Of course, part of building a program may require describing some components using a textual representation, where it makes the most sense; but with a wiki-like "outliner" platform, parts of the working system could be programmed using visual code, and be integrated with those other components programmed in traditional languages. Heck, GUI builders and visual tools for database schemas already work that way; the only thing they lack is the capability to be expanded with new components, which this new family of visual tools have.
Because it doesn't. Watch Bret Victor's last talk about Eve. All those ideas fail when you give them to novices and try to do anything more complex than describing a few static relationships.
Maybe there's a way, but I'm skepitcal.
Of course experimental tools will be severely limited compared with industrial IDE systems that have been polished literally for decades; it will take years before we get these new tools into shape and learn what their best use cases are. But I'm not skeptical, because I've seen it happen before, and the conditions for it are now better than ever.
I've thought about what could be done to improve on their weaknesses (like strong typing, bridging the gap to databases, better table naming), even wrote some code in this direction, but anything I can think of involves either making them less visual, less user friendly or both and doesn't even begin to address the lack of generality
Finally I have to point out that some spreadsheets have been around for longer than many programming languages in current use so they don't actually lack maturity, yet they have seen little improvement in over a decade.
They significantly drop the barrier to entry and allow basic controlling of systems for non-programmers.
It should be remembered that in the early days of the web, some people argued that it should remain text-only. Basically, if you couldn't read it on the command line with Lynx, they weren't interested. Of course, the web seemed different at 9600 baud (or even slower).
Being color-blind, I am not enthusiastic about the idea of visual programming, just because a lot of visual tools are color-coded in ways that are hard for me to work with, esp. if the default settings are not easily modified. Even syntax-coloring in a text editor is a chore to deal with.
I don't believe in it in the general case simply because I've never seen a general-purpose visual programming language work. Show me one and show me lots of people choosing it freely and I'll believe just fine. I'm not holding my breath, though. (There are certainly specialized ones that meet this bar.)
Most analyses of this problem fall prey to the "anything can be made to look good if you only consider the positives of one side and the negatives of the other" effect. Visual advocates generally make wild promises about the positives of visual programming, which turn out to much harder to manifest than imagine in your head, and then make silly criticisms of textual languages, mistaking accidental issues of complexity for fundamental issues in the language. (Visual languages, just as one example, have no special protection against spaghetti code, any more than anything else does. Spaghetti code is not caused by textual languages, it's caused by developers creating spaghetti code from the spaghetti concepts in their heads.)
So not only are the analyses incomplete, examining only the positives of visual languages and the negatives of textual languages, most of the ones I see are even pretty terrible on those terms.
If you want do this, you need to look at both the pros and the cons of both sides, fairly. And if you haven't got a solid set of positives on the textual side and negatives on the visual side, go back and do it again.
(I'm not saying you must conclude that the net benefit is to text. You may have things that outweigh what I mentioned and your sum may come out in favor of visual programming. I'm simply saying if you can't explain in a sensible manner why text has been successful and why existing visual languages have failed, you do not have an analysis. You have a wish.)
While that serves as a reminder that visual programming isn't a cure for spaghetti code, (a) it was never claimed to be, and (b) terrible spaghetti code exists in all sufficiently powerful programming environments. I don't know that I'd enjoy tracing through that any more than untangling a mess of objects and functions that call each other—but then, I don't know that I'd "enjoy" it any less, either.
We are in the middle of the Visual Programming renaissance, now that the hurdles that limited composability of visual programs have been theoretically solved in the last decade by Programming Languages researchers, and computers are finally powerful enough to move visual languages without slowing to a crawl.
Unfortunately, Visual Programming suffers the same curse as Artificial Intelligence: for every breakthrough that produces revolutionary new tools, it is no longer considered Visual Programming but merely "how the tools should work".
[1] http://aprt.us/
They are very limited compared, lacking even maps. When it comes to connecting existing behavior together they are king. Consider the parts of game development which cannot be made generic. By sticking those in Blueprints we get nice high level code.
IntelliJ has done more for visual programming, by integrating visual/wizard aspects seamlessly into a text-dominant environment, than almost all the visual programming advocates of the last fifty years.
Visual environments do demo well, and sometimes convince businessmen that they won't need to pay those annoying and expensive programmers so much money. Hence its enduring appeal to the business world.
Another problem is how to explore and manipulate a flow graph? I like the idea of using a force directed graph. Here is one, where I mapped a visual music synthesizer to Jeffry Heer's Prefuse toolkit: https://www.youtube.com/watch?v=a703TTbxghc
Would love to explore authoring flow, using force directed graphs on an iPad pro + pencil (and its ilk), where one could not only draw arc between nodes, but also between convex hulls of a collection of nodes.
http://blog.interfacevision.com/design/design-visual-progarm...
The issue isn't visual programming but rather that we haven't found many good cases to use it still.
I do think visual programming is going to have a bigger and bigger impact in things like robotics, space exploration etc where you need more tactile access to computation power.
It's kind of like with the idea of a 3d interface. It only make sense when the holistic whole rather than the details are important.
It makes no sense to create a 3d warehouse I can walk around when I could just do a search and a list.
But you wouldn't necessarily want to control a robot doing repairs on your spaceship by having to program it every time you wanted it to do something new.
(From my submission just 6 days ago.)
https://docs.unrealengine.com/latest/INT/Engine/Blueprints/i...
I for myself think visual programming is not really programming, since by following that train of thought any GUI like Photoshop Actions could be considered programming.
It would be like trying to program musically, or by touch, or by (oh no) smell.
There is also the productivity danger of futzing, or whatever the modern term is. In the early desktop publishing era, we were promised that instead of taking ten minutes writing a memo, we'd type one in five minutes, but reality was we'd spend an hour using a minimum of seven fonts and three text colors and pictures. So much for productivity improvement! Likewise a visual programming language would invite people with authority but no ability to endlessly rearrange "put that box over there, looks nicer" "Make that text red" "that needs a larger font".
It is common to draw diagrams when you are trying to understand or explain a problem. If you want to give somebody a rough idea of how your program works, it is infinitely more likely that you use some kind of drawing than music, touch or smell.
I don't agree neither with visual programming inviting people to lose time in details. The same you do not lose time thinking about how to structure a sentence to write a program, you would not lose it changing colors and fonts in a visual programming environment with an enforced "syntax".
I do not like visual programming (I have had very bad experiences with LabView), but I think it may be as good (or as bad) as text if the field had evolved differently.
The only time we generally communicate visually is when we want to express something with less detail (e.g. charts, diagrams, graphs, emoji, etc).
Hmmm, someone should create a visual programming language using Dilbert characters & iconography. :)
And I don't believe it worked within the US Army [0][1][2] - unless I'm not correctly remembering the history of Outsystem's Visual Programming there...
[0] https://www.outsystems.com/company/news/2012/outsystems-us-a... [1] http://www.army.mil/article/96352/SEC_s_Cloud_Mil_solution_f... [2] http://www.afcea.org/content/?q=us-army-innovates-cloud-comp...
[1] https://en.wikipedia.org/wiki/VPL_Research
I'm sure various business apps can be built successfully on top of their Visual programming IDE, but even Microsoft has mostly abandoned that route.
But I dislike that even they propagate the lie that systems are slow when built on top of "High level language". That is mostly an optimization problem and you can often optimize well beyond what people would be willing to do in lower level language.
I've used outsystems, and it was straightforward. Although I prefer C#.
I have not used visual languages extensively, but it seems like they are better suited to simple control flows with flat data structures and limited state mutation. This would explain their success for graphics shaders, audio, industrial control, etc.
We may not be Visual Basic big (yet!) but we're running a successful business on a visual programming IDE with lots of users who love it!
But I believe it is the future. I bet you won't be carrying your laptop, enslaved by the physical keyboard, which is necessary for traditional 'coding' with you in 2030. You would be dragging and dropping stuff, perhaps in Virtual reality headset or through voice or gesture to build what you want, just like you would author in a visual platform like Adobe illustrator or in a 3d authoring tool.
Visual programming have it's own strength and weaknesses. For e.g. Creating algorithms from scratch in a Visual programming is not very productive (yet), but authoring business logic is highly effective in Visual programming. How many times, as a regular programmer you have had to write an algorithm? Most often it's just business logic that we tag under the category of an algorithm, which is not right. Algorithms are very generic piece of logic and business logic applies to your own problem/data specifically. If you have a closer look you realize that we are simply reusing algorithms or modules (created by a smaller group of people comparatively) to build the rules/business logic. What we do is mostly write glue code. we don't really need to code for that.
There are tons of examples for highly successful application of visual programming. But as an average web developer you never get to see that. Reason - Visual authoring tools are highly expensive and hence are confined to fortune 500 companies. That along with lack of quality open source implementations are the reason for visual programming to never take off among general web programming community. Even the newer generation ones like Treeline (for backend) and Bubble.is (frontend) are also closed source. NoFlo is open source and it almost started to create some buzz, but the team got sidetracked apparently, and they are using dataflow method for authoring frontend which is a highly inefficient.
You want to see successful application of Visual programming in real life? Ever heard of Enterprise Application Integration (EAI) ? Business Process Automation (BPMN/BPEL) ? Extract Transform and Load (ETL)? They all employ Visual programming to a great extend and are multi-billion dollar businesses. Do a google image search of following, in no particular order.
Dell bhoomi (EAI)
Tibco Business Works (EAI)
Snaplogic (Founded by Informatica ex. CEO) (EAI)
MuleESB/Anypoint studio (EAI)
Oracle BPEL (Business process automation)
Tibco iProcess (Business process automation)
Mendix (Web app development)
Webratio (Web app development)
(+lots of others if you dig out)
I disagree. I wouldn't have thought that Microsoft, author of Visual Basic, would be basically forced in 2015 to add improvements to their console system -- but here we are. If anything text-based programming is more prominent now than in the last 10 years.
> You would be dragging and dropping stuff, perhaps in Virtual reality headset or through voice or gesture to build what you want, just like you would author in a visual platform like Adobe illustrator or in a 3d authoring tool.
Your arms will get very tired.
> but authoring business logic is highly effective in Visual programming.
I disagree. We have one giant visual-programmed workflow solution and it's a very modern and powerful system but it's also insane to work with. You can describe the problem in text with much clearer than trying to follow a diagram that won't fit on a dozen screens in either direction.
> Reason - Visual authoring tools are highly expensive and hence are confined to fortune 500 companies.
They are snake-oil; nearly something you can only sell for millions of dollars to people who make billions of dollars. You need a team of highly paid expert consultants to make changes because nobody else knows these systems or wants to know them. It's like going back 30 years in terms of expressive power.
> I disagree. We have one giant visual-programmed workflow solution.
Why couldn't you break it up into multiple sub-workflows?
> They are snake-oil
Agreed. we need great open source solutions to move the needle from where it's been stuck for over several decades.
Oh it is broken up into multiple sub-workflows. There is even a way to call workflows like functions and there is a workflow that decodes XML.
So many of the problems are trivially solved with small programs, or even gasp just plain SQL. I concede there are features delivered by these fancy ETL tools beyond the quick (but not dirty) of custom solutions, but the overheads are higher too and I believe a cost-benefit analysis would invalidate the supposed business cases for using these tools.
This is not not-invented-here syndrome. This is simpler and cheaper. The problem for my case often is previous attempts to do it ourselves are shit. That's not a problem with the technology. :P
Behind the scenes I've solved hundreds of problems - one-off or on-going, small and large - by writing custom applications. This is the real increase in productivity that justifies my fee.