Hacker News Reader

Well, for a node oriented approach we could have standard auto layouting that we could augment with hints for grouping, layouting prioritization etc. to achieve some kind of "universal editing" approach for visual programming.

After all, we use lots of comments to give some structure to text-based coding.

There was a post about https://github.com/samuelmtimbo/unit recently, which at least uses some kind of hinted auto-layout (besides the more propriety fancy)

I really want to see more graphical coding for years, but node/graph-based and blockly seem to be the only approaches that got sone traction so far. So I like this thread and it seems at the right place.

I'd wish to see more use of 2D and now 3D space for structuring, annotation and overall overview and navigation. Text plus a directory tree view can't be everything, especially since we have so many structural views for coding, runtime, versioning that basically are connected in our mind, but also could be in a space with more dimensions. Not sure if we could find/agree on a common denominator, though. Text is one because it's basically the bottom of common ground. Not sure if that's not just the trivial solution (ok, I am).

How is this different from “drawing is better than writing a book”?

Some people write, others draw. Each method has its own strength and problem spaces.

Encoding a complex issue sometimes is just shitty with a visual metaphor. Other times writing code is a really bad idea. If you want to express “flow” or “connections”, filter or stream systems like modular synths, that’s when the visual metaphor shines.

You use whatever metaphor works best at the moment. One cannot replace the other.

The “problem” is that people want to push all of “programming” into the visual metaphor. That’s like pushing all of programming into only functional, only declarative, only imperative or only OOP (or whatever), but worse. That’s not going to work, but I guess people can try.

Yes, attempting to re-implement generic programming (as opposed to a particular domain) as node-based seems doomed to failure.

I agree. I love modelling state machines in node-based systems, and I think user interfaces could be fine in node-based editors (I’ve never tried, but they’re already node-based internally and they seem well-suited in the same way state machines are). But I’d hate to model something like a flow of concurrent tasks with retries and exponential back offs or something where the visual queues don’t seem like they’d be obvious and the editor couldn’t (as far as I can imagine) offer useful simulation or testing tools.

It could be a failure of my imagination. I’ve designed similar things though, and these weird cases (especially temporal cases) become extremely hard to represent more universally. I suppose you could offer a “timeline” to indicate how things would behave on a time scale, but even that… I just don’t know.

It also seems key to have a node-based editor work with one language or a set which map together extremely well.

OTOH, whenever somebody is explaining their multithreaded async code lines on a web page or in a book, there a graph is shown, since it seems to be the ideal metaphor. Just not for implementation purposes...?

There's definitively some hard limitation in dimension, though, so there might be the idea that, if one can only show problems of a certain limited complexity visually, why not do all the lousy (text-based, relational/reference-based way) but universal way that at least can be stored and edited everywhere.

So a central question might be: can we have a universal gain from, let's say, a three dimensional notation that still does not limit the problem space to these dimension.

We already have had good UI designers, they just never made it to the web.

As a UI designer (at times), I have to say... It isn't easy. I've made much nicer UIs on MacOS and iOS than I have on the web, with considerably less effort.

Part of it is what people expect. You get pushed and pulled when it comes to designing things for the web. Partially by constraints, partially by people's desires and expectations. It's better than ever these days, but my career has generally had the greatest challenges presented on the frontend of web projects.

I find backend development a bit "smoother". I won't say easier, but... I don't know, the frontend of the web is a jungle and a bit of an uphill battle sometimes.

I suspect my phrase “UI designer” was misunderstood: I learned to program in 1996 or so with Delphi and the graphical UI design paradigm of that environment failed to make the jump to website design

I think node-based can work well for data flow or control flow. But probably not both.

Idle thought: I sometimes wonder if SQL could be implemented visually. It's basically sets, after all.

I suspect it would quickly become too complex, but there's plenty of people smarter than me...

You often see SQL joins represented with Venn diagram, but it realy isn't a great representation of join.

You sure? I tend to think in terms roughly like this when I'm selecting a JOIN: https://dk81.github.io/dkmathstats_site/set-theory-sql.html

Eg (A LEFT JOIN B) = (A ∩ B) + A

Maybe I'm missing something though.

https://blog.jooq.org/say-no-to-venn-diagrams-when-explainin...

Hmm. It's true, the set model breaks down when you remove the key relationship, or add filters.

I worked on a visual language design for a long time as a hobby, and the biggest problem, I think, is how you communicate about the code.

Imagine asking Stackoverflow about a coding problem that involves a visual node structure and code attached to some of them.

So yes, the round tripping is essential, but it also needs people to understand and be able to talk about the code in its textual form.

I came to the conclusion that realistically these tools will always be severely limited unless they focus on being good automated visualisation tools of syntax that is text first.

I'd like to be wrong about that, by the way.

Although it's not quite the same, I do like what Enso[0] is bringing to the table, especially the 1:1 visual node/language interop. Whether this is generalisable to a fully decoupled interface remains to be seen, but there's definitely potential.

[0]: https://enso.org/

We need a generic "graph editor" (let's say VS Graph) and then use-case specific "graph server protocols" (Unreal Blueprint GSP).

A graph editor is a large undertaking, having a few generic ones would avoid so much duplicate work. Having a generic graph editor would also allow an extension system to develop around it.

Each application would just need to implement the specific server protocol which does game rendering etc...

> I think the core problem is that - unlike text-based programming - the editor is intimitely tied to the rest of the authoring environment and runtime.

Does not sound like the core problem to me. Computers used to be like that: everything was tied to the OS and compiler. That didn't stop banks from developing pretty complex systems.

IMO, the problem is that visual programming, despite its name, is visually a mess. You can't read it, you can't orient yourself in it. It's not structured, for starters. And piping an input through a sin function takes up 10 times the screen space of Math.sin(x), especially if you then have to keep that node's output from overlapping other parts. And you always must search back where the input to a node comes from, whereas in textual programs, you can put that information in the variable name. It's just not suitable to our way of thinking in chunks.

That is the outcome of people coding in visual languages like BASIC, instead of making use of the structure programming tooling, that they also offer.

> Off topic (and a bit "hijack someone's post") but I keep thinking about the inherent suckiness of visual programming and what needs to happen to improve it.

Visual programming presents huge challenges.

> Text coding benefits from standardised tools

That's the biggest, IMO. Some degree of text processing is essentially universal to computing after the early 1950's. As you mention, that provides a huge amount of prior art in editors, source code control systems, various kinds of messaging, display tools... the list goes on and on. Textual languages can natively reuse all of this. Graphical languages need some sort of mapping to text in order to use tools that do not quite fit the way they work.

The other challenge is the way visual programming complicates the mechanics of the job. Laying source out in (~) one dimension is hard, laying it out in two dimensions is harder. Naming a function is hard, drawing a representative icon for a function is harder.

You also lose the notion of linear ordering when you move from a list of statements to a 2D graph of nodes. This is useful when you're modeling a data flow graph or a state machine, but harder when you want to express a branch or a loop.

The problem here is that a fundamental question has not been answered, and as far as I can tell, has not been addressed by any of these visual environments:

What does an algorithm look like?

Herman Hesse alluded to this in his novel _The Glass Bead Game_, but despite decades of discussion and work, no one has made a convincing pysical representation of that system.

I love the concept, and have made some moderately complex attempts, e.g.,:

https://www.blockscad3d.com/community/projects/1430644

https://github.com/WillAdams/gcodepreview

it always devolves to screen size being out-paced by problem complexity --- one gets something of an inkling of this at:

https://scriptsofanotherdimension.tumblr.com/

Alternately, one can just break a project down into modules, but then the top-level view becomes the wall of text representation (albeit w/ nice lines or captured into pretty boxes) which one is ostensibly trying to escape.

I'd love to see someone succeed in this, and I've been using:

https://github.com/derkork/openscad-graph-editor

quite a bit, and put a bit of money towards:

http://nodezator.com/

Something I realised recently is that "visual code" can represent different paradigms. If you use Node-RED[2] than the visual code presents events and data movements. If you use something like Blender nodes or Noisecraft[1] then the visual code presents a state machine which is always on.

It helps to think of unix pipes when working with Node-RED, but when working with state machines to think of wiring on a circuit board. I wrote down my ideas in blog[3] - if anyone has more representation, please share!

[1]=https://noisecraft.app/ [2]=https://nodered.org [3]=https://blog.openmindmap.org/blog/pipes-wires-nodes

I think the inherent suckiness with visual programming is...

Imagine I was a consultant who offered to sit and watch over your shoulder and comment only on syntax issues. If you make a syntax mistake, I will help you fix it; "you need a closing paren here". I will also offer syntax tips constantly while you work; "looks like your going to create a list, first choose what you want the first item in the list to be". You should expect to be a little slower in exchange for these consultations I offer. Would you hire me?

These are essentially the tradeoffs offered by visual programming. Slower work for syntax level help. And visual languages often have some premium cost to them.

People doing significant programming don't need syntax level help.

A long time ago I worked at a company trying to commercialize a visual programming framework for general use. And before that I was on a team that built a visual programming environment to define simulation models for the largest flight/other simulation company in the world.

What I learned was that when a problem domain needed a visual representation they had already created it (using pencil and paper, decades ago). Think of schematic diagrams, piping and instrumentation diagrams and such. In those cases we were very successful. In cases where no such diagrams exist, we always failed to create one that stuck with the users. Text is great for code because is is extremely compact and takes advantage of things like visual rhyme. Uml, for example, can never achieve the same results

It’s not syntax help thats useful about visual programming it’s the 2-way communication between the programmer and the running program, and also being able to refer to other parts of a program spatially rather than by a name.

Most kinds of software don’t need these things but if you’re writing code that generates graphics or audio signals then you do, because you get to manipulate your program and see/hear effects in real-time, and also seeing how different parts of your program connects together is more important than giving names to them (where there may not be a useful name you can give them anyway)

Theoretically you might be able achieve these things with only a textual representation and it could be best of both worlds

CGI has used "visual programming" for decades not with great success. Quotes there because it depends on what you consider visual programming. It's not syntax level minutia as it's more of a functional blocks or expressions even that mutate data from start to end.. which in its essence programming is all about anyways. See Autodesk Flame, Digital Foundry Nuke, SideFX Houdini for examples.

Yes - I'm aware of these and have used several. And they all suck in various ways compared to writing code in a capable IDE.

I can't imagine a situation where given the choice (and assuming sufficient familiarity to be fluent) I would choose a node-based system over writing code. As I said earlier - the main advantage is discoverability (plus a certain robustness against making errors which arise from the constraints).

Discoverability is cancelled out by familiarity and if I'm going to use a tool seriously then I will become familiar with it.

Every other aspect of visual programming is a disadvantage and in many cases a crippling one. The sheer laboriousness of constructing expressions, the lack of sophisticated refactoring and the amount of busywork tidying up nodes so they aren't a jumble of spaghetti.

I think these are solvable but not until there's a way to avoid reinventing the wheel for every single system.

Every time someone tries to standardize UI, we end up with XML again. There's gotta be something else.

You've go to store your graph in some structured form. Ideally text based. XML seems as good (or bad) a format for this as any other.

Could be XML, could be SQLite tables, it really shouldn't matter - the format would not be meant for human consumption, much less editing by hand. It would best be diffable, yes, but not necessarily by diff(1).

I disagree. Unless the Visual editor was incredibly sophisticated, then there's always thing that can be done better by manipulating text.

The textual representation doesn't have to be the primary way of interacting with the program and it doesn't need to be optimal for everyday use - but it should be sane and visually comprehensible when the need arises.

Yes, but the benefits of textual representation you mention are UI too - the text you manipulate does not have to be the same as the serialization format/canonical representation.