Hacker News Reader

Hacker News

Comment by Pamar | Hacker News Reader

Parent Full thread

Pamar

I have some experience with a visual language (webMethods, currently owned by Software AG).

Guess what I miss the most when I use it?

diff -c

And all the rest you get with text (versioning, global search&replace, static analysys...)

View on HN

Programming by using a GUI to generate an XML file was a 1990s trend that still hasn't died. The thinking was "we need to make a visual programming tool for non-programmers," which is sort of condescending to begin with, and then the people who use it run into all these real problems that normal text-based programming languages have already solved (automated testing, version control with diffing, merging, etc.)

Programs are stored on disk as lists of program elements in S-expression format, so are still accessible to grep, diff, etc., and there's no reason why a version control system couldn't be used. Even with text-based languages, use of diff runs into problems when text is moved about or reformatted.

It was never my intention to develop a language that anyone could use. It's for programmers. You still need to write algorithms. It will attract different people, just like different text-based languages.

I am planning to incorporate automated testing into the language, but haven't decided upon the best way to do this yet.

In my experience (and, as I said before, this is confined to one specific product/language) while the whole stuff was, in fact, represented as XML files, trying to work at the XML level was a nightmare. Each single graphical element had 2 or 3 different XML "source" files, distributed in different subdirectories, and there was no support (official or unofficial) to operate at the source file level. In other words, any trivial mistake would probably corrupt your project in some subtle and not easily recognizable way.

Now that I think about this, it was exactly the same with a large Sharepoint installation (around 2009/2010?).

In my experience there is a world of difference (and a world of pain) between "this is a source file, you can compile or interpret it and if there is a mistake you will get an error message" and "this is the XML serialization of a complex hierarchy of objects and if you mistype a GUID somewhere you will only find out at runtime, usually because something not even remotely connected to your mistake starts behaving erratically".

I am not saying that your project has no merit or that it will fail, I am just commenting on why, in my opinion, visual programming never really became a serious contender for general purpose programming.

That sounds absolutely horrendous, and there's no good reason why they had to be like that. It's unreasonable to assume all visual programming languages do things the same or a similar way.

Granted. But keep in mind that both products were made by capable companies and were used to deliver value to customers.

I would be more than happy to explore viable alternatives to traditional text-based development but so far I haven't found anything that really impressed me.

But you said yourself that the s-exp storage format for your language is neither easy nor safe to modify, so how is it better than XML?

Anecdotally, I've also used webMethods, SSIS, and several other visual dataflow programming languages, and they look remarkably similar to FMJ.

I was referring to this:

> Each single graphical element had 2 or 3 different XML "source" files, distributed in different subdirectories

S-expressions with equivalent content are smaller and just as readable, and don't need an extra parser. XML is OK for text mark-up, but not as good for storing data (including programs).

As someone working on TIBCO BusinessWorks, a competitor/alternative to webMethods, this sounds all too familiar. We have had major challenge ourselves in bridging this gap - semantic diff/merge on process/flow engines like BW, WM, etc. with visual tooling.

Typically developers care about this, business buyers don't and they direct budgets and in many ways, product prioritization. To a large extent the proprietary or obscure nature of integration DSLs never helped bootstrap any meaningful tooling for users.

That's changing though with developer-power driving prioritization and investment. In our case, we hope to make good progress with an upcoming project called Flogo. Currently in stealth and soon to be OSS.

Disclaimer: I work at TIBCO on integration

I think the problem here is that if you want to go visual, you need to go visual all the way. There should never be a case you need to manually debug a serialized representation. If your environment can't support that to an acceptable level of reliability, then you need to explicitly stick with a double-form approach, where the visual form and serialized form are "equal citizens" and both are meant for the user to work on in parallel.

Trying and failing to go with option one is what leads to the XML mess you describe.

For what is worth, the reason we tried (and promptly abandoned) the idea to work at the XML level was the following: in a visual language some stuff like:

- "Find all places where I am using records of type X and replace that with a record of type Y"

- "where did variable RunningTotal gets initialized? when do we reset it"?

- "Can I write a simple script to verify which temporary variables are not removed from the dataflow after being introduced?"

(Debug in the more traditional sense works reasonably well in webMethods - but this was not the problem - the problem was that it is impossible to reason about a "program" as a series of instructions and analyze it as such, even if at the end of the day a webMethods flow service is just that: a series of steps that manipulate data in a tuple space).

So this is not a critique of webMethods (or FMJ) - I am explaining what the problems you encounter are when you try to use the graphical metaphor to work on reasonably complex programs. Most of the successful examples cited so far seems to be mostly about audio manipulation. I suppose this means most of the time you compose smallish "scripts", you work alone and not in a team, logging is not an issue, building large, reusable components is not an issue, extending existing logic is not an issue, and so on and so on.

Having also worked a lot with an ancient 4GL application (where the language was supposedly "more expressive") I can sum it up all my doubts as follows:

Certain paradigms work well as long as the programs are written and maintained by a single developer to automate simple tasks but do not scale very well past that point

Reply on news.ycombinator.com