Rete – JavaScript framework for node-based visual programming | Hacker News Reader

Rete – JavaScript framework for node-based visual programming

github.com

134 pointsImmortalin24 comments

The name reminds me on the Rete algorithm, which is pretty cool, but unrelated:

https://en.wikipedia.org/wiki/Rete_algorithm

I found the name confusing for exactly this reason. Rete is a big deal in rule-based systems.

Agreed. It's such a unique name for a classic algorithm and is so thoroughly established in symbolic pattern matching for production systems that I'd seriously recommend against adopting it for any other purpose in CS.

Any time I see something like this I'm reminded of Yahoo Pipes and how ahead of its time it was. Is there anything like it that's designed for enterprise ETL these? It occurs to me it's a much better way of thinking about data flow problems than code.

I think https://flow.microsoft.com/en-us/ intends to be an enterprise version of this idea, but I've never used it.

for enterprise ETL - there is also kettle which has been around for a long time- which has a desktop and web authoring client. https://github.com/pentaho/pentaho-kettle

Node based / dataflow UI are also the bread and butter of CGI and compositing industry. It's hyper efficient both for machine because it's often freely memoized/cached and user (think of it as a simplified haskell at your fingertips) because the stateless and unlimited-undo aspects.

I'm still pretty sure it should be the default UI for half computer introduction. Text for the rest, with the right paradigm of course.

Node based user interfaces can be great indeed. Unreal Engine 4 has a visual scripting system called Blueprints that is very impressive. https://docs.unrealengine.com/en-US/Engine/Blueprints

Another example of a node based user interface can be found in the open source 3d modeling and animation software Blender. https://docs.blender.org/manual/en/dev/render/blender_render...

The open source virtual modular synthesizer VCV Rack is node based as well, though it is so “by nature” in the sense that hardware modular synthesizer work in fashion of connecting inputs and outputs with patch cables. https://www.vcvrack.com/

When done right, as all of the three I mentioned above in my opinion do, node based user interfaces are great.

Not all node based user interfaces are equally good. I used LabView a bit at a course at the university and did not enjoy their system. The LabView node based user interface felt very clunky the whole time, even for doing simple things.

I’ve been wondering occasionally whether certain kinds of data processing that I am doing with Unix tools could benefit from a graphical system.

It’s not that I am not comfortable with the command line, but even as a rather die-hard user of Linux and FreeBSD I must admit that the command line has several shortcomings.

Of course, designing a graphical successor to Unix pipelines is a major undertaking that will require through thought. But I think that keeping that idea in the back of ones mind while implementing some smaller node based systems could eventually lead to realization of most if not all of the considerations that must be kept in mind for the design of a graphical successor to Unix pipelines.

Domo[0]. Their visual ETL is pretty good.

https://www.domo.com/

Two come to mind.

Apache NiFi: http://nifi.apache.org/

StreamSets: https://streamsets.com/opensource/

Side note, I'm an Apache NiFi committer and have had great success using it for enterprise ETL in a previous role.

Seems likestreamsets don't include a visual designer in the open source core? And no price, which seems standard with ETLs

https://streamsets.com/subscriptions

Yes, Bizztalk would be one of them.

We have one that's a little more complex than Yahoo Pipes. Small Demo: https://s3.amazonaws.com/ecyber.public/lucy-use-cases/1/vide...

http://lucy.iviva.com

This seems great for creating smaller graphs as long as they don't get too complex. I'm kind of obsessed with node-based environments for DSP and media stuff (Reaktor, vvvv, and many others). Whenever my graphs attain some level of complexity, some features become very important such as selecting/operating on multiple nodes and most of all grouping. Rete.js doesn't seem to be quite there yet (no multi select? and Modules only hint at proper grouping), but it's definitely on the way.

Try this: https://github.com/jagenjo/litegraph.js

Really nice, haven't figured out what it's capable of, but reminds of a similar interface I wrote from scratch about 15 year ago in Java Swing. Blows me away how powerful browsers and JS have become. Really amazing that you can do this with so few lines of code, compared the (tens of?) thousands lines I wrote to do similar.

Node Red is an other node based visual programming tool that is inspired by Yahoo pipes.

The recurring problem I've found with node-based visual programming is that it's very challenging to work with multiple layers of abstraction and that refactoring can be incredibly challenging.

Those are definitely two of the major problems that need to be solved for visual programming to really work. Many people look at the problems and just throw up their hands, but I personally think they can be solved. I look at things like people building giant systems in Minecraft, something definitely not even designed for laying out logic or processing, and the way they talk about the physicality and geometry of the processing they lay out suggests to me that some breakthroughs might be right around the corner. Years ago when Supreme Commander was released (the game), they introduced an interface where you held down Ctrl and scrolled the mouse wheel in order to 'zoom' from high level overviews all the way down to single unit detail. It felt so intuitive and natural after using it for a bit that I found myself accidentally trying to do it in a file manager application after exiting the game. I think that would work for visual programming as well. And who knows the possibilities that VR brings along....

Is there a way to represent recursive nodes or recursion in general?

It can be represented visually, but the engine does not support processing recursive nodes

This is nice, great job! I started one framework very similar a couple of years ago, but never finished. I'm going to take a good look at this :)

Anyone know a similar thing in Python? Need it to integrate it into a project.

Looks good. Time to brush up on my websockets and glue this into something.

Reply on news.ycombinator.com