Node-RED: Low-code programming for event-driven applications
nodered.org
nodered.org
What is something people use the software for, where you would say they should not do this?
Node-RED seems so far optimized for IoT and online services. Is there any ability (or plans for them) to use it locally for desktop-automation, similiar to AppleScript & Automator on MacOS, shell & batch-scripts, Selenium for websites, Tasker on android, etc.?
How would you see the state of the project generally and feature-wise ? Is it still growing for the next decade? Or has it reached it's peak and will move to settle down at its optimum somewhere in the next years? Should it reinvent itself every some years and stay fresh and adapting?
> What are the flaws of Node-RED? The areas that you plan (or hope) to give many, many necessary love somewhere in the future?
It works really well as a single user system. One of the areas I really want to improve is the multi-user/collaborative features available in the editor.
You can certainly have >1 users editing the same flows in parallel, but you do end up having to merge back in each other's changes. That user experience isn't ideal.
I'm also keenly aware the documentation and getting started experience needs attention.
> What is something people use the software for, where you would say they should not do this?
I wouldn't tell someone not to use the project if they had found something that worked for them. As with any solution, you need to make sure it satisfies your needs and requirements.
I would say you need to acknowledge the limits of the system - partly born of the single-threaded nature of the underlying Node.js runtime.
> Is there any ability (or plans for them) to use it locally for desktop-automation, similiar to AppleScript & Automator on MacOS, shell & batch-scripts, Selenium for websites, Tasker on android, etc.?
It isn't something we're thinking about in the core of the project. But the project is designed so that this could all be done via 3rd party nodes created and shared by the community to integrate with those tools. In other words, this sort of integration could be done by anyone - it doesn't have to be the core project that does it.
> How would you see the state of the project generally and feature-wise ?
There's always more to be done.
> Should it reinvent itself every some years and stay fresh and adapting
There's a risk for all software projects that continually reinvent themselves that they sacrifice their existing community in search of something else.
A core principle we have is stability. That doesn't mean stagnation, but it means evolving in a considered way that brings the community with us and doesn't leave users stranded because they depend on something that has since been 'reinvented'.
It is primarily functional programming. This means, you have an input, that is processed through blocks (functions) and does some things.
This is GREAT for things similar to IFTTT (If this then that) applications. E.g. you receive a datagramm via MQTT, and then format it, add some info, and upload it into influxdb.
You can also easily add a small dashboard with some buttons and labels for easy home automation.
What needs some working around with context store are side effects, or persistance. I have two examples that can be worked around, but are rather hacky:
1) Reading a list from a homepage, and only sending updates for new items. This is the prime example for context - you save a list of seen items in the context of the function node, so the next invocation can filter out the existing ones
2) Using a switch in the dashboard for selecting between automatic and manual mode. E.g. I want to have my rack fans controlled by the temperature, but with an override to set it manually. This needs a bit more hacking with context where you have two flows into one node, and you have to set the internal state of the context depending on the override button.
my use of it for hobbyist projects helped me get my foot in the door for my current employment in automation and controls. I have multiple instances of it running to track various things, run my chicken coop door, send alerts, aggregate data... It's uses are endless and I enjoy the puzzle of using the new updates to the software to make my old flows better.
They span a wide range of industries and technology areas. Quite a few are industrial controller companies who include Node-RED on their commercially available edge devices.
But as an OSS project, we aren't looking to create a hosted SaaS offering - the core project isn't trying to compete commercially with those vendors.
But there are companies looking to create commercial services around Node-RED that certainly want to be competitive. (My own company, FlowForge Inc, is one of them)
I'm not an NR user yet but I've implemented my basic Home Assistant set up and plan to begin using NR extensively as I evolve my home automation beyond "Hello World".
Suggestion: when I last used it, loops was a third party add on, you should look at getting that into the core, maybe hire the dev of the add on?
For example: https://flows.nodered.org/node/openapi-red
If you're looking for something that can take a Swagger/OpenAPI doc and generate a stubbed-out set of flows - that would be a neat idea. Sounds like a cool project to create as a pluing.
It's sometimes overly complicated to follow the flows, or do something relatively simple, but otherwise it's a nice middle ground between built in YAML based automations, and something like AppDaemon which requires writing python scripts.
All in all I use both. I use the native ones for simpler automations (exterior lights on at dusk, blinds up at sunrise, etc.j and node red for the complex stuff (bathroom fan on for 5 minutes unless if the door is closed or the humidity is above 65%, turn off two minutes after the door is opened and the humidity is below 60%j.
-REST
-GET, POST, PUT
-JSON
-Javascript
-MQTT
And all that helped me a lot in my career, just because i wanted to automate a few things. And now Node-RED is just irreplaceable in my private life.
Node-RED 2.0 - https://news.ycombinator.com/item?id=27892226 - July 2021 (62 comments)
How-to normalize home volume levels with Node-RED - https://news.ycombinator.com/item?id=23349902 - May 2020 (28 comments)
Node-Red – Flow-Based Programming for the Internet of Things - https://news.ycombinator.com/item?id=18859019 - Jan 2019 (77 comments)
Replacing a Pool and Spa Controller with Raspberry Pi, Arduino, Node Red - https://news.ycombinator.com/item?id=18142335 - Oct 2018 (3 comments)
Node-RED: A visual tool for wiring the Internet of Things - https://news.ycombinator.com/item?id=13495156 - Jan 2017 (6 comments)
Node-Red, a tool for wiring together hardware devices, APIs and online services - https://news.ycombinator.com/item?id=8419984 - Oct 2014 (8 comments)
Node-RED - https://news.ycombinator.com/item?id=8310536 - Sept 2014 (1 comment)
Node-red, a visual tool for wiring the Internet of Things - https://news.ycombinator.com/item?id=8217664 - Aug 2014 (3 comments)
Programming the Missing Links in the Internet of Things - https://news.ycombinator.com/item?id=6459564 - Sept 2013 (1 comment)
Plus, it is the perfect accompaniment to Home Assistant.
If you're interested in something more generic, check out if https://enso.org could be worth a look! Thanks to GraalVM, you might be able to import JS packages too, among others (Java, Python, & more...); OTOH with the caveat that the project is still in ways in-dev...
Interesting thing google missed when trying to prevent people from doing this exact thing with PWAs: If you have a valid manifest and service worker, Chrome won't run the service worker if it's not served over SSL, but it WILL include the app name and icons and let you save it to the home screen on Android; it will even cache a good deal of what you intended.
For auto-discovery, I used a fuckton of hidden iframes loading asynchronously and waiting for any window.postMessage that matches what I want from a local device.
Seriously, Google has made my life fucking miserable as a solo dev. They don't want anyone to be able to write an IoT app without going through them.
Fascinating discussion here with a Plex dev expressing similar frustrations with the general direction of these context barriers... which in their case affects millions of users and threatens to pull down their entire platform. Their solution currently is bizarre; they actually have to hijack DNS resolution and issue SSL certs on the fly for each of their users' servers to allow their web app to connect to a local node. Not possible in my smaller corporate sphere.
https://community.letsencrypt.org/t/dns-providers-who-easily...
Our solution has been to basically mandate that the user install either a small embedded device (ie. RPI or equiv) or manually launch a container. These essentially just serve as a local websocket / api router so Chrome connects to some local "device/container" and then that connects to all the local devices.
Although this is a logistical hurdle, its actually simplified other aspects of development since now this device/container has a bit of intelligence and can act as real middleware to simplify what the UI needs to contemplate. Basically MVC-style development using hardware bits as Controller.
* Local RPI / Container connects to message queue in cloud * Web User inputs request to Web UI (cloud) * Local RPI / Container retrieves request via queue and access final device * Reports status back to queue which is then reported by Web UI
Basically the local device is your router using whatever message queue you want.
In my house, the central lights turn themselves on when it becomes too dark (except after bedtime), and off when it becomes too bright. Lights turn on automatically when someone enters a room (when it is dark) and turn off when they leave.
When the last person leaves the house, all lights go out, except when I am on vacation, in which case the lights turn themselves on and off in a pattern that looks like I am at home.
After lights come extractor fans (keeping the humidity constant) and heat/air conditioning.
It is all about maintaining a comfortable home environment without having to manually turn things on and off all the time.
The outside lights on the front of my house have a motion sensor. Alone, they turn on to about 50% at dusk, and off at 11:30. If there's any motion between dusk and dawn, they go to 100%, then eventually fade back to 50% or off depending on the time. They also come on (and stay on) at 100% if the garage is open. End result: they are at 100% when useful, and otherwise at 50% don't (annoyingly) shine as much into the front windows.
I also experimented with turning down the kitchen lights when there's no motion. The effective of this was described to me as "What did you do to the kitchen lights?! They just randomly turn off and it's driving me crazy!" -- even not in the room, the lights in an adjoining room turning off unexpectedly was bad. End result: Motion controls removed.
There's also some simple time-based controls. If no lights have been manually adjusted recently, at dawn a couple lights come on (to make it look like someone's home) and likewise turn off sometime later at night. There's also an absolute off at 3am, and again at very early dawn. These have the effect of never coming home to a dark house and never leaving lights on all night.
The best automations blend in so you don't even notice them. But absolutely make sure humans can still control things, and don't do things that interfere with that control or expectations.
If you get used to candle or other low-power nighttime lighting, typical house lighting seems obnoxiously bright. I'm convinced it's a major contributor to the sleep troubles that, it seems, nearly everyone in the modern "West" has. It's way more light than you need to get around, play board games, play or listen to music, read, et c. It's like we're trying to make our houses as bright as they are in daytime, which seems like it'd have to be really bad for sleep.
I'd just put dimmer bulbs in all around and forget about automation, but I kinda do want the daytime-like brightness for at least 2-3 hours after sundown, in Winter, or the option to turn it back to full brightness if I've got people visiting and they don't like it (it's not that way so you like it, it's that way so you get sleepy when you're supposed to—but whatever)
The kitchen in the place I rent has this island (its more like a peninsula) that just barely makes the light switch for the kitchen out of reach, so instead we have to walk around the way around.
Obviously a manageable situation, but automating things has been a covid hobby. So I automated the lights, obviously, via replacing the light switch and added a motion sensor to the kitchen wall. There's also an echo dot for voice control.
Walk into the kitchen, lights turn on, eventually they go off. If they don't go on, just say "alexa turn on kitchen" and they go on, motion detector eventually picks up and turns them back off.
Use the light switch, and it disables the motion sensor for an hour and just leaves the lights on, or off, whichever direction was pressed.
Time saved for everyone not walking back and forth, also at night, has definitely been greater than time spent for me debugging.
YMMV
Unfortunately automating normal curtain opening is pretty tricky mechanically. It was easy to get as far as "Ok Google, open the curtains" -> stepper motor moves. But connecting the stepper motor to the curtains is tricky. I think that's why smart blinds are way more common.
Lights come up automatically based on time of day and motion in rooms.
Our family channel in Telegram gets notifications if the house is too hot/cold dry/humid, maybe a humidifier gets turned on automatically.
Eg built one that tries resolving an ad domain to check if ad blocking is functioning correctly and publishes that as a end point that is suitable for an uptime dashboard
I’m looking for ways to offer practical/accessible low-code (or not considered scary code) composition for our internal users who are running light industry factory production lines, but open to learning about commercial applications in any field to draw inspiration from.
Telegraf has the ability to phone home and ask for its telegraf.conf as it starts up, but the server-side of that equation is frustratingly left as an exercise for the reader.
There's not many ways to skin that particular cat, which is odd and unsettling in its own way.
I have Node-RED installed and running alongside HomeAssistant, but I never use it because automations that are easy to do in NR are just as simple to create directly in HA. Automations that are complex enough to need NR are easier (for me) to code and debug in Python.
I think NR is great for non-programmers and maybe for quick prototyping, so I am not disparaging it in any way. The functionality and implementation is excellent; it's just not for everyone.
I wrote one then never used it again opting for node instances inside of our Kubernetes cluster instead.
The other huge PITA is that touching a node in the UI causes the json to update even if you didn't do anything, and changes will often cause nodes in the json to reorder. That makes git commits very cluttered.
Merging is also very hit-or-miss and duplicate nodes are common. You can mostly fix this with another plugin that saves flows to individual files instead of one big file (default). As long as you split your flows and subflows up enough, team members will usually not step on each others toes.
I used this at work. Not a fan. If you only touched the system once every 4 months I can see the usefulness- It's easier to see the overall structure than wading through code, particularly if you're a visual person. It makes it relatively easy to pick something up that you have mostly forgotten.
It is a huge PITA if you are trying to build API features quickly with a team. Especially if there's nobody to tell you to use the plugins.
[1] https://flows.nodered.org/node/node-red-contrib-flow-manager
https://taoofmac.com/space/blog/2019/01/13/1900
These days I've changed the setup a bit and now take advantage of Node-RED's debugger plugin and Projects/git features, but the rest still applies.
https://gladysassistant.com/blog/integrate-node-red-with-gla...
I've used it for an insane amount of things, starting with home automation and telemetry but also fairly sophisticated uses, including:
- Sniffing for multicast streams on a LAN and exposing IPTV set top boxes as HomeKit devices
- Automating image conversion and video transcoding
- Building an RSS aggregation and transformation pipeline that does translations, scraping, etc.
- Building a simple GUI for managing backup snapshots and batch file copies
- Monitoring Azure deployments (and exposing the data as simple dashboards).
- Doing App store price checks and sending Pushover notifications when an interesting app goes on sale (backed by SQLite).
- An embarrassing number of Slack and Telegram bots for various user groups/events/etc.
As a general rule of thumb, Node-RED can do pretty much any kind of automation you can get a REST API for, and quite often someone out there has already built a module for it.
I think of n8n more along the lines of a graphical workflow/BPM tool for SaaS, but with much less flexibility/ability to go "low level".
About how many feeds are we talking here? One per hour? Ten? Some hundred? I've thought at some point doing similar, but was not sure whether node-red or similar tools would perform well enough with this kinds of jobs.
You can see an example graph to get a user from the DB by id here: https://github.com/julius/node-red-contrib-sqlstring#example
Does anyone have any recommendations and/or recommended reading on putting these two together?
https://taoofmac.com/space/blog/2019/01/13/1900
Essentially you can use Node-RED to either act as another HomeKit bridge or to handle MQTT events from homebridge itself (I chose the latter).
You can easily "fake" and inject new accessories into homebridge via the homebridge-mqtt plugin (which allows you to dynamically add accessories and "see" them in MQTT).
I have them send MQTT to Node Red.