Robotics and ROS 2 Essentials
henkirobotics.com
henkirobotics.com
There's also a reason why it's a standing goal, rather than a completed goal.
It does little or nothing really well, but it does everything kind of okay.
There's a lot of "alternatives", most of which do something much better, sometimes much much much better, than ROS, but none of which can replace it all completely. PyRobot, Viam, OROCOS, Webots, all of these do some things much better than ROS, but none of them can drop in and easily replace everything without a lot of developer time, often adding new features to the platform.
So ROS can get you past your A-round -- but count on addressing scalability and stability issues as go. Which is why "write out ROS" is on everybody's to-do list.
In any case, learning ROS is well worth it. As a hiring manager, I know I can easily teach you our system if you know ROS already.
Two examples come to mind of packages that do NOT do this:
1. plotjuggler [1], which is a tool for plotting time serious data. It can connect to ROS, but also supports many other communication paradigms, like mqtt, zeroMQ, websockets, custom data and some more I forget.
2. nvblox, which at its core doesn't depend on ROS but instead provides a ros2 compatibility layer.
[1] https://github.com/facontidavide/PlotJuggler
[2] https://github.com/nvidia-isaac/nvblox?tab=readme-ov-file#c-...
Works well.
https://docs.foxglove.dev/docs/connecting-to-data/frameworks...
I had never encountered ROS before but, having scoured the documentation to figure out how to write a malicious node[1] and exfil data via `rostopic`, it sure felt great not to have to use ROS professionally, and to know I was probably never going to encounter it again.
ROS 2 moved away from the codebase that the quest targeted many years ago.
Yes. ROS is basically mediocre interprocess middleware with a standard interface used by much academic software. So you can hack stuff together without too much trouble. It won't be efficient or reliable, but it's useful.
Think of it as the PHP level of robotics.
* it's a loosely coupled messaging framework (so you can have 5 daemons that talk to each other to accomplish control at multiple levels)
* it's got robotics libraries with difficult robotics algorithms (PID tuning, forward and reverse kinematics, a way to run control loops very fast, ...)
* it's got sensor libraries (algorithms) and drivers
* it's got a simulator
* it's a linux distro specialized for robotics
You can keep going for a while with this.
As far as I'm aware very few real time operating systems support CUDA, and none that are free. You could look at Concurrent's RedHawk Linux.
Don't use ROS2. The benefits just aren't worth it. It's great for school but it's terrible for production.
Using ROS is a great way to turn your project and business into a cautionary tale.
The build system for ros1, catkin, didn't have this problem because it turned all your packages into a single cmake build. Just one of the many ways that ros2 was a regression from ros1.
* https://wiki.ros.org/rosbuild
* https://docs.ros.org/en/jazzy/Tutorials/Beginner-Client-Libr...
* https://docs.ros.org/en/jazzy/How-To-Guides/Ament-CMake-Docu...
It doesn't feel like a joke anymore.
Highly recommend if you want to get into robotics you have ROS experience and demos / stories at hand.
Usually people get stuck with ROS in commercial environments either because they inherited it from an academic project (company is a spinoff), or they used ROS for a demo and afterwards never planned with long enough roadmaps to make the cost benefit analysis obvious enough.
There's simply no reason to skip ROS as a student. My advice was not for businesses, whatsoever. And every single ROS replacement I've seen is ROS-like enough that the language ROS defined for all of us is effectively lingua franca.
But the antibodies that are triggered by even mentioning ROS should tell anyone how much it is not a fun thing to learn or work with. But it is a common touchpoint. It's really hard to understand _nearly any_ design decision in robotics software without first understanding why it isn't ROS.
I think a lot of the "I don't see >40y/o swe" is a combo of:
There were more CS/IT grads from 2000 to today (<40 years old) than were ever produced prior to year 2000.
After 40 you tend to move into management, consulting, or just switch careers, b/c SWE pays well.
After 40 you don't work at companies that churn through young devs (so young devs are less likely to work with > 40 y/o)
ROS has some fundamental design flaws, but does integrate most development software from various research areas. In many cases, the sub-projects were abandoned decades ago, and only ROS developers keep these components operational.
The primary mistake most naive academics make is assuming you are dealing with a single problem domain. Robotics is different from simple automation in that it has 4+ different primary concurrent systems that must share state.
There is also the IP issues with parts of projects like OpenCV, and the patent holders (university legal departments) make it nearly impossible to license. i.e. unless you have >$3m to drop on each component... expect to get ignored or sued in a commercial setting.
This is why we can't have nice things, as most FOSS software is perpetually Beta ... Thus, nearly impossible to "clean" and deploy in a commercial setting due to coincidental contamination from the original IP rights holders (not the coder.) =3
What about buy 2nd hand/use/resell some hardware? I do not know how much cost a Ms in CMU, but sure does bot even compare to 20k.
Sorry maybe I’m a little bit negative, but if you have already one good title, that is more than enough. The problem is I fully subscribe to Stallone[0]
I would recommend the masters for the coursework and practical background. A degree from CMU will get you through the pedigree filter at most jobs. I've hired plenty of CMU masters students and have always been happy with them.
Robotics itself does have many of the job archetypes you'd expect from a hardware/software heavy industry.
Regardless of the path, specialization is key. You might be (woefully incompletely) categorized as:
- systems folks (middleware, CUDA, distributed systems, hard C, C++, ROS-like, etc). Even things like CI/CD knowledge as applied to robotics can get you a long way.
- Vision (CNN/DL, but do not stop there. You should also have basic knowledge of tracking, homographies, and the old school CV stuff to make it work well in a system (as opposed to just scoring high on training data))
- Controls (PID and the bigger cousins - this is not my area)
- Tracking (EKF and the bigger cousins - in particular MH-EKF and please don't skimp on batch filters we have enough CPU to do that nowadays)
- Mapping & Prediction (often this is done so poorly that a person who has lots of practice building 2d/3d and semantic maps is a godsend, especially incorporating uncertainty - critically predicting future maps (+5-30s) is an amazing thing to have)
- Planning (99% of planning nowadays is trajectory planning which is kind of "just" figuring out from a map and destination what the signal is over time that goes to the control subsystem. But there's also route planning- where should we go an in what order, activity planning (what are good destinations and in what order), or even multi-agent planning (how do I get a bunch of agents to not just collide when trying to do individual activities))
- Proprioception (Sometimes just called Estimation) you'll want to be an expert in IMUs, Gyros, the EKF filter, using vision signals, magnetic signals, GPS, etc
- Exterioception (more general than Tracking) options are LIDAR, Radar, Bog-Standard Dispartiy, or any other sensor you can think of.
- Simulation - rapidly becoming extremely important or foundational to a good robotics program, especially one built on learning. This often has a lot of overlap with games programming so this can attract a certain type.
Roughly speaking, if you decide you like one of Sim, Controls, Planning (what kind?), Proprioception, Exterioception/mapping, Tracking, or Systems, you should tailor your coursework and projects to match.
Oh and I should mention - once you pick one of these, they are often very different when you are working on wheeled vehicles vs legged ones vs aircraft vs spacecraft vs plain-old-arms/ manipulators. But the skills are transferable so learn strong fundamentals.
Anti-recommendations:
- LLMs. I have heard they are going to be helpful, but are just hype so far. Skip for robotics AFAICT
- UI/UX. Everyone says they are going to invent the easiest-to-use automated system, just as soon as it "works". Spoiler alert, just like Akins laws of spacecraft design, if your system involves building a robot to build some other product, you are defacto a robotics company and will spend all your time making the robot work.
There are very few "Generalist" robotics positions -- you'll be competing with PhDs who are specialists but picked up enough experience doing their PhD to have generalist level skills. You want to be on a specific team doing a specific thing deeply. It's also the most fun to be on a specialized team!
Remember, there are two hard problems in robotics: Perception and Funding. So Simulation, Proprioception, Exterioception, Mapping, Prediction, esp Vision (nowadays) are hugely important to the success of a program. As a planning guy, the rest is "easy".
But, the moment you ship your first product, open your task tracker and add “Epic: Migrate off of ROS to a custom framework”.
Source: I’ve been working full time building a custom ROS replacement for a large company for several years. Results are good.
When robotics moves from the lab to the field, expensive certainty becomes more attractive than cheap flexibility.
I love ROS. There are setup files for stuff inside ROS (pathfinding etc) that I haven't enjoyed, and I've definitely struggled with things like keeping it all standardised with like dockerfiles, but the ROS itself? the concurrency is amazing. I constantly wish I had it on non-robotics projects...
as others have mentioned, it has scaling issues. But for getting things done on a base level, its super powerful. The fault-tolerance it allows has benefited our robots a lot - it's often possible even with severely degraded systems to manually guide one around.
I do agree with what others have said about how the best usages are often to not go too deep into the ecosystem and instead use its messaging and concurrency protocol in limited ways, and have a lot of your internal logic siloed, to avoid hardcore dependency. After all, the idea of this kind of system is to allow new nodes to listen to sensor data (coming in) and/or planning goals (going out) to be able to make their decisions. Not passing some kind of intermediate calculation to some other subsystem
If you want to use ROS for IPC on the same host, fine. That can all be debated but okay. But if you want to communicate to different robots, especially over WiFi, ROS is missing most of what you need, and it's things that roboticists do not need to reinvent as the Web has been grappling with these issues for a lot longer by millions of more developers.
- You will have to worry about versioning, either on your terms of when you're inevitably forced to
- Even if you don't plan on operating multiple versions in harmony (years later you'll regret believing you could get away with this), you have to deal with the edge effects of when you upgrade them remotely/on-site.
- ROS API definitions ("ROS Messages") are defincient. Yes, you really need to support `null` but you also need a whole lot more expessiveness in general. You'll invent your own string-but-actually-json message at some point.
- You end up manually dealing with well-trodden problems like compression, authentication, authorization.
- Failed comms, reconnecting, retrying, etc. will be a huge pain.
I think the simplest question to ask yourself/team is: "Could this be an HTTPS API? Could this be a Websocket? Do we even have the correct skills on the team to answer this question or did we only hire ROS people?"
If I could decide (and in the area where I am, we did) I would ditch the whole thing. After all, if you squint, ROS is a collection of things:
- a launcher (which is a very bad scripting language embedded in horrible XML). Can be very easy be substituted by some python or shell scripts. - a description language of messages, that can be read by C, Python and Lisp; can be substituted by raw sockets or google protocol buffers or whatever. - a parameter/configuration distribution system, which can be implemented based on libconfig (https://github.com/hyperrealm/libconfig)
All that options are pretty much standard, stable and well supported, with bindings for any mainstream language.
I would run away from ROS2 to avoid another disaster when ROS3 comes.
ROS came about because before ROS existed, robotics researchers cobbled together ad-hoc ROS-like systems using the tools you point out. ROS itself started as a project called "Switchyard", which was built to operate one of Stanford's robotics projects. Research labs around the country each had their own take on this kind of system, which made sharing research very difficult. What ROS did was standardize the platforms between all labs, enabling us to share our algorithms, which at the time mostly revolved around localization, mapping, and path planning.
I agree with what you said about launcher/ipc/config - my plan if I ever use Ros is to keep it in a box and use my own communication layer to connect with things in various environments (rather than trying to solve incompatible conda environments).
however one thing you've missed, which may well be the biggest offer from Ros and the reason why I haven't sworn off it entirely is the library of robotics functionality such as SLAM and various planning algorithms.
in your opinion, are these at least well implemented? would they be easy to rip out and run separately?
My suggestion: start with ROS, but make absolutely sure to separate ROS from your “business intelligence“. The one think I hate from ROS, is that it makes it difficult. But keep them separated. Also the launch IPC and config parts, don’t do them dependent from each other. If possible do not rely on launcher. So later you can switch away if needed. And you will need. ROS is great for prototyping, changing things, development. But is just not for deploying in production. The whole possibility of inspecting the messages between nodes, means you have to pay a price.
> in your opinion, are these at least well implemented? would they be easy to rip out and run separately?
Not a simple answer: - from the architecture pov i think many questionable decisions were made: you need to master many languages: XML, launch, message files, yaml, python or C, Cmakefiles for catkin… why?! Why not making launcher files, messages all just yaml or json? Just too complex for nothing. - The code, you can look yourself, is write only. No way you can change anything without needing 1 year understanding… - BUT: it does work, and works well. I’ve not found a kill bug or something. So, I guess is ok.
I would say about half of the industry (read: for-profit robotics startups from early stages through to 10,000s of robots in production) uses ROS, and of those probably 2/3 are on ROS 2 at this point, and the rest are in some stage of migration. ROS 2 solved a lot of problems that didn't need solving, but like it or not ROS 1 is abandonware at this point so almost no one is planning to stick with ROS 1 longer than they have to.
Most companies aren't maintaining their own fork of ROS (other than to submit patches upstream), but a small number of companies on ROS 1 forked it and diverged so significantly that there is no point trying to rebase on ROS 2 - of these GM Cruise was the most notable, although the future of their stack is unknown now that GM canceled the robotaxi project.
The other half of the industry uses some sort of in-house stack built from parts (compare batteries-included frameworks like rails/django to building a backend using libraries like expressjs and sequelize). There is usually some form of pub/sub messaging architecture, because pub/sub is a natural fit for robotics and makes it easier to log and replay/resimulate. Some common things I see are zeromq, vanilla DDS (no ROS), zenoh, or write their own pub/sub (sometimes using shared memory). The messages themselves are often protobuf, flatbuffer, cbor, json, or sometimes just raw c structs.
Building your own stack isn't hard, but its much easier if you have used ROS before and know which concepts you want to reuse, rather than reinventing everything from first principles.
Some newer robotics frameworks are also starting to spring up which is great to see, for example https://github.com/copper-project/copper-rs and https://github.com/dora-rs/dora
There are also frameworks more specifically targeted towards robot learning, for example https://github.com/huggingface/lerobot
If people are just starting out in robotics, or just starting a robotics company, I still recommend ROS despite its warts, it is worth learning because it has had such a big influence on the current ecosystem. There is no "right answer" though, many companies have been successful with each approach.
"You either die trying to scale ROS to production, or you live long enough to repeatedly reinvent it?" - Johnny 5
A previous job used ros2. IMO, it's worse in most ways compared to ros1. The launch system of ros2 alone is enough reason not to use it.
I just got banned from their forum for questioning what the ROS Foundation has done in the past two years--specifically on the build system. Yes, their middleware is unforgiving of alternative ways other than their wrong and outdated development practices, but their community is more religiously attached to their precious tools for development.
This is the post I wrote: https://discourse.ros.org/t/build-systems-package-management...
So it is good that you come and read the comments here. Welcome!
Robotics Developer Studio started off in a promising direction. It had the CCR (Coordination and Concurrency Runtime) and the DSS (Decentralized System Services) which together made it possible to coordinate real time robotics in a RESTful environment across distributed hardware. It had also had full access to .NET. There’s never been anything quite like it since. Unfortunately MS pulled the plug before it had a chance to mature into an easy-to-use well documented platform.
I am thinking of raspberry pi with cameras and then using just building the slam algo from scratch. Is this a good option or am I trying to bite off more than I can chew. I understand ML theory but haven't done any other than the basic 101 ML programs and the coursera course. I am fluent in OS (systems), C, python.
Can folks with knowledge opine on this? Are there better ways to learn ML? I chose the robot so I can have something to keep on building/adding
At least for vision, deep learning should be very useful. For the others I have no experience how much machine learning is relevant. Geometry and physics based methods should work well here, but there is probably much room to tack on some ML.
The SLAM backend (optimization) is mostly old-school methods like nonlinear least squares optimization or particle filters. Not sure if that counts as ML today.
I'd go for g2o or Ceres for mapping (unless you expect to have no loop closures), as there's really no need to reinvent that. It's definitely useful to learn about the backend, but usually the combination of sensors and their properties will demand more algorithmic tailoring than the backend, which gets more abstract input (i e. motions & uncertainties) and can be used more black-boxy.
[1] https://github.com/Ekumen-OS/andino/
[2] https://docs.google.com/spreadsheets/d/1TMbENKlHr4g9bBA-EPNu...
Researchers, hobbyists, and some industry professionals (for prototyping?) seem to love ROS for its big community, logging/debugging/visualisation tools (rosbag, rviz), the pub/sub architecture (easy integration and loose coupling with third party modules), existing support for many sensors/peripherals/hardware components; plus off-the-shelf libs for SLAM, navigation, inverse kinematics, etc.
But ROS is hated by others for its custom build system (on top of CMake), the custom language-agnostic IDL for message types (like Protobuf but worse, esp. for versioning and backwards compatibility), non-determinism and untestability of the pub/sub model (esp. for safety-critical stuff), overengineering and layers of indirection / learning curve, poor performance, and high platform lock-in.
Looking at ROS as a robotics noob, it seems like useful 'muddleware' for toy projects, but where every sub-problem that it abstracts away from you can be solved better by other existing tools. I'm looking more into: Mobile Robot Programming Toolkit [1], Yet Another Robot Platform [2], genom3 [3], and Open Robot Control Software [4] (all C++). Or the Rust ones like RoboPLC [5], Dora-rs [6], Copper-rs [7], and Basis [8] (which supports Protobuf!).
If you're not using ROS for a "serious" project, do you use other middleware offerings or just pick 'n' mix lower-level libs tailored for your use cases?
[1] https://mrpt.org
[3] https://git.openrobots.org/projects/genom3
[5] https://github.com/roboplc/roboplc
[6] https://github.com/dora-rs/dora
Told him we should go together to industrial expo - starting from 3d printing, molding, material science and robotics and spend some time together to hack into hard core C++ SDKs used by big tech like Nvidia and Co, and look how to developed linear algebra and statistic on real life examples.
His dream is to automate small mid business manufacturing, and become a Steve Job like figure one day.
What’s the good information and resources to follow and educate about real life robotics?
Any tips or even sarcastic comments are welcome. Please help to be a cool uncle ;)
As for an Expo, I highly recommend ATX West show in a couple weeks. Depending on your employer, you probably could get it covered by your job and have your nephew come along.
(Was a Design MechE for a decade or so, but I'm a SWE now)
Quick aside: Automate small/mid business manufacturing? Admirable, but will probably choke on the scale problem, so I think the journey will be vastly more interesting than the destination...which is good! Turns out there's a lot of robotics that can be broadly applied.
* Manufacturing --> robotic manipulation, controls, actuation, sensing, decision-making
** Robotic manipulation --> Linear algebra, likely Python and/or C++, some simulation tools
** Controls --> Manipulator platform, drivers, physical- and protocol-level choices
** Actuation --> You want to pick something up right? Air powered? Electric? Probably not hydraulic, but worth mentioning?
** Sensing --> What's your sensor suite? RGBD? LIDAR? Forces-torques? Combination of the above?
** Decision making --> FSM? Behaviour trees? Purely functional?
Repeat the decomposition and you'll probably be able to get down to the basic level of the robotics hierarchy of needs.
3Blue1Brown and Khan Academy are great places to learn math of all sorts.
It largely depends on what you’re automating, no? That standard seems absurd.
Let's keep the useful image parts.
“How many of you are from manufacturing companies?”
[Some hands go up.]
“Oh, excellent. Where are the rest of you from? Okay, so how many are from consulting?”
[A number of hands go up.]
“Oh, that’s bad. Yeah, the mind is too important to waste—you should do something.”
“I think that without owning something over an extended period of time—like a few years—where one has a chance to take responsibility for one’s recommendations, where one has to see those recommendations through all action stages, accumulate scar tissue for the mistakes, pick oneself up off the ground, and dust oneself off, one learns only a fraction of what one can.”
robot wise, I'd recommend something they can program with a raspberry pi and python, if they have some light background in programming. I had a quick look on Amazon and I have no affiliations with these, but maybe look for something like this: https://www.amazon.co.uk/FREENOVE-Raspberry-Tracking-Avoidan...
note this one doesn't have either a raspberry pi nor any batteries. you might find a better one.
Thankfully, I have fully isolated ROS parts into their own dockerfiles.
These were generally designed to communicate over UART (to a physically-connected host) or a LAN (wired or wireless) using MQTT topics and plain text message formats.
We have a primary, core machine built around Linux SOMs like NVidia's Orin or the RPI5, which could run real software (like ROS).
I expect to have dozens and eventually many hundreds of nodes in a variety of facilities.
So far, things were going pretty well, though very sparse in terms of tooling. For example, I've got a makeshift midi controller (for physical knobs) that uses a Python script, debug connectors, and messy gnuplots to do PID tuning.
I got pretty pumped about the Foxglove Studio product and pulling on that thread led me into the (completely awesome) YT channel "Articulated Robots" (https://www.youtube.com/@ArticulatedRobotics), which led us down a path to seriously considering a move toward ROS2.
I wrote it off a couple years ago (2021/2022-ish) for a handful of reasons:
a) The move from ROS1 to ROS2 triggered my PTSD from my time at Google where everything had 2 versions: One that's deprecated, and One that's not ready yet. It also triggered trauma from Python's ongoing and complete disregard for backward compatibility.
b) It's obviously designed to run on Linux, but I have a bunch of low-cost bare metal micros driving around. What the hell kind of robotics platform doesn't (by default) run on microcontrollers?
c) CMake is already one of the absolute worst piles of trash I've ever encountered in nearly 25 years of software development. Wrapping CMake (twice now!?), which is already a crap wrapper is so obviously such an awful idea that it's tough for me to trust anyone who had part in that decision.
d) C++. No thank you. Slow builds are a non-starter. C is fine IMO.
Now, I've spent 4-5 days watching videos, doing tutorials and bringing up ROS2 topics and basically fiddling around with it and just learned that Google(ish) bought the core team, which has me deeply concerned about the future of the platform.
I'm actually still kind of interested in the DDS portion of ROS2, but the whole mess trips my spidey sense and has me feeling like dragons be lurking.
Seeing this thread with so many people speaking so negatively about it has me desperately wanting more information.
Can anyone point me to anything that presents concrete problems and (better) alternative systems that work?
Otherwise, concrete problems are a bit harder to talk about, because they're very diffuse: while ROS has a lot of components and tools you would want in a robotics system, and they are all integrated together, the average quality is very low: they are on average slow, buggy, hard to use, and tend to lack important features, from build systems to GUIs. While it's tempting to use a it out of the principle that you shouldn't be duplicating work, in general I would say most robotic applications are far better served by using a custom framework that better fits their system design, using appropriate libraries for each part, but not an off-the-shelf do-everything framework (and it feels like there are gaps that could make this less effort, but in my experience writing your own replacements for ROS components is less effort than dealing with all the problems that come with using it: it's shockingly low effort to reach feature parity).
(NB I haven't touched ROS for a while, but it doesn't seem like this has improved much, though there's been plenty of code churn)
We're digging into Zenoh as a potential "library" level integration.
Do you have any experience/thoughts on their tools?