Towards robot accelerators, democratizing hardware acceleration in robotics
news.accelerationrobotics.com
news.accelerationrobotics.com
Any given cell may be in production for 10 years - some only for 5, but some for 20 or more - and you need <24 hour response times to any mechanical issues that may arise over that duration. Also, one can't expect that the original programmer will be available to diagnose and debug issues that arise in the field after a year or two, so you need a large pool of technicians and maintenance personnel that can understand the software and be productive in the first hour after being introduced to the machine. None of this is valued by academics, instead, the fundamentals of the process of a single lab trying to prove merit as PhD students and, once that credential is acquired, move on to greener pastures is anathema to both of these ideas.
The problem is that academia is working in ROS (now ROS 2), writing Python and VHDL, while the real work in the industry is being done in proprietary vendor tooling that's backwards compatible with training from the 80s. They're proudly advertising touchscreens and full-color 5.7" displays on their teach pendants, as if their customers were still 20 years in the past.
Yes, academia, I do want sub-millimeter repeatability, and flawlessly smooth motion planning. I want collision detection that accommodates both the inertia of the arm itself and any end-of-arm tooling, including cable harnesses with distributed loads. I want automatic singularity avoidance and trivial point and frame manipulation. I want 24-bit encoders for resolution, and I want angular velocities to the moon, and to make those work together I need servo loop rates and high-speed skip responses at 1, 2, or 10 kHz. I want 4D stop position prediction to avoid intersection of the robot arm with complex assemblies and safety zones imported from CAD. I want all these things cheaper and faster, with more storage.
But I need these to be accessible by Billy Joe, whose only credentials that got him the job in the maintenance department is that he helped out his daddy working with a welder and an old clapped-out Bridgeport on the farm growing up. Billy is probably my best tech on 3rd shift, and he's never learned to type on a full-size keyboard, just his smartphone... and on a robot teach pendant.
In the PLC space, Beckhoff controllers using commodity x86/64 hardware and standardized IEC 61131 programming languages are gobbling up Rockwell and Siemens' market share.
Given they're perpetually about 20 years behind the times, perhaps this year someone at Fanuc, ABB, Kuka, and Yaskawa will catch up to the year 2002 and read Spolsky's admonition [1] that says:
> Smart companies try to commoditize their products’ complements.
Stop trying to turn every division of your company into its own product center. I loathe paying $800/year to Rockwell for the privilege of reading "update your firmware" (and subsequently bricking my controller as an unwilling beta tester) on TechConnect. They're a PLC vendor, they should focus on selling PLC hardware, not making their customers hate them by nickel-and-diming them for software licenses and support contracts. They were down for FOUR DAYS this weekend for planned maintenance. FOUR DAYS!
Likewise, Fanuc should focus on selling servos and castings - robotic manipulator hardware - and stop trying to get the training division to turn a profit. If, instead of a 40 hour course costing $2,000, it was free, there would be more people able to use their $40,000 robot arms! If Roboguide software was free instead of $2,650, clients could more efficiently build more robot cells, and sell more yellow paint! Gah!
[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
A very brief search turned up assorted overviews[1][2][3], and a github topic[4]. The PLCopen standard[5] is paywalled - only $400 single user, with low low per-seat multipliers! :/
[1] https://en.wikipedia.org/wiki/IEC_61131-3 [2] https://dc-us.resource.bosch.com/media/us/products_13/produc... [3] https://www.controleng.com/articles/which-iec-61131-3-progra... [4] https://github.com/topics/iec61131-3 [5] https://plcopen.org/iec-61131-3
We wrote about this, and raised-funding, and worked for years on it. Hell, we even created an extension of ROS for it called the "Hardware Robot Operating System" (H-ROS) that aimed to address many of these issues you describe using ROS as the common language and Programmable Logic (FPGAs) to deal with physical interfaces. More on this at https://ieeexplore.ieee.org/document/8046383.
Problem was that the market didn't really accept it. I still believe on the tech problem landscape in here. I'm just not so sure anymore if there's any real market/business for such a solution. After all, vendor lock-ins are great business, and generate business.
> it's legacy product support and ease of onboarding new hires.
I don't see how academia can help with this, this so very much an industry problem.
> But I need these to be accessible by Billy Joe, whose only credentials that got him the job in the maintenance department is that he helped out his daddy working with a welder and an old clapped-out Bridgeport on the farm growing up.
While accessible interfaces and forms of human-computer interactions are researched by academia, they are not building products. Research has to be relevant, but it's still research. Those problems, they might be better addressed by a startup than academia, if that's even realistic. Academia can't solve a problem if it's not a research-problem.
We do not need hardware acceleration in robotics. Can we use it if it exists? Sure, but do we care about "democratizing" it? No.
There is a place for hardware acceleration in robotics that is making a huge difference and that's in simulation. GPUs, the once and future king, make training robust neural networks possible and fast. If you're an investor reading this who needs a reminder, bet on software.
While I agree with you that in 50 years, advances in AI will make fast hardware obsolete, it may well be that in 5 years, fast hardware will be able to avoid problems that we cannot yet solve in software. That would mean this company has 45 years to cash in before they get replaced by software.
How much neural net can you run on silicon with 20 watts?
I strongly disagree with the rest and your comments indicate a clear lack of (hardware) understanding. I often encounter this in the AI world. Where folks building ANNs just "forget" that this computational abstraction only "grew in popularity" (there were previous accelerators though) when CNNs got implemented as an accelerator in a GPU empowering newer results. The same's likely to happen in robotics.
Robots are real-time systems. Meeting time deadlines in their computations is the most important feature. (robot) Behaviors often take the form of computational graphs, with data flowing between compute abstractions (Nodes), across physical networks (communication buses) and while mapping to underlying sensors, compute technologies and actuators. ROS enables you to build these computational graphs and create robot behaviors by providing libraries, a communication infrastructure, drivers and tools to put it all together.
From a more technical compute architecture perspective, ROS 2 presents an event-driven programming interface. The resulting computational graphs built with it are "event-driven software architectures". Mapping these event-driven software architectures to hardware using CPUs leveraging classic control flow architectures (von Neumann architectures) leads to various issues. A key challenge in applying classic event-driven programming is that CPUs hardly provide real-time and safety guarantees. The de facto strategy in industry to meet the timing deadlines is a laborious, empirical, and case-by-case tuning of the system. This “whack-a-mole” approach is being realized by some, but unsustainable and hard to scale due to the lack of a hardware-supported timing-safe even driven programming interface. This is where accelerators come in. As already adopted in other industries including aerospace, automotive and healthcare, through the creation of custom compute building blocks (accelerators), a hardware/software co-design strategy provides clean behavioral specifications, describing clearly its guarantees in terms of timing (in other words, avoiding memory-centric von Neumann architectures).
Hardware is important. Note the current scarcity of semiconductors, which is one of the drivers of this research. Note also that creating custom accelerators for robotics application not only can lead to performance improvements (i.e. software that runs faster!) but also to more deterministic responses (which affects the downstream pipeline of software!). Rephrasing Alan Kay's quote, if you're serious about robotics, you should care about hardware ;).
I wouldn't be surprised if they developed their own FPGA programming environment based on C++ and CMake, and all kinds of other custom language and tool shenanigans.
I started building a ROS alternative called Mech (see my profile) after becoming frustrated with ROS. It’s getting there but not ready for prime time yet.
To answer the sibling comment, my biggest problem with ROS is how inscrutable it is for undergrads. I’ve witnessed students struggle to even install it while following directions step by step. It is a huge mess of incidental complexity, layering the complexity of C++ build systems on top of an IPC system on top of an expansive middleware.
The learning curve is so high that when I spend a semester teaching robotics, students still struggle with ROS basics at midterms. The course turns into more about learning ROS rather than learning actual robotics.
I’m hoping one day Mech will work along side and in conjunction with ROS to provide an easier way of doing things for those who want to get right to the heart of robotics.
The issue with ROS isn't the general idea of communicating nodes over a blackboard and the accompanying standard libraries, but the huge pile of low quality tools, sparse documentation, ad-hoc architecture, bad code, and mediocre introspection tooling. A lot of that can be traced back to C++ and CMake, but ROS manages to take already complex tools and make them even more complex in an attempt to make them easy with automagic.
If ROS2 had just opted for ProtoBuffers/FlatBuffers/CaptnProto over MQTT/NATS and build TF et.al as a library over that, we could just point students to the excellent documentation of the Serialisation Format, of the Middleware and to the (with all that freed developer time) much better documentation of the Robotics libraries, and call it a day.
Instead they spend months with the idiosyncrasies of the build system, only to continue struggling with the documentation, and the sub-par tooling when something goes wrong. Cough only two seconds of time travel in ROSBags cough
But yes, I recognize the struggle. At my RoboCup team (TechUnited Eindhoven), the learning curve is also steep: from non-CS students that use Matlab on Windows, to using Linux with C++ & Python, ROS, git, not to mention our own stuff on top.
It's a filter, to say the least.
Maybe there's room for an Eve-Mech bridge where Eve can be used on the knowledge representation side, and Mech for the lower-level motion planning stuff. That's a fun thought!
A hybrid system is a really interesting idea. I actually wrote my bachelors thesis on a system heavily inspired by EVE to be used for knowledge representation in robotics and have been experimenting a lot with implementations of the idea over the years. You should really open a Mech discord channel, I'd love to hang around and help build something like that :D.
GPU/FPGA Mech has been my dream since like... forever. I started taking a crack at it here: https://gitlab.com/mech-lang/core/-/blob/gpu/src/bin/gpu.rs, but mostly I was just figuring out wgpu's capabilities in Rust. But that was months ago and I've since decided to push it off until after I launch what I have done already.
I have a private slack that I've been using to work with my students, I'd love to send you an invite. Send me an email at corey@mech-lang.org and I'll send it to you. I've been putting off doing a discord channel because I wanted to code up a chat app in Mech instead :P I think I'm almost there but I've been thinking that for years >_<. If I don't have one by the time I launch in October I'll probably bite the bullet and do discord then.
Check out industry adoption of ROS 2 in various industries including automotive, healthcare and warehouse automation. ROS 2 is (and will remain as) the de facto standard for robot application development.
The atrocious build system has stayed the same. People continue to add even more brittle automagic to the pile. (.e.g. automagic compilation of C++ to FPGAs)
The grown semi ad-hoc messaging semantics for services has stayed the same. Even if DDS was an acceptable foundation, adding a ROS message translation layer and ROS semantics on top of a perfectly valid schema language is bringing the same pitfalls that Object Relational Mappers have, in that you now have to understand three systems, the source, the compiler, and the target. And given that ROS1 had a 'wontfix' issue on the message schema checksums being computed wrongly (ignoring the cardinality of fields), which could result in your robot literally ripping your head of when it received messages from two different library versions, I don't trust anybody from the community to get it right this time.
The whole DDS vendor based ecosystem is antithetical to open source. The DDS standard is way too big, and a bunch of vendors meeting every couple years to demonstrate that their systems can send messages about shapes between one another simply doesn't cut it to prove correctness.
ROS(2) is an attempt to make C++ more easy by layering automagic and complexity on top. Most robot shops I know waste most of their time fighting that magic and complexity. The other ones don't use ROS(2).
ROS should have been:
* A bunch of libraries (not frameworks) to solve robotics, vision, and knowledge representation related tasks.
* A bunch of best practices to work with common middlewares, none of which are developed as part of ROS.
* A repository of competing data schemata for the above libraries and middlewares to facilitate interoperability but allow for experimentation.
* Dataviz libraries for common notebooks, e.g. jupyter, observablehq, that work with the common data schemata.
Instead we got one of the hairiest C++ projects and a bunch of shell scripts to open your default text editor more conveniently. Yay.
Yes Ros looks hood on paper but the moment you open it, it just screams academia and academics.
Also yes FPGA is faster for in general. But who is gonna program it ?
Regarding who's going to create the accelerators, enablers like this allow any ROSser to jump into creating their own accelerator from C++ by purely adding some metadata into their CMakeLists.txt files (which every ROS package has already).
On top of that, organizations are creating professional-grade kernels meant for production. NVIDIA and AMD are investing in that direction and so are we at Acceleration Robotics. Here're a few of the first ones announced: - Accelerated perception stack (ROS 2 API-compatible) https://accelerationrobotics.com/robotcore-perception.php - Accelerated coordinate system transformation (ROS 2 API-compatible, tf2) https://accelerationrobotics.com/robotcore-transform.php
From there, ROS is a step up! Both are programming environments designed by mechanical engineers, who care more about making easy problems really easy than making hard cases any better (after all, to a mechanical engineer, hard programming problems are really hard on their own).
There's a huge gap between robotics development (more sw-centric today) and the average hardware engineer in big semiconductor companies. Totally different perspectives and viewpoints. We felt bridging the gap was necessary.
When optimizing dataflows for lower latencies and higher throughputs, you typically seek specialized compute architectures and that's wherein GPUs and FPGAs come in. That's what this work really enables.
Results using AGX Orin are in the pipeline and will come out soon as well.