Salute to a real robotics veteran. I can't imagine the hardware gore you have seen.
181 karma · joined December 9, 2020
http://www.spieswl.com/
I believe your intuition about layout experiments needing to be of different genres is correct. I think you could have a pretty wide range of debugging opportunities (imbalanced belts, inserters fighting for items, insufficient power at full load leading to throughput loss, etc) for the first. The second feels like it would be nicely encapsulated by focusing on optimizing for ratios, although seeing an agent realize that they can get more throughput by simply copy/pasting a block and upgrading a belt would be pretty wild (depending on the recipe, of course). Maybe nuclear power / heat exchanger ratios are a bit too far down the path, but optimizing for copper cable use in green circuits is pretty important and fairly early in the tech tree?
It seems like there are a lot of interesting experiments to be had here. The lab-play scenarios having a time-related component seems like a good idea, I assume most Factorio players that keep biters on treat them as a combined temporal-spatial constraint, so you have a sort-of proxy comparison to a real game situation when you put the agents on a timer.
I like the way that the framework design is testing different things than micromanagement proficiency, such as what we have seen in DOTA 2 or StarCraft 2 experiments. Notably, severe worker micromanagement (in the case of the latter game) becomes a way to squeak out extra minerals when you have infinite APM available. This is an interesting learned behavior in a narrow context, but that tactic is really control intensive and has a high chance for even pro players to screw it up when attempting to do so. It also doesn't seemingly give additional insight into an agent's longer-term planning, execution, and analytical performance. FLE seems way more interesting as a higher-level "thinking" evaluation framework, with all that in mind.
Any plans for layout optimization benchmarks? As in, start with a given factory cell with X inputs and Y outputs, and optimize its performance.
Quick aside: Automate small/mid business manufacturing? Admirable, but will probably choke on the scale problem, so I think the journey will be vastly more interesting than the destination...which is good! Turns out there's a lot of robotics that can be broadly applied.
* Manufacturing --> robotic manipulation, controls, actuation, sensing, decision-making
** Robotic manipulation --> Linear algebra, likely Python and/or C++, some simulation tools
** Controls --> Manipulator platform, drivers, physical- and protocol-level choices
** Actuation --> You want to pick something up right? Air powered? Electric? Probably not hydraulic, but worth mentioning?
** Sensing --> What's your sensor suite? RGBD? LIDAR? Forces-torques? Combination of the above?
** Decision making --> FSM? Behaviour trees? Purely functional?
Repeat the decomposition and you'll probably be able to get down to the basic level of the robotics hierarchy of needs.
How things are made is so important, not only for our society but also as learning experiences for engineers, planners, logicticians, and more. As a career roboticist, the time I spent in the manufacturing industry seems invaluable to me now.
[0] https://news.blizzard.com/en-us/article/23053098/starcraft-c...
The visualizations are so similar to integrated circuit layouts; they immediately reminded me of some of the coasters that GamersNexus sell which represent simplified computer subsystems.
How much did you learn from the lessons of other contemporary robotics frameworks that are out there? Do you envision focusing in on particular types of tasks later, or is it still uncertain how your robot design will evolve as the dataset grows?
- How much are you able to use the robot's internals to estimate the gripped bag's inertial properties? If you're trying to put rigid, heavy bags below light, amorphous bags, are you adjusting final placement location on-the-fly?
- How dynamic is your scene beyond what we can observe? If you're using light curtains with a single robot on a track, and you're able to estimate some rough geometry of and track the bags down the conveyance, and you are updating occupancy of the bin as you're going, what else is there?
- Is this just inside for the foreseeable future or are you all going to tackle unpacking outside, as well as all the, ahem, baggage that comes along with operating outside the terminal walls.
Nice and straightforward problem, relatively speaking.
I think the best thing for learning robotics looks pretty similar to learning a programming language: Have a specific task in mind that the robot/programming language will help you solve. Even if its just a pick-and-place and a camera, or a shaker table with a camera over top, or a garden watering timer/relay combo. Just work on something specific with your toy robot and you'll naturally encounter much of the difficult things about robotics (spatial manipulation, control, timing, perception, drivers (GODDAMN DRIVERS), data, you name it).
Going right to a high-DOF arm or trained LLM is always cool, but the person who hacks together a camera/relay/antenna to automate some gardening task or throws some servos and slides together to make a Foosball robot is doing the most interesting things, in my opinion.
The hardware has really made strides too, easy and cheap sensors, controllers, cameras. It's awesome how quickly someone can plug-and-play a Realsense with a servomotor or pneumatic slide and start manipulating the world.
The thing that's usually underappreciated is that once you understand how to code a robot, you are barely closer to having solved a practical problem. There are lots of practical problems in the world where spending 4 hours learning how to use $PERCEPTION_API would be actually better spent spending 4 hours understanding more about the widget being perceived or the object being manipulated. Getting into robotics has never been easier, getting something useful out of robotics is still the trick.
You are 100% correct about the tooling being crap and ladder representation being useful as a visual mapping.
[0] https://library.automationdirect.com/wp-content/uploads/2013...
[1] https://multilinedesigns.com/wp-content/uploads/2018/02/IEC-...
Anywho, cool, robust, reliable things still got made, you just had to think differently. It was very interesting to think about those programs and their relative programming simplicity when compared to complex, modern C++ that is running in other codebases I've seen since then.
Maybe later when my focus or responsibilities have changed, but that runs the risk of keeping a library of curated content that I can't validate at the time I capture it (_because I will have to validate that content later when it aligns with whatever-later-task-is-relevant_).
And this says nothing about the creativity that's sapped by the organizational inefficiency, the clumsy handoffs, the incomplete understanding of what is truly wanted and/or needed at the end of a creative endeavor, and all of the other things that are the sources of hundreds or thousands of cuts at your psyche.