RoboCat – A Self-Improving Robotic Agent
deepmind.com
deepmind.com
On the other hand, I think a more promising direction is to raise the ceiling of robot arm manipulation sky high. OpenAI kind of did this with Dactyl but I would like to see more of it. Can we get robotic arms to tie a shoelace, knit, perform pottery etc (with an arm like morphology, no special mechanisms). I think this can actually then lead to large scale generalization, kind of what we are seeing happening with NERF's now. I would like a robot arm NERF, overfit to one hard task but reproduces it with human like precision and dexterity. Deepminds approach to me (with GATO, robocat) seems like a red herring, they will never reach the kind of results we want from our arms.
And about floor vs ceiling, what's really important is robustness, only robust robots can be deployed in the wild, at this time Dactyl with all its fingers is still too difficult to control, RoboCat got the grippers right, the problem really is they are again doing cute things with large models instead of raising robustness.
Now that AI based task planning can learn from so few examples and can do extremely general tasks like say the instruction:
"Summarize my gmail twice a day at 7:00 AM and 5:00 PM filtering spam and stuff I don't read" would spawn an agent with a plan to do exactly what you want. And if it could not you could show it once like "recording" a super smart "macro" only this time with AI agents instead of Selenium.
This would be an "inversion" of API way to do things. Pretty soon people are writing tons of these "bots" and multiple cases land in the some(supreme?) court over what is a bot and what is personal property.
If anyone could easily get an LLM to automate this, you can bet service providers will scramble to find ways to defeat it (justifying it with some bullshit reason like "security", so it's not blatantly obvious they're artificially constraining their product, to prevent you walking around their most profitable toll booths). In general, it doesn't matter what new technology is promising in the abstract - if anything, it's just marketing noise. The technology will deliver only the things its wielders can make money on, and will not deliver things that don't have a good business case. It's the standard tech industry bait-and-switch.
Want proof that this is more than just cynical rambling? Take a look at IFTTT, Zapier, et al. See what you can actually do with them. Notice that it's entirely limited by what the integrated services expose to those automation services. See how carefully those integrations are crafted, so that you can't exactly do much with them. If you want anything non-trivial, you'd better be a business, because individuals aren't supposed to empower themselves with technology. And if you're a business, well, anything is possible if you're willing to pay enough.
The few of us who know how to operate advanced automation tools, and whose frustration outweighs the cost of micromanaging brittle DIY hacks, can beat SaaS tools into submission. Sometimes. But only as long as it stays below the noise floor of vendors' financial metrics.
I had that exact conversation with Chamberlain about MyQ garage door openers. They make it difficult to integrate with and their stated reasons are all about security.
But they'll constantly try to get you to work with their (presumably paying) partners that have far more privileges than you as the lowly product owner.
There isn't a great business case for the latter, so it languishes.
If you are paying for a service, which adequately charges you at least how much it costs to run + margin, the service should not care whether you are giving them your eyeballs, and you are entitled to request features like automation friendliness and APIs and e2e encryption because you can vote with your wallet by changing suppliers.
If advertisers are the ones paying, and specifically paying for your eyeballs, then naturally your interests are at odds.
(There are edge cases where the service may in fact charge you, but 1) their billing structure does not let them cover operational costs, 2) they are afraid to alienate users by raising prices or otherwise hostage to current billing structure, and 3) they are incapable of optimizing performance to turn profit with current billing structure; in those cases crippled API and automation-hostile GUI may serve as a throttle.)
> If you are paying for a service, which adequately charges you at least how much it costs to run + margin, the service should not care whether you are giving them your eyeballs,
They may not care about your eyeballs, but their understanding of "adequate charge" is "what the market can bear". More specifically, if they allow for some automation, and that automation saves their customers a noticeable amount of money, then they'll try to capture as much of those savings as possible. This is business as usual - the entire market economy works like this. The problem with SaaS is...
> and you are entitled to request features like automation friendliness and APIs and e2e encryption because you can vote with your wallet by changing suppliers.
...that you can't actually vote with your wallet. Software resists commoditization, and SaaS companies are especially good at this. Whatever the service you use, it's highly likely there is no good alternative for it. There may be some competitors whose offering partially overlaps with what you use now, but it's highly likely there's some unique aspect that, combined with hassle of moving your data and overall inertia, will make switching service providers an act of last resort.
Those sticking points often aren't software-related at all! E.g. Netflix and HBO Max and Disney+ aren't really substitutes, because their catalogs do not overlap. Tech-wise, Spotify is mostly a shitty audio player (that gets worse every update) - their entire value was always in the deals they signed with the labels and artists. Etc.
Combine this with your typical SaaS targeting a global market, and you can see the power imbalance. They don't give a shit what you want. You ain't gonna "vote with your wallet". Hardly anyone does. And you're just one user of a hundred thousand, or a million - a rounding error. You're not worth caring about, not unless you're in a position to start a cascade of users leaving - which might happen if they try to screw you over and you're quick to Twitter, but which definitely won't happen just because they made their UI suck on purpose, or put basic automation behind a high price tier.
> 3) they are incapable of optimizing performance to turn profit with current billing structure; in those cases crippled API and automation-hostile GUI may serve as a throttle
That's a possibility in some cases, but I think that, if the "normal" mode of user interaction involves a website or a mobile app written using modern frameworks and development techniques, then it's probably a wash - increased utilization of the core service may be canceled out by reduction in the flood of requests their bloated GUI generates.
All actual services are interchangeable, or should be—and if not then there is network effect and/or user lock-in due to double-sided market malpractice. In all other cases there is a niche ripe for a competitor to come in and profit!
> increased utilization of the core service may be canceled out by reduction in the flood of requests their bloated GUI generates
I think in most cases a swarm of automated requests to core service would outweigh overhead from a front-end GUI, architecture must be really mismanaged otherwise.
> You ain't gonna "vote with your wallet". Hardly anyone does. And you're just one user of a hundred thousand, or a million - a rounding error.
Why not? This is a defeatist attitude that either ignores how markets work or assumes users are ignorant and never learn.
People do vote with their wallets. Once bitten, you choose services with full export functionality, start reading ToS and looking at who runs it. Word of mouth helps people make better decisions and pick better vendors, even if those vendors are more expensive. (The only problematic ones are “free” vendors, for a customer making a jump from 0 to N is much more difficult than from N to M. So we’re back to ad-supported double-sided markets ruining everything.)
And yes, there is information asymmetry, but there is also regulation that could compensate for it. For example, the government could require services larger than certain size to provide fully featured APIs.
Yes. I know that's absurd. :)
Now that I'm leaving Academia I was trying to figure out how to get a robot arm so that I can continue doing stuff at home. I think an approach like the one here would be good: https://www.trossenrobotics.com/aloha.aspx. A single robot would cost 5k. You could also just buy Dynamixel servos and build it on your own, shouldn't be too hard with 3D printing ~ closer to 3.5 or 4k with that.
You could try to go cheaper by building a dynamixel servo on your own. I would not recommend this, the main difficulty is the controller which requires a lot of tuning, which is what you're paying for when you buy Dynamixel. But the raw parts of a servo are only around 50-100$ I'd guess.
Lastly any arm you build this way is going to be position control which is very different from the human arm that is torque controlled. Trying to get torque control on your arm is all the rage on Youtube these days, perhaps because the motion seems so lifelike. To do that you need to build a Quasi Direct Drive Actuator which is a BLDC servo with a low gear ratio and then try to centralize all the motors to reduce inertia. There is no accepted solution to do this, but the best results come from cable driven mechanisms like AmbiDex. Basically building a human like robot arm is a huge research problem on its own, and if you want to focus on intelligence best avoid this.
Is it really just tuning? That seems like something that would pay off for an aliexpress copycat. A servo is pretty simple/cheap to build, right?
I guess there are only a handful of us who really want high-accuracy, high-torque component servos...
Beyond that, it's DIY, with a set of strong servos and a few budget microcontrollers/motor controllers, from Alibaba, and...a 3d printer to manufacture the rest, for something like an arm.
Don’t think it’s cheap nor a “platform” but maybe it’s enough to get you started ;)
He just recently put out a video on making a carbon fiber arm piece for it.
Interesting, but this doesn’t sound like something that can be useful. For practical industry applications, you need high success rates. Assume you are running operations where dropping/flips/damages to the items that a robot handles are costly and not acceptable.
Its only for a hobbyist pick-and-place, dropping a Lego block or wasting some components is not a big deal.
Alright, let's assume we're not too though, just for completeness sake.
Plenty of industrial applications where an 86% success rate is fine, especially if you can try more than once.
Sounds like a disaster?
Metal and plastic Stock
Castings and billets
Clothing
Bulk sacks of material
Lumber
Items to be recycled
Most consumer goods after packaging
Hand tools (wrenches, pliers, etc)
Most food items
Such operations are rare and in those cases you're probably not picking them out of a bin to start with. For most items dropping or flipping the object a few times is perfectly acceptable, indeed that's probably how they got into their current position to start with.
Are they really?
This capability will help accelerate robotics research, as it reduces the need for human-supervised training, and is an important step towards creating a general-purpose robot.
The future people are striving to build just seems so fucking creepy to me. I don't really understand the enthusiasm for it. Maybe I would if I saw said future and maybe it would be awesome and I'd regret saying this, but right now, I just don't get it.
Sometimes risk needs to be taken.
This isn’t demonstrating good robot safety, someone will get hurt, and likely already has been, at least to minor degrees.
"robot can plant, water and harvest food that only poor laborers from other countries did, replacing whole job categories in agro business along with other things like making blue collar immigration moot"?
EDIT: page says that's not the real price, but https://www.ground.ghostysky.com/en/product/unitree-b1/ claims 70k€ so who knows.
That is DeepMind succeeding on building robots that can build robots.
If someone from twenty years in the future appeared in my living room and informed me with certainty that robots are 100x cheaper in 2043 and asked me how that could be true, my first guess would be that there are significant energy breakthroughs making aluminium and lithium much cheaper as raw materials, and that robots took off as a very popular consumer product and are now sold in quantities between 100 million and 1 billion units every year, driving prices down through economies of scale. I'm not certain that would be enough on its own to get prices down that far, but it would be my best guess.
let each flourish
explore
new territory
and then they rejoin
bits and bytes
entertwined
making a new kind
And is some of the learning from observing other cats?