Disney packed big emotion into a little robot
spectrum.ieee.org
spectrum.ieee.org
The impossible parts of problems could be solved by visual sleight of hand or some artistic trick, leaving you open to explore and solve some really interesting application specific problems with technology. The much, much harder part is making any of this research work robust and compelling enough to actually show in the parks.
The uncanny valley problem?
This intrigues me even more.
I was at Disney for close to a decade, and I don't think I could've gotten a better education in how engineering acts in service of the experience the user is having. Disney is really a master of this. Maybe Apple comes close, but I wouldn't know.
Of course the “easier” solution was to put those funky eye stickers that move on ours. (Sorry I’m not sure what they are called, jiggly eyes?) My daughter also painted little hearths on it and gave it a name. Not for any particular reason, just for the heck of it.
https://en.wikipedia.org/wiki/Great_Moments_with_Mr._Lincoln
(Oddly, Phillip K. Dick's We can build you featuring a Lincoln simulacrum was written in 1962, a few year before Disney's Lincoln)
The emotional impact of that Lincoln was carried by the voice actor
https://en.wikipedia.org/wiki/Royal_Dano
working under Disney's direction. As others have pointed out this is an example of where a system like this can be carried by getting a few things really right and faking the other ones well enough.
Disney
Why would that be odd? It's very common for speculative fiction to have a version of something before it is realized in the "real world".
I would though think that this Star Trek episode must have been informed by Disney's Lincoln
https://en.wikipedia.org/wiki/The_Savage_Curtain
even if there is no explict connection drawn (everybody knew about Disney's Lincoln then, it was talked about a lot) It is mentioned specifically in the 1967 film
https://en.wikipedia.org/wiki/The_President%27s_Analyst
where the protagonists are being lectured to by an animatronic figure and once character says "He's like Abe Lincoln" so there is some expectation at that time that people would know about him.
Then you have two pop culture references to an animatronic Lincoln being released within two years of each other.
That is odd.
I would agree with you if the book had been out longer before Disney immitated it.
The nice thing is that it doesn't even have to understand perfectly, a GPT-4 level of vision/language understanding would be perfectly fine. It doesn't have to hold a conversation, just be able to be happy or sad when it should.
What I would bet we'll see at first is some of these little guys in Disney parks, in fenced-off areas or small stages that are kept meticulously clean by staff, where they can interact with visitors without needing to handle all the complications of the real world yet.
Dialogue is the area where the interactivity of video games falls down completely. I mean, dialogue in a video game can be just as good as the dialogue in a movie but it has to be "on rails" because of the lack of linguistic competence of today's computers.
Even with a much better system it has to be "on rails" in that a video game character will get in trouble if you get it to enough an extended conversation which can be somewhat answered by "people have various ways to set boundaries" and that that of course can be part of the characterization.
Y'all are probably sick of me talking about how Tamamo-no-mae in the game Fate/Extella is the pinnacle of characterization in modern games, but being based on a legendary character who is able to charm people by talking intelligently about any subject who is connected to a photonic crystal computer that has recorded all the Earth's history and having the relationship she does with the protagonist any answer she has to why she doesn't realize her promise would be terribly disappointing.
... a Disney of the past would build four of these and just have them wander around EPCOT's Future World, just to set the tone. This Disney will probably purpose them for their Star Wars experiences. Still great to see.
Whether a toy market is a trillion dollars is questionable.
The Achilles' heel of autonomous mobile robots is power storage density.
I am always fascinated ever time they release a research video (I remember the one talking about snow simulation for Frozen).
It seems like it could be a fascinating place to work and (from the outside) seems unique given their priorities. It isn't to release a consumer product, a new b2b project, its to support some other narrative purpose.
I do find it fairly fascinating that at one point in the video I was watching it and figured it had to be CGI. I wonder if that is just still not used to this on a regular basis.
Side Note: This robot is CUTE and I want one.
He is, in fact, adorable.
Like ok I get it, Disney isn't going to paint up a prototype or reveal too much of their plans with something like this likely in the works years in advance.
This wouldn't be the first time they showed a prototype that was rather clearly for a specific purpose (like the spider man flying robot)
But uh... that is 100% Star Wars.
Now, if it could vaccuum a floor, I'd buy one!
But from the outside looking in what is "Disney". I would wager that for the vast majority of people Disney is Mickey Mouse, Frozen, Lion King, etc. basically media. Obviously I would say that people would also say the parks. But I doubt the word "Tech" is anywhere in the vast majority of people's opinion on what Disney is.
It is wild to think that behind what is a media company, is an organization with strong engineering talent. I mean it seems like that the robot engineering specifically rivals most of what else is out there.
That isn't saying there are not more advanced robotics, Boston Dynamics projects are a big example.
But there is also something to be said that this is probably going to be in a park in just a few years, if not sooner. It isn't going to sit in research stage forever.
But it is a really weird timeline that Disney is associated with advanced robotics and other crazy things that Disney Imagineering does.
Hollywood was the Silicon Valley of the 20s and 30s. They were using and inventing cutting edge tech!
I love applying this to our technology companies. Sure, we get products like Google Search and Gmail and Facebook and Messenger and Instagram and Tiktok. Everybody calls them tech companies. Like everyone assumes Disney is media. But in reality they are ad companies. Like Disney is a theme park company.
Disney himself was a huge fan of technology and both him and the CEOs after him recognized that there's a part of the industry they're in (entertainment) that requires secrecy; literal "magic" is mostly "things you can do that other people don't know how to do." So they keep that research in-house because they want first-mover advantage on illusions, effects, and experiences that nobody else can do.
That's the best way to conceptualize Imagineering: it's a magic factory. In that sense, absolutely essential core-business-model stuff for the park-and-show entertainment sector of the company.
(Some of the more recent CEOs didn't grasp this aspect and actually did outsource some of the work done in the past half-decade. I'm going to be real interested to see what the park scene looks like in the Orlando area in the next decade or so as the technologies third-party vendors developed on Disney's behalf diffuse directly into Disney's competition).
Disney’s FY2022 operating income was $12.7B. $7.9B (!) of that comes from theme parks. That’s 65%.
Everything else Disney does (Pixar movies, Disney movies, Star Wars movies, television, streaming, touring musicals, merchandise, partnerships, IP licensing) is the remaining 45%.
Robotics is essential to the parks.
That's why I don't care one bit about box office stats for any Marvel or Star Wars movie. It could literally sell 0 tickets and still be worth it for the downstream effects on merchandising and promotional tie-ins.
Robotic platforms are a few iterations away from it at most
Prompt engineering is an extremely simple form of behavior authoring. I'm curious about how you arrived at the conclusion that newer models make prompts and prompting strategies less relevant.
> Researchers from Microsoft tested GPT-4 on medical problems and found "that GPT-4, without any specialized prompt crafting, exceeds the passing score on USMLE by over 20 points and outperforms earlier general-purpose models (GPT-3.5) as well as models specifically fine-tuned on medical knowledge (Med-PaLM, a prompt-tuned version of Flan-PaLM 540B).
[0] https://en.wikipedia.org/wiki/GPT-4#Medical_applications
That’s a pretty concise description of behavior authoring that functionally work
I don't know what it was about those fellows but I felt emotionally attached to them. It was not that they had any visible means of conveying emotions. They just went about their tasks quietly and diligently.
For some reason they pulled at my heart strings. It was quite heart wrenching to see them destroyed in the last part of the movie.
The movie was made by Douglas Trumbull, otherwise famous for his special effects contributions:
Yup
It feels like it knows that it has to do it this way, but it's something you wouldn't expect from any other robot, like Boston Dynamic's Spot or Atlas. They'd know that they have to recompute after the fact that they're partially on the ground where one leg still gets pulled away while the other one is already on fixed ground.
The right foot was on the floor while the left was on the rug, which was in motion and and raised delta the thickness of the rug. In such a mode, the bot is inclined to prefer the stable right leg and lifts the left leg. (I wonder if it can balance on one leg indefinitely? If not, that makes this sequence even more impressive.) Putting the center of gravity over the stable leg increases the probability of the bot staying upright. As opposed to leaning or shifting weight with both legs remaining aground, lifting the unstable leg entirely from the moving surface maximizes the goal of staying upright by minimizing the force of friction. I also think the rug was jerked at just the right moment such that it maximized the visual impression.
This explanation avoids the need for the bot to predict visually the impact of the rug pull, since the rug pull was not instantaneous and the bot’s reaction came after the rug pull began.
This is a very impressive demo. I do think they nailed the goal of expressively and emotion.
https://www.cnbc.com/2023/05/18/disney-scraps-lake-nona-flor....
It has a four-degree-of-freedom head (able to look up, down, around, and tilt),
I'm not seeing any way that up/down, left/right, tilt-left/tilt-right are 4 degrees of freedom.
It seems like that has to be either 3 degrees of freedom or 6.
How can up and down be two separate degrees of freedom, but lefr and right only be a single degree?
Not my area is expertise, but I can't see any way for the math to line up.
So we got four: lift head, rotate head, tilt left/right and tilt front/back?
Interesting.
It of course has a lot of human elements to get at that association. The antennas as ears. Having eye-likes that can close. The tilting of the head is very good looking and I must say it holds itself well.
Oh, and Aldebaran Pepper are easily found being sent to recycling. Because the onboard Wi-Fi has unfortunate tendency of showing early faults, of course. Totally not because it’s not about having human features.
Apparently the software used to convert CG animations into motions for this guy is part of novel development too. That’s what allows him to have these hallmark Disney movements. Are there open source or hackable alternatives to that? To what…
I am curious if the two robot monkeys forget all about the history of the ladder.
Will the future not-robot monkeys discover the truth?
I am always amazed that I am not upset at the computer but at my failure to operate it in a way to get the desired outcome. The ability to generate emotion exists within the imageo and not the observed or learned.
I was thinking the exact same thing while pondering the parents assertion.
Qualia is just the notion of being something. The classic example is a bat: what is the qualitative experience of being nearly blind, but having wings and a highly accurate echolocation system?
Plants too have qualia. What is it like to be a tree. To feel the nurture of the sun, the change in seasons, and to heal a broken branch.
The qualia of a stone is less exciting, but we can still imagine being a dense pack of minerals. Perhaps rolling down a mountain and losing some weight. Suddenly finding ourselves on the ocean floor, drifting aimlessly, while providing cover and nutrition to all sorts of life. After staying stationary for thousands of years.
That rock has seen things, man!
Down at the atom level things get more excited. What is the qualitative experience of interacting with other atoms? Or a photon? Of being entangled? State transitions?
Anyway. It's clear to me that this robot absolutely has qualia. I can imagine what it's like to be be a circuit board, with an operating system, a bunch of sensors, a neural processing engine, and a hard shell. How it feels.
It makes me appreciate being human more. But also evokes empathy for that adorable lump of silicon, metals, and plastics. Don't let the GP comment bring you down, little one. You are perfect just as you are.