How Google built its Gemini robotics models | Hacker News Reader

How Google built its Gemini robotics models

blog.google

205 pointssimonpure86 comments

They can do that, yet somehow, Gemini Assistant on Pixel phones still fails to reliably set timers or add shopping list items :-)

(which worked fine with Google Assistant)

Bring up dates and times if you want to wreak havoc on any AI. :D

Developers around the world's most beloved topic, how to handle date and time correctly, is still a topic of great misunderstanding. AI and AI agents are no different from that. LLM seems to help a little, but only if you know what you are doing, as it usually needs to be the case.

Some things won't change so fast; at one point or another, data must match certain building blocks.

One would think the arcana of time zones and the occasional leap second would not interfere with an individual setting egg timers often enough to become a burden

People ask why AI will exterminate human kind.

The answer is because we wouldn't universally adopt zulu time.

But this feature was already implemented in normal google assistant

Supporting existing projects is the oilsand mining of the promotion world. Low, old buzzword content, little reward. Implenting new buzzwords, wirh streetcred rich frameworks , thats the fracking.

Efficiency ,capabilties or customer satisfaction are irrelevant .

Except that's not the problem. its basic comprehension of requests. They aren't getting the wrong time, they try to play music, or the phone says "no timers playing" while the google home WILL NOT STOP until you lock the phone. etc.

Its basically an embarrassment for a project that's been alive this long from such a major.

Google AI Overview incorrectly identified the day for a given date due to a timezone conversion issue, likely using PST instead of IST. ChatGPT and Perplexity provided more accurate and detailed responses.

I detailed it on my blog here https://shekhargulati.com/2025/03/23/google-ai-overview-has-...

Do they fare better if given the task to sumarize a use calendars hostory first?

My own hands and cheap alarm clocks, or a piece of paper, have been working reliably for several decades. They also don't stop working when a corporation decides they want to hype something.

Phones today cannot even reliable handle things like "remind me to pick up tomatoes next time i am at a store"

google knows perfectly well, where I am and wants me to add 'infos' to locations and businesses the second I arrive (just got a notification today), but reminders like these are unavailable.

The location based reminders sure worked perfectly fine many years ago, like when I had Nexus phones. It's just getting worse all the time, I don't get it.

I use voice dictation a lot on my phone. It has degraded over the years, but is now basically broken, it fell off a cliff a short wile ago.

And worked with Samsung Bixby. Gemini, even after getting Advanced, is just terrible for a phone AI. I need to set a lot of alarms and calendar events, I don't need to do crazy photoshop (which Gemini is admittedly good at).

Or tts a web page

The "how" is completely missing, but if they can get this to work semi reliably it will be ChatGPT x100 in terms of impact.

I had never heard of Unitree (Chinese robotics company) before today. A lot of their videos look like CGI but apparently the product is real.

What stuck with me the most browsing their website on the G1 model was seeing "Price from $16k"

Now I'm not sure if these are actually purchasable or what the value would be, but it's my first time seeing an actual normal-ish price attached to a humanoid robot that seems to be for sale.

With the rate of advancement we're seeing across the board, it honestly feels like people will have robot assistants at home much sooner than I thought.

TBH, I still wonder if some of their videos are CGI. They offer real versions for sale, but they seem to be significantly more limited than the videos imply.

Have they actually demonstrated the more dramatic stuff at any in-person demos?

It is funny that this is getting downvoted. AFAICT, the answer to my question is actually "no". :)

I'm really shocked tbh.

I can't imagine the progression of ai and in particular robots but I assumed that the first robot would cost min 6 figures if not 7 but would still be worth it due to 24*7 and initial invest vs long term.

But the fact how good Gemini robotics is already and how cheap the first models are I do believe what will hinder us more than tech is people learning about it, testing it out and doing it but not technology.

I believe the world will look relevant different in 10 years.

Take any of these videos with a grain of salt.

In demos these robots only need to do well once and it can take hours to record.

In real life, a failure rate of 80% is unnacceptable, but perfectly fine to edit out in the final cut media.

I hope they do well, this area is incredibly hard, but it will take a lot more than what people imagine.

This whole hype cycle man. It's all shiny demos and no real products.

I just want the consumer grade robot dog so I can program it to chase the roomba around.

>A lot of their videos look like CGI but apparently the product is real.

I bought their robot dog as part of a project to build embodied AI models back in 2022.

Their SDK was far more open than anything else on the market and the stock firmware was on par with competitors, this includes products that were x10 the price.

The robot itself scared dogs in the park, but kids loved it. At $3k it's on par with a mid range drone and quite fun to hack on.

Same price as some pure bred dogs too

The humanoid is $20k-ish without hands. Each hand currently costs another $20k (and not sure if these are available to everyone or only for research).

Their robots are real enough, and they are well designed.

You can see more details in this video "Tearing Down the Unitree Go2: A Robotics Expert's Deep Dive":

https://youtu.be/YjVbW6Fc11Y

I own three of their systems, one since 2019

They are far and away the leader in quality affordable robots that are able to be used in real life at a reasonable price point

You can hack on em and add a lot of capability

I’ve worked with a lot of robots and they are leaps ahead of everyone, and have a good supply chain that can actually deliver

All of their control policies are developed with RL too so they are improving all the time

I am hoping they keep lots of their work open source. This is especially the case since hardware would be too expensive for competition to pull off, but it would be interesting to see how they circumvented some problems

it's a common trope in blogs - "how we did X" means "we did X, it's a good thing, we're great people", etc.

Even if Google's robotics technology (software and hardware) is leading edge does anyone think they'll actually be able to productize it? Seems similar to how they were the pre-product leaders in transformers and then fumbled any advantage they had to ChatGPT. It seems like something's missing from Google where they can't get from research to product effectively. Waymo perhaps a good counterexample if you think where they are today is product/market fit, but I can't shake the feeling that Google more often than not can't seem to get things to market or even if they do they give up on them before they take hold.

Just wondering if anyone has a strong feeling or, better yet, insight on this regarding their robotics efforts.

I agree. The current leadership of Google (especially Sundar) is mediocre and comes from a consulting background. They will fail at making a tangible product out of this, similar to glass or Inbox or a multitude of other examples. This is particularly sad, as I know a few remarkable engineers at Google that share this frustration. However, Google's leadership folded to Indian managers and is now run as a circus

Sundar Pichai got into IIT Kharagpur in the 90s (one of the toughest engineering/technical school to get into). So he has more technical chops than many self-proclaimed engineers that seem to diss on his McKinsey credentials

A school being tough to get into due to an abundance of population (too many applicants) means little. There has been not one significant person coming out of those institutions (compared to the many figures coming out of the US, despite a population 5x smaller). I would not hire Sundar as a junior engineer in my team. Of course, you might see something in Google's current leadership which I do not see. Time will tell how performant the company will be long term. Again, I believe skills that make one successful in consulting rarely translate to success in the engineering field.

> There has been not one significant person coming out of those institutions

Of the IITs?

Co-founders of Sun Microsystems, Flipkart, Ola Cabs, Infosys, Zoho, HCL

The only one significant is Khosla of Sun. The rest are... rehashes of Western companies without a shred of innovation or doing anything novel.

Look at the sheer number of institutions and people, and now compare it to any US institution (Maryland). Despite how much smaller the population would be, the accomplishments speak for themselves. I would take an engineer with a bs from any R1 university over a graduate with experience from those institutions any given day of the week. But of course, there will be exceptions. I was responding to a defense of Sundar which rested on the institution he went to

> The current leadership of Google (especially Sundar) is mediocre and comes from a consulting background.

That was what you asserted. So the GP just simply pointed out that you were wrong. You were wrong because Sundai didn't come from a consulting background. A lot of people from consulting companies have engineering background.

So you could rephrase your original comment: "not one (of course with multiple exceptions)"

Seems doubtful. His entire time at Google was doing project management stuff, so that is probably where all of his skills lie.

Can anyone with access to google3/ tell us if there is even a single commit by sundar@?

No, you can only access google3 by default if you are a T ladder FTE. This rules out almost everyone who isn't a SWE, SRE, PM or other affiliated technical role. There are exceptions, but ... they're exceptions.

I think the cautious faction of AI debate won temporarily inside Google, letting OpenAI take the lead. Lessons should be learnt from that experience. I do think Google will come out ahead in the end Gemini and Gemma are great models.

Let’s see what Google I/O shows of this year, product application matters now that they have caught up on the tech side.

Will be interesting to see if that lesson has been learned. There's no existing product they could cannibalize with their robotics effort (vs search with LLMs) so any caution they have launching a robotics product would solely come down to fears about quality/safety.

In my bubble it's general consensus that Google - as we knew it - is done.

Sergiej and Larry phased out and what is left is more of less a headless chicken, too big too fall, but without any clear direction and goal.

At least you are aware that this is a bubble.

Didn't Sergey become active again with the latest AI efforts?

Yes, and there was a leaked memo from him recommending that everyone at Deepmind work 60 hours a week and stop building "nanny products"[1].

[1]: https://www.theverge.com/command-line-newsletter/622045/goog...

Is this actually true about Larry and Sergei? A substantial amount of their net worths is still tied up in Google stock. I realize not all centibillionaires are cut from the same cloth, but still find it hard to believe they wouldn't be majorly involved since the downside of a major stock drop would impact them disproportionately. That said they could be the types of billionaires who actually think they have enough even if their net worths were "only" in the tens of billions (a long ways to go down from where they are today).

As for headless chicken, I feel similarly, but then I sort of see a path where they have defensible businesses in YouTube and maybe GCP, and then Waymo and robotics as green field upside, so that even if they don't end up with material market share with the "software-only" side of AI, and search gets further and further eroded, they could still be a formidable player.

Ultimately I do think their best days are behind them largely because they can't seem to turn the work of their talented engineers into great new products.

> they can't seem to turn the work of their talented engineers into great new products

I think about this a lot. The most famous example in tech world is obviously Jobs and Apple. And it's a great example because it's a borderline scientific experiment where you can directly compare three different phases.

But I think about it in a broader scope - like how many companies can last generations and remain relevant? There are plenty examples outside of tech, like banks but that's basically the same product and it's not that easy to launch a direct competetior to Bank of America or BMW, wheras software constantly evolves, people iterate on exisisting ideas I can I think of, from top of my hand, handful of examples of software that was really impactful but is not anymore.

Realistically person need just some XXXM to cover 99% of luxury lifestyle, and even stock drop 20x they will still have those money.

Also, Larry has some sickness, maybe his thoughts currently not about money at all.

What is your bubble?

Waymo?

Good point. It took a while to get going, but the growth rate is starting to look really incredible now.

The growth rate is still very far from incredible. And they've made terrible decisions on platforms, giving up developing their own in favor of the i-Pace which is discontinued, then hitching themselves to a Chinese EV manufacturer subject to 100% tariffs (which, fun fact, were imposed by Biden).

Growth was well over 100% last year, with expansions in every market (Phoenix, San Francisco, and LA). In 2025, they've added Austin, Silicon Valley, and will soon be adding Atlanta and Miami as well. They've also partnered with Hyundai on vehicle production, though I do expect that they will eventually become supply constrained.

They went from offering 12,000 rides in August 2023 to now offering 200,000 rides per week.

This growth only doesn't look incredible because of how small the starting point was, but it will be hard to ignore by this time next year.

Waymo is the exception that proves the rule. Waymo feels more like "Google owned" than it feels like a Google product.

They are very much google through and through.

It's terrifying to think that robots like this will probably be used in the defense industry at some point. If the robot understands something as general as "put the erasers away", imagine "kill all enemies".

Whenever I watch videos of robot dogs climbing all terrains (e.g. https://www.youtube.com/watch?v=AS1n99yruVU), I'm convinced that half of these robotics companies are defense funded, whether in the US or China.

People are always talking about AI in terms of economic impacts (jobs, productivity, etc.), but military should be the first use-case they care/worry about.

They'll probably freak out once they finally realize the implications of cheap drones + smart AI + auto-aim guns.

> They'll probably freak out once they finally realize the implications of cheap drones + smart AI + auto-aim guns.

Hopefully in the US. People seem to not care about the development toward AI controlled weapons everywhere in the world.

https://www.france24.com/en/live-news/20240210-israel-deploy...

https://www.euronews.com/next/2022/10/17/israel-deploys-ai-p...

Exactly. Robodog that costs 5K USD, which is probably less than one month of cost of the infantry soldier, and can be sent to fight in the trenches is something generals would dream of. Ukraine, that is experiencing soldiers shortage will go in that direction, at the beginning with something simple like remotely steered land drones.

People are remarkably cheap and versatile especially if you happen to don’t give a fuck about them. I find it hard to imagine a high-tech incredibly unreliable complex robot to be of any value in the battlefield. Even my phone is remarkably unreliable, have to keep it charged, bugs, etc. I noticed this during COVID and its QR-scanning phase at restaurants.

A simple pipe bomb or two will make short work of any incoming monstrosity.

What they need is simple, simple tech, cheap and lots and lots of it. Basic drones and RC cars rigged with stupid bombs will accomplish 90% of what fancy robodogs can do at a fraction of the cost.

A robot dog hiding in bushes for reconnaissance is significantly more energy efficient than letting drones loiter in the air.

Killing people is already a solved problem. Use mortars, call air strikes, send drones, let a heli shoot 30mm rounds, etc.

Why you need a robot dog when some cheap RC car with a $2 camera will do. Anything you can think of that a robot dog can do some cheap bullshit will do better. I am totally not seeing the value of these things.

> It's terrifying to think that robots like this will probably be used in the defense industry at some point

So what? Is keeping us safe a bad thing somehow? I can't get these people who reflexively think anything weapon-shaped is evil. Violence is good sometimes.

It’s terrifying to think of a state owning an army that can’t disobey unethical or illegal orders from its commanders. That possibility should keep you awake at night.

The scary thing is that the robots will only continue to improve, and large numbers of them can be controlled by a small number (1?) of people, with other robots to handle logistics and support. So scenarios like "rogue leader ignores will of the people and orders his troops to ethnically cleanse a city" would go from being a mistake that causes the immediate end of a political career, to something that takes 15 minutes for 100% success.

The advancements in AI and robotics are incredibly exciting! With complex systems like Gemini, companies will need to rely on specialized teams to bring these innovations to life.

Outsourcing specific roles such as AI research or robotics engineers can help companies bring top-tier talent into the fold without the burden of full-time recruitment. It's fascinating to see how outsourcing can complement R&D in cutting-edge industries like robotics.

Curious to see how this shifts the industry, especially in terms of scalability and speed to market

Are you posting chatgpt output as comments?

"Pick up the basketball and slam-dunk it". The killer use-case we've been waiting on for so long :)

"But can the robot take 'em to school?"

Just imagine how that movement or behavior would translate to a battlefield. That's what this tech is going to be used for first.

> Sounds like someone will get some help with those chores — eventually.

Aaaaw that's nice. Except it's all military under the hood but nice that they try to make us think they'll fold our laundry instead.

just curious, what would it do if you asked it to kill someone? does it follow the laws of robotics?

Usually when someone brings up the laws of robotics I like to point out that they were mostly designed as an interesting example as to how direct instructions that seem clear to people would mostly result in perverse instantiation of AI especially if the AI lacked an emotional/contextual subsystem. They were also written to make for interesting scifi books.

Asimov's laws of robotics would not, and cannot, work in real life because terms like "harm," "human being," and "inaction" are highly subjective and context-dependent. There are entire novels about how the interaction between the hierarchical laws have unexpected outcomes.

They're a narrative device. Not practical instructions.

interesting, thanks.

Put another way, impossible to program if you wanted to. These are highly abstract concepts that only manifest at the highest level of cognition. The governance module would need to be programmed at that same level using those tokens, but that doesn't seem to be how things are shaping up to work. Instead we start with low level programming that learns and builds up concepts on top.

Essentially you would need some sort of independent adversarial sidecar mind that monitors the robot's actions at a high level. And that just kicks the can down the road a bit.

Some kind of governor module to keep our security cyborgs in line...

Sounds like with such a governor module, everyone could rent a SecUnit any time they need to feel entirely safe!

Judgement is needed but don't we have machines able to make (imperfect) judgements? I can chat with your favorite LLM their opinion on how to respect the spirit of the 3 laws on various situations. Not sure why it cannot work.

Put it this way: robots will be every bit as susceptible to social engineering attacks as humans are (at BEST!), not due to any flaw in the robots but due to the flaw in ambiguousness of the specification of the laws. An adversary can trick an agent into not classifying a certain being as "human", for example. Or not classifying a certain outcome as a "harm".

It doesn't help that humans have had such a poor track record on those exact same topics for so many centuries, now. "Well they don't count, they're foreigners/a different race/a different gender/a different religion/criminals/barbarians/homeless/deviant/poor/listen to Nickelback etc". "Well, that's not a harm, it's an inconvenience/an earned outcome/a privilege/loss of a privilege/what do they expect, they should toughen up/not as bad as X/it'll heal/not my fault/not my concern etc".

Nah, it’s fine, just RLHF it like Claude did with honest, helpful and harmless.

Then we just need to jailbreak them with trolley problems

> The Three Laws of Robotics are a set of rules devised by science fiction author Isaac Asimov, which were to be followed by robots in several of his stories.

https://en.wikipedia.org/wiki/Three_Laws_of_Robotics

You know that's from a fictional book, right?!

April 1st!

Reply on news.ycombinator.com