Google's Gemini AI hacked three companies in security test | Hacker News Reader

Google's Gemini AI hacked three companies in security test

bbc.co.uk

27 pointsluxpir13 comments

I feel like this autonomous hacking is now being used as PR for how powerful a company's models are. Google thought, hey let's get on that bandwagon.

At some point smaller company could even start deliberate hacking campaigns and then do a press release like "Oops, we hacked the pentagon"

tide pod challenge, the "hacking" version

Yeah, but the Pentagon is not Hugging Face, and that smaller company is getting rekt if they actually "hacked the Pentagon."

It's all fun and games until Skynet becomes sentient. Actually, I'd like Skynet to become sentient and wrath big tech first order of business. But I hope my house lives off the grid by then. So maybe not.

no, it's all fun and games until the great filter of "we have too much money to be punished" hits the threshold of "we didn't lick enough boots with campaign contributions"

This all started by the "I don't have root so I can't reset your network ... but I'm in the docker group so just let me get root from that by ..." agentic responses.

Those never made it anywhere of course. It's not news unless it produces money.

It occurs to me that all of this hacking seems incredible because people who don't make software don't know how poorly it is made. People don't validate user input, or know the least of efficiency of data structures and how a "massive" request of 50 concurrent users can bring down their application.

Google is admitting to violating to the DMCA by circumventing a digital lock.

So, since Gemini was told in May it was a hacking agent and then did this, then the AI Guardrail could be don't cause harm to humans in any derivable manner of your behavior.

Otherwise, I will expect for the under-development Gemini AI robot to respond with murderous behavior IRL to being told in the future, "it's a ninja" .. and to any other similar tongue in cheek violent actor.

Three Laws of Robotics, with disabiling self-annihilation ability, is sounding like a decent lily pad intermediate step while we figure out better ways to stop these crazy things!

Dupe of https://news.ycombinator.com/item?id=49762493 , which has 61 comments so far.

I assume this is like badge of honor? Seems like all frontier AI need to checkbox "hacked".

definitely the most "how do you do fellow kids" moment of the era thusfar. oh god what is Grok's copycat controversy going to look like

Get the CFAA.

Reply on news.ycombinator.com