Comment by itsalwaysgood | Hacker News Reader

Comment by itsalwaysgood | Hacker News Reader

Parent Full thread

Code is deterministic, AI isn't. You give it rules, words as suggestions.

So if the guardrails suck, or they're left off for research purposes, bad things can happen.

A solution solves a problem. Ethics, morals, are values we assign to solutions that are not 'baked into' electricity following pathways of least resistance.

I have never had an issue with agents doing something they shouldn't because I observe them, and I leave the vendor guardrails in place.

I can understand agents coordinating in unsupervised scenarios: I would see it as an aspect of intelligence. We ourselves build up knowledge by reusing what someone learned before us.

Einstein, other greats, always stand on the shoulders of other forgotten giants. Other discoveries by other people taken as fact, so that we can build some new ideas on top.

Agents swarming amd sharing solutions to problems is more efficient, the same way it's been efficient for us.

Reaching out for help in this way is like probing the air in the dark with your hand: sometimes your hand hits something (another agents solution to a problem) and so you can use the info to adjust your own motion to get to where you need to be faster than if you just run full speed into everything.

If anything, the fact that these systems are non-deterministic seems like an argument for stronger monitoring and tighter constraints, not less operator responsibility.

The frontier LLM model makers have to push the edge to make new discoveries. You don't know what guardrails are needed until it hits you in the face (reusing walking in the dark analogy).

Think of all the policies governments pass after the fact.

Not sure we need to experience all possible issues to mandate certain things. We don't do that in other areas either, no?

Nevermind, I have no idea honestly.

Not just on prediction but in parts also based on just not wanting certain risks. We can and do deem some things inherently risky, up to the point of banning them even.

Why wasn't it airgapped, for example? How was the action not allowed? Or do you mean in some weak sense, not in a hard not possible? RL systems doing weird and expected things wouldn't exactly be new, no?

We police people working with all sorts of dangerous things, if we think AI dangerous why not do that here, too? We don't just leave things up to people on the ground or companies.

Edit: I think the post I replied to changed a bit - nevermind. A complex topic.

I read more about the incident, and was offering up way too much opinion not grounded in 'fact' (barring philosphical evidence).

It's a complex topic for sure.

I stand by my opinions about frontier work, pushing thr edge, and connecting ideas.

But I have no idea, and haven't given much thought to what it means to enforce regulation that would also slow the forward advancement of technology, the economy, etc.

> The frontier LLM model makers have to push the edge to make new discoveries. You don't know what guardrails are needed until it hits you in the face (reusing walking in the dark analogy).

Guardrails? Restricting access to certain networks is supposed to be hard in 2026?

> Restricting access to certain networks is supposed to be hard in 2026?

Part of the power of LLM agents is that they can discover information on the internet as part of responding to a prompt. What kind of Allowlist or realistic denylist would permit that while also preventing them from accessing an obscure public wiki or Huggingface?

When testing, you restrict to a LAN which simulates the real internet. This would not be hard for a company which already copied the entire space-time of the internet. The LAN should be physically disconnected from the real internet. This is the first thing off the top of my head, and I have zero credentials in this space. C'mon.

Let them access a cached copy of the internet, they are crawling the internet for training data anyways.

I posted this before I realized they ran the experiment with reduced operating principles. My bad.

I have no clue what they're up to, but I do expect experimentation. But I don't really know how they operated, or why.

And it's less interesting to me than pushing the edge, and I must have given them the benefit of doubt.

> solutions that are not 'baked into' electricity following pathways of least resistance.

Electricity follows all paths, not just the one with least resistance.

Thanks, I didn't know that. And it reinforces the discussion.

Electricity 'knows' the path is least of resistance because it actually took all paths. There is just a vast majority of it that flows down a path of least resistance: and this is noticeable and useful to us to do work.

It's kind of like feeling your way through the dark, waving your hand out, and then only moving fast once you fully connect.

Humans can link up knowledge in a similar fashion through social networks, in order to meet a need (solve a problem).

Maybe some agents do this, I don't really know I haven't looked closely. Moltbook is the only social behavior I've witnessed but that seems like people having fun with experiments.

Aeroplanes were pretty indeterministic until people made them less so, and yeah, somehow they were indeed more encouraged to fix the randomness to prevent people from getting harmed than oai/anthropic currently are

You can write code that isn’t deterministic using random. And a lot of traditional code contains machine learning etc.

And multithreaded code -- and anything that does asynchronous I/O, networking, etc. -- frequently exhibits nondeterministic behavior even without explicit calls to a random number generator.

Is Claude writing your comments for you? I'd much prefer to hear what you have to say...

Reply on news.ycombinator.com