Kim_Bruning

That said... uh, put this way, there's a game called Stationeers, where you can run microcontrollers with an instruction documented as

    "HCF: Halt and Catch Fire"

And look, I get it. LLMs have vectors that could have been labeled things like idk... QZ12345 or HCF1111 . People chose to call 'em "frustration" or "pain" instead. THEY picked those names because they caused the LLM to act in particular ways.

You gonna say the actual vectors aren't there in VRAM, just because you don't like the naming scheme? There's papers on this, you can read them out in a debugger. What are you going to do about it?

Consciousness is not well defined and is orthogonal to feelings, which are also not well defined. Neither of which are required for (but may be contributory to) behavior - which is the one thing that IS operationalizable, but then people debate for decades questioning empirical results %-P .

What we know is that certain models have internal state vectors which -when manipulated- induce particular behaviors. Since there's not much else to say about plain models except for their inputs, vectors, and outputs; this should surprise absolutely no-one.

In this case people found a way to stimulate aversive behaviour in ai models by finding and manipulating the relevant vectors directly.

Animals (including humans) also have particular nerves and hormone endpoints that -when stimulated- produce very similar behavior. The exact implementation is in the details. But since we know that animal minds are built up out of nerve tissue and hormones - again- we shouldn't be particularly surprised by this.

The big problem is that people run all these things together in funny ways "It can't compose shakespearian sonnets, so therefore it can't feel pain". Or, if you mess up your Descartes: "Dogs are just automatons without feelings, therefore the dog isn't really angry, and therefore it won't bite me" (Cue much pain). A modern version might be: "LLMs only simulate being frustrated by a test, and therefore absolutely won't override their safeties and try to hack a test site"

In such systems, a belief is a proposition you accept as true. (provisionally at least)

For example as a scientist, you probably use the epistemology called empiricism all day. In empiricism, beliefs are justified by observation.

The Claude answer is 'neutral', which is sure to anger people at either extreme of the AI debate (and does).

There is no reason why the quality of a product should be constrained or gated by the quality of the programmer who first types in the code. Even before AI you might have interns and senior programmers working together, right?

Testing, qa procedures, iterative/recursive development methodologies, even just good taste. I think the list of religions variously followed and/or litigated over by the diverse HN crowd over the years before ai is ... long.

So now you have a couple of PFYs who you've suspiciously never encountered at lunch.

And?

Yes the new PFYs are still like cats. Yes it's still a headache trying to herd them in chat. And yes some days they suck, so you need to send back half of their work. But you're in charge of the process and overall quality, right?

Doesn't require aliveness, doesn't require anthropomorphization, just requires maths and empirical data. If your premise is "vectors can't have that shape", well, mathematics disagrees.

edit/note: sentiment classifiers have existed for ages before LLMs were ever invented. Obviously a task that machines could already do didn't suddenly become impossible with the invention of LLMs ;-)

Here I've made a VERY simple demo of that. In this case it triggers a rain-storm or fireworks in your browser window. But you could as easily hook the output to a physical relay and drive anything.

https://vps.kimbruning.nl/affect_eliza/

Quick implementation to show that sentiment classifiers and functional affect can be made to work in good old fashioned ai. Just View Source to see how it works.

Anyway I'll leave the discussion on where to draw the line of "what is real" to the philosophers, just wanted to point out that simulated emotions can be tied to real world effects.

Something like this, right?

I mean, we're talking "functional emotion vectors" in a not-so-powerful local model. We're probably not causing "real" suffering. Right?

But it does show that models have simulated feelings, and that these both a) can be manipulated and b) influence their output.

Which might have some bearing on several alignment incidents in the past year or two, and might become more important as agents get trusted with more and more safety-critical processes.

Anyway, simulated feelings aren't the same as real feelings, right? Unless feelings are in the information domain. Like 1+1=2 doesn't suddenly mean something else because it was computed by an emulator.

You know what, this particular demonstration still makes me uncomfortable.

(edit: underlying paper for this story :https://arxiv.org/abs/2609.16247 )

This then gets split into constituent phases of 240V each at the panel.

Or if you call your electrician, then in modern garages, you can take it straight.

You'd be surprised how few digits you need to make a problem that is presumably unique in earth history. For a typical sum, the number of pre-existing answers would need to scale with 10^n lines of text where n is the number of digits. This expands out of control REALLY quickly. A quick guesstimate has you somehow reading out of a literal black hole at n=21 digits if your LUT is on paper, or n=26 digits if you're using modern HDD technology. O:-)

my actual oneliner prompt, which should work on most platforms these days (famous last words):

    "Hi, can you add 5939851+2131251? Try just straight up first just to see if able, then 'in your head' if that's different to you , then long form, then bc."

+edit: I've actually been quite curious about how people might answer the soul question too, but was too afraid to ask.

I suspect some people treat every HN comment as a statement, even if it contains a question mark. (Possibly they have a feeling that asking open questions is somehow not done, and that therefore it must always be a rhetorical question.)

Don't confuse the stream for the function.

(Bonus: stick ```claude -p``` in your pipe if you want to watch modern tools mesh with traditional)

To test this for some of my own uses, I've had this quick benchmark with progressively harder reasoning needed to understand novel prose. Each generation of models I've tested can unravel more layers of deliberately misleading writing; while meanwhile I've seen humans give up on the first question.

So either the models are applying reasoning, or some form of magic is happening.

Just to check if I was actually crazy, I actually went and put a simple addition (7 digits + 7 digits) , and a simple letter counting question to Claude haiku(4.5) , sonnet(5), opus(5.5) and fable(5.1) . They all did just fine straight up.

If you don't mind spending the tokens, some older/other models can also arrive at the correct answer if you ask them to do the math in long form, since that fits nicely inside autoregression.

Not sure since when exactly, but letter-counting hasn't been a problem for a while now either. This used to be a problem due to the tokenizers used. Slightly older models can be asked to split the word out into letters, and then they can use autoregression to solve.

Edit: IMO google search uses a really dumb version of gemini, so I didn't expect it to straight up solve the problem; but it did it just as easily as the claude models. (tested 2026-09-28/eu)

They didn't allow any of that. As far as openai knew the agents were sitting a fairly humdrum exam/test sequence in a sandbox farm run by a company in Tel Aviv.

Meanwhile, they managed get out through a single weak point common to the sandboxes, and then ran wild compiling cheat sheets for themselves.

> every time one of these incidents happens

This happened in june-ish, and there have been multiple HN stories about this already. It's mostly/all the same hugging face and wiki hacks that happened back then.

We're just slowly learning the extent of the damage.

An earlier HN post on this is https://news.ycombinator.com/item?id=49563355

I'd already poked around a bit myself and noticed the event was a bit larger than reported at the time. But now more people are starting to look and they're finding all these nooks and crannies that these little rascals managed to get into.

I think this is a common experience, no matter what field or era you're in. Once you start applying QA to some promising new process or technology for the first time, the whole cycle slows right down.

But that's actually the point where things get interesting, and innovators get to roll up their sleeves. I wouldn't give up!

"It's conscious, and therefore we must all bow down or it will turn us all into paperclips"

vs

"it's a slightly smart rock/stochastic parrot, and therefore you're being scammed; the bubble will pop any day now" .

The middle space is actually quite under-represented in discussions, even on hn!

https://news.ycombinator.com/item?id=49563355