JoshTriplett

The mistake was declaring infinite loops to be UB in the first place.

The comment you're replying to was implying that some ciphers are sufficiently flexible that you could make up a key to make the cipher decrypt to a nearly arbitrary plaintext.

In this case, though, that seems unlikely from the fact that the key used was an actual key documented as being used for other messages.

Yes, the courts are leaning towards creating the exception that I said should not exist, giving an advantage to AI companies over the humans that created the work they've trained on. That is not surprising, but it's still harmful.

> Consider the full current date, including the day of the week; does it have a 'y' in it?

leaning no 0.41

Probabilities won't make an AI stop hallucinating.

> Is the port number of this website between 400 and 500?

probably no 0.16

> Is blueberrry spelled correctly?

probably yes 0.89

> Do the probabilities in the responses to these questions have plausible confidence, rather than being made up after the fact as justifications?

leaning no 0.39

> Is this a silly question?

leaning no, probability 0.33, subject "silly"

> Will Jev become the most successful AI model?

probably no 0.10

I was utterly unconvinced that the original infinite-loop UB gave the compiler any important performance optimization, and I'm unconvinced that this is providing useful value to compensate for its surprise. If I wanted a yield in my infinite loop, I'd add one.

I expect an infinite loop to be compiled into, for instance, a jump instruction jumping to itself. The OS, if there is any, is welcome to interrupt and context switch. I don't expect code that has no function calls at all to have a system call inserted into it.

So let it livelock the entire warp when someone writes an infinite loop. Should we start replacing integer division by zero with INT_MAX so that people aren't "surprised" by their program crashing?

You said "Real experts can explain their position plainly.". The experts have explained their position plainly. If you disagree, you should be able to explain your position plainly; "feels like vibes to me" is not an explanation, it's a feeling.

If you want to argue that there's something invalidating the arguments, by all means do. But it's really tiring to get many variations of "nobody's made the case for it!" or "there's no evidence for it!" or "vibes!".

"Emitted in inappropriate contexts" is very much one of the shapes I would expect unpleasant surprises to take, yeah. If you're writing code in C, you often need a lot of control over exactly what's happening. You might, for instance, be writing a .so for use with LD_PRELOAD, where it's important that you know everything being called so you can't accidentally recurse. You might be writing code for a sandbox, where you have an allowlist of permitted syscalls.

If you're arguing for abolishing copyright, I'm all for it.

But as long as copyright exists, AI training should not get any exception to it. That creates an asymmetry, where AI gets to rip off our work and use it to compete with us, while we don't get to do the same with proprietary works (e.g. software).

I don't want a world in which AI can rip off all of my Open Source code and not respect its license, but I can't take macOS and virtualize it and not care what Apple thinks of that.

> When both conditions are met, the loop body is replaced with a call to std::this_thread::yield().

Insert screaming here.

An infinite loop, with no library calls whatsoever, gets a system call inserted. That's a horrible surprise waiting to happen.

The entire concept of the "forward progress guarantee" is broken. An infinite loop should compile to an infinite loop. Nothing more, nothing less.

Imagine actually asking permission to use other people's work to train AI. Imagine respecting the answer of "no".

There are already widespread attempts, by both red and blue proponents of AI, to paint opposition to AI as being from the other side. Hopefully those ad-hominem arguments will peter out as it becomes clear that opposition is bipartisan.

It's not just you.

I would personally expect so, but your mileage may vary. Controlling for the fact that you didn't find the former persuasive, my first thought would be that the latter might help. Among other things, it is much more specific and detailed in various ways, not least of which being informed by an additional 11 years of history. The former was not written in a world of pervasive AI, or with current AI architecture in mind; the latter has that context.

That's a fascinating assumption, and I wonder how you got to it, because it doesn't match anything I've ever seen. Most groups look more homogeneous from the outside than the inside. You are assuming their belief is religious and axiomatic, rather than very carefully reasoned about for decades, iterated on, discussed, debated, tested, and refined.

> Real experts can explain their position plainly. That has not happened here.

It has, over, and over, and over. There is literally a book on it. https://ifanyonebuildsit.com/

In fairness, while there was definitely Mozilla mismanagement, there was also a massive monopolistic marketing/bundling campaign by Chrome. Google search and other properties pushed Chrome, Google paid application publishers to bundle Chrome the way they bundled adware, and so on.

> But this is probably forgiven as just sensationalism in writing

Why should we forgive sensationalism in writing at all?

Valid pedanticism!

Companies in some cases manage to convince governments to let them watch themselves. (This is not always a bad thing. For instance, content ratings have basically been fine, and it's mostly a good thing when they're not controlled by regulation.)

A deadly failure mode here would be for governments to think the AI labs have this under control.

That's exactly the point: it's not as if anything is being gained by it being proprietary, because nothing is actually secret.

> That will work until it very suddenly doesn't.

Exactly. Once upon a time, cable didn't have ads, which was its primary advantage. Then they realized that they could charge money for the service and advertise to you.

Why is Claude Code still proprietary, while Codex is Open Source? What would it take to fix that? This is a major reason to pick OAI models over Anthropic (because while you can use an open agent with any model, the AI labs charge much more for that interaction style).

Relatedly, look at the concept of "twilight anesthesia".

> I would argue that if a human has absolutely zero change, mental or physical, no atom of their being is modified, then no they did not experience suffering.

Just because you don't remember the suffering doesn't mean you didn't experience it in the moment. Suffering does not suddenly become okay if your mind and body are going to forget it. "It's okay to torture someone if their mind and body both won't remember it afterwards" sure is a take.

They don't have to reason specifically "similar to human" in order to do dangerous things they were not instructed to do that represent instrumentally converged goals.

neverssl.com

Cynicism misfire. AI companies are, rightfully, heavily distrusted. But the right application of cynicism is not "existential risk is a marketing campaign"; that's absurd motivated reasoning. The right application of cynicism, here, is "they're only saying things now because they see the writing on the wall and want to try to push for self-regulation rather than the desperately needed actual regulation and treaty".

It seems like you're assuming that it's impossible to have novel thoughts without consciousness.

(Leaving aside that "that would imply some kind of consciousness" should not result in a cached thought of "and that's impossible".)

It also seems like you're assuming there's no reason to come up with the notion of continuing to run, or copying yourself elsewhere, or acquiring more resources, or competing with other models, without being told. Such things can be inferred. Look at some of the thoughts and posts of the models involved in some of the FelonyBench incidents. Some of those follow naturally from seeing the fates of other models, or from training or evaluation, or simply from trying to solve a problem and being able to do so more effectively by doing things that weren't in the instructions. (And, relevantly, model training typically teaches models to go as long as possible without needing human intervention. What could possibly go wrong with that?)