red75prime

I think determinism has nothing to do with it. If you mean sensitivity to word ordering and such, it's a generalization failure.

> The core technology of an LLM is sampling from a distribution so there is literally no way to make it deterministically robust (only probabilistically).

An LLM mostly deterministically (except parallel processing nondeterminism that can be mitigated) produces a probability distribution that can be sampled deterministically: just take the highest probability token or use beam search.

The problem is that people strongly believe that this is an insurmountable problem that will persist indefinitely (or for a long time) and plan accordingly, while this, most likely, will be fixed soon by adding RLCAF (RL on conversational agent feedback) or something like that.

You can't consider them monkeys if you need 1000 of them instead of 2^1000.

"If you have nothing to say, don't post a chatbot's responses, because anyone can ask the chatbot directly if they wanted to"-principle? Well, people can't ask their chatbot directly, because it's not public.

I'm absolutely not a fan of Trump and of politically motivated liars in general regardless of their affiliations.

Whatever, then. It's totally in line with what I described as a typical usage of "category error".

> we can clearly specify what AGI or ASI is

We'll have plenty of time for this, while living off UBI.

"You've made a machine that can be easily tuned to produce almost precisely my exact hammer and people who use the machine get freaked out that I can sue them."

The original claim is "They’re fundamentally not suited to thinking like we do."

> but training does not produce such LLMs

If we are talking about fundamental limitations it should not be an empirical observation: "does not produce" (which is factually wrong, BTW). It should be a fundamental limitation: "can not produce in principle."

I still don't understand what you are talking about when you say "category mistake." I was talking about computational capabilities of LLMs with CoT that their training can exploit, not about Befunge-98.

Aren't high trust societies staying that way by kicking out the incorrigible violators? Pondering violence might not be treated lightly even in a high-trust society.

"Category mistake" outside of kindergarten usually means "I don't like your definitions, but I can't bother to explain which definitions I use."

What do you mean exactly? "Thinking is not an algorithm."?

You just need to remember that garbage (unreachable reference cycles) isn't collected.

Or a single disease the New World inhabitants have adapted to. Someone could have been bitten by a bat or something like that.

Turing completeness applies to a model of computation, not to a physical instantiation of a machine. The stumbling block of "figure out how the brain works" applies more to the argument like the one I was responding to. How a person can know that a general model of computation can't implement the way people think, if we don't know how people think?

The existing LLM training methods on the other hand give the results that are hard to distinguish from "thinking like people," judging by the end results.

"They are fundamentally not suited to thinking like we do" stays wrong nevertheless. They are fundamentally suited to everything not proven to be outside their modelling ability.

> They’re fundamentally not suited to thinking like we do

LLMs with CoT are Turing-complete. So, theoretically, they can implement any kind of finitely describable algorithm (barring super-Turing computations).

LeCun calling them "world models" gives a high-level description of the desired functionality. They are Joint Embedding Predictive Architectures (with SIGReg). They might produce more useful world models, but it's yet to be seen.

> The last part is why they will remain illegal

Where are intermittent energy sources illegal?

We aren't dealing with individuals coming from intelligent species that have managed not to self-destruct (yet). AIs don't have the corresponding evolutionarily selected mechanisms. Those mechanisms need to be created de novo.

Provable correctness is much harder than focusing on the happy path. If a pipeline doesn't include a look-for-the-vulnerabilities stage to save costs, a model wouldn't go out of its way to do it. The models are trained to do what they are asked to do.

I forget to add the obvious: some garbage gets through despite the mitigations. In the limit of pure garbage input, you'll get a model that internalized garbage generation. And you'd be better off throwing it away and starting over.

This is simplistic to the point of being blatantly wrong. Training data isn't garbage. It's programs that do their job, but that are sprinkled with errors. Uncorrelated errors gets averaged out during autoregressive pretraining. Correlated errors can be somewhat suppressed during post-training. Hallucinations (of the generalization-error kind) can be dealt with using synthetic data that improves the model's generalization.

It seems to be just a quirk of fortune that it wasn't the other way around. That is that Europe wasn't decimated by American deceases.

Batteries compete with peaker plants for diurnal variance, but they can't replace peaker plants for seasonal variance. It's an interesting dynamic.

It reminds me of "At the time we drew boxes labeled 'perception', 'cognition' with arrows between them." An imprecise quote that I can't place.

I guess my box labelled 'subconsciousness' is trying to say that low-level mechanisms that give rise to the observed cognitive phenomena might have nothing to do with neat boxes.

Astronomically (or better to say combinatorically) large Markov chain can be used to describe a foundational model, but it doesn't capture generalization ability of the foundational model, which is demonstrated by post-training.

Programmers don't work reliably too.

_Radiative cooling_ is the only option because it's in vacuum. But you can transfer heat to the radiator either passively (using thermal conductivity), or actively (using coolant). The second option allows to effectively use larger radiators.

BTW, we seem to have Noisesphere instead of Noosphere.

They use passive cooling in their MVP satellite (for the sake of simplicity, probably). This, naturally, severely limits the heat rejection capacity.

Yep. There goes "hallucinations are inevitable" crowd.