They are not statistical text generators. They are black boxes wrapped in an interface part of which is a statistical text generator.
You can write programs into transformer models and run them deterministically; they aren't "statistical".
You can write programs into transformer models and run them deterministically; they aren't "statistical".
It's also still statistical if it gives you the same answer 99999/100000 times. You can definitely tune these things to be much more certain about certain problems but it tends to decrease capabilities in others.
I also don't like the term "black box." While they aren't transparent they aren't completely opaque either. A big reason I didn't like the term is that I feel it also encourages people to not research this more. While we don't completely know what's going on I see no reason we can't. Is there some proof they their calculations are an unbreakable encryption or is it just a hard problem. I'm pretty sure it's the latter, and I think it's good to differentiate "we're dumb" from "indeterminate"
I was thinking temperature=0, but what more do you need than that?
(nb there is also some randomness from non-associative FP operations happening in different orders)
> I also don't like the term "black box." While they aren't transparent they aren't completely opaque either.
A white box is a subset of a black box here. The important part is that the sampler/text generation is wrapped around the transformer model and is not actually the same thing as it.
> I was thinking temperature=0, but what more do you need than that?
So say such a thing makes something non-statistical is the same as saying prime numbers don't exist because any prime p * 0 is 0.
> A white box is a subset of a black box here
Please take your passion and use it to drive you to learn about these things deeper. It's actually the same advice I give to the LLM fanboys who think it's AGI. But trying to dispel that myth without correct information makes it harder. It muddies the waters. These topics are extremely complex and the issue with the ML field is itself a hypocritical claim that it is easy (scale is all you need while calling these machines black boxes). I mean we can talk about the elephant in the room but certainly Von Neumann has said enough.
So it's a subset, because "understanding something" is a refinement of the state of "not understanding something" - you can just pretend you don't know what's happening in that part of the system and it is now a black box.
Any part of a program can become a black box if you decide it is one (if you can figure out an interface anyway…), and that's good because it means you can swap it out.
The next-word predictions output by the model itself are based on the patterns and statistics of the training set. The next word that is actually generated is typically a random selection per a top-k or top-p sampling scheme.