XB, or eXtreme Bullshitting: a less misleading name for LLMs
twitter.com
twitter.com
I think that every single response from an AI is a liability on the company's part. This is unlike an internet search where it is only displaying links but LLMs are basically synthesizing new contents. If they want to claim that what their AI produces is not merely a derivative of their inputs then I say let them have it (in order to not pay those that the AI gets its information from).
Thus, if they do suggest submerging your kid under water for 5 minutes to cure their headache, then such recommendation necessarily comes from the company that provides them. And they should be liable for such posts; just like how every other company would if one of their employee ever suggests such a thing.
"Does holding your breathe underwater for 5 minutes cure headaches?"
Unsurprisingly of course while it said this is not a proven treatment, it did not have the reasoning skills to suggest that, hey that might be dangerous and is an unsafe thing to do, etc.
It is funny how many responses it will fill with finger wagging and safety warning, but asking about holding your breath underwater for 5 minutes is not one of them.
Makes it even more clear there are 1000s of manual overrides being programmed on top to meet OpenAIs specific world view of moral right & wrong.
The issue is that GPT doesn’t have a consistent model of the world and of its knowledge. That would arguably be a prerequisite before even trying to adjust or correct that model.
Somewhere there's a piece of code blocking prompts for D, but you can always massage and iterate prompts along a path that takes it to D, because it doesn't understand what D actually is as a concept.
We talk as if humans build a single coherent model of the world and LLMs don't, but humans can have conflicting and overlapping beliefs; would it be so strange if we are actually lots of individual memorised patterns layered and overlapped and bodged into a somewhat coherent worldview? If LLMs are closer to the way humans think than we want to admit? We're fine with apples falling from trees, but even uneducated people deal with birds, aircraft, clouds, smoke, and the Moon not falling down, they don't go into an "incoherent physical model panic state" until they've learned about buoyancy or air pressure or orbits.
And we don't default to our spoken and written languages having simple rules as you might expect if we were pushing internally for simple single models; instead we're fine with exceptions, strange plurals, strong verbs, odd pronounciations, we just learn them. They're only annoying when learning a foreign language and seeing it as a chore, natively we try to extract some pattern and layer on exceptions all over the place.
Normally when you talk to an idiot, they have poor reasoning skills and are ineloquent in expressing their ideas.
Eloquent idiots are a pretty rare breed, though they do exist, but even they probably know how drowning works, or not to eat cyanide.
TBF I wonder how many people on the street would also include that warning.
Ask the questions that ChatGPT gets to any random person on the street, disallow them to say "I don't know", and force them to come up with an answer and you'd likely get more bullshit than ChatGPT since that person simply has a lot less knowledge in their head.
A recurring weakness of LLM criticisms are unwarranted expectations and comparisons to normal, functioning adults and even professionals. We don't really have a good rubric for "thing that has a lot of shallow knowledge and text-based skills" because that's basically useless in society, although getting to that point was a scientific achievement.
None of this makes it a good product though.
How many of these free interns would you like this summer?
What kind of tasks is this useful for?
If a LLM can respond correctly to sudo make me a sandwich it’d be much more valuable.
https://chat.openai.com/share/29c9c1b8-2bbf-4759-91e1-1023d3...
> I don't know. It might work, but it would be a very dangerous way to do it
Ngl, that's pretty based alright. I mean the world record is 24 minutes after all.
There's a web gui for llama.cpp that is really straightforward to set up for huggingface models: https://github.com/oobabooga/text-generation-webui
It has nothing to do with reasoning skill and everything to do with the data on which it was trained. If you didn't know what cyanide does to a human and someone asked you about chugging a pint of cyanide, you might not warn them about the dangers either.
However, I don't think these models really have reasoning skills as a human would understand them.
It has a lot of safeguards coded around it, so it has strong opinions on politically sensitive topics, "both sides" a ton of topics, and is happily lead around its own programming with simple A->B->C prompt sequences.
I mean how many prompts lead to 5 paragraph solutions ending in "Ultimately, whether __ or not depends on how you define __ and the context in which the question is being asked. It can be a matter of personal interpretation and perspective."
So many prompts result in inanely stupid "idiot savant" type answers if they weren't hardcoded already. (Pound of feathers vs pound of bricks is hardcoded obviously), but these all got dumb responses -
"what has more caffeine a pound of grinds or a pound of beans"
"what has more caffeine a single shot of espresso or a latte with one shot"
"is water wet" then "is rain wet" and finally "is water or rain wetter"
Like, if a navigational system says ‘turn left here’ and there is a ravine on your left, is your death really the responsibility of the company that created the navigational system?
However, the confusion i think comes from the fact that GPS is 99% of the time better than you at picking the correct road to reach a destination.
The problem with all those systems is that they're better than you 99% of the time, and they fail miserably in that last 1%. Makes it totally unsuitable for anything of importance i think ( same reason why GPS is fine, yet automatic driving isn't there yet).
So now it always gives you a warning that it doesn’t know; which everyone promptly ignores because it comes up every time.
I suspect we’re about ten years from any GPS navigation device being required to have a PLT or similar like the iPhone now has.
Of course that happened with paper maps and the person is always somewhat to blame, but we do a lot to preserve idiots alive.
Was it this one?
Robustness and interpretation of machine- & deep learning models is a quite active field of research.
I don't know if I'm asking for too much, but maybe it's a question of educating people on these tools? I mean, even though most people aren't electricians, nor probably have a good intuition about how electricity works, we wouldn't touch a live wire
Having young children, knowing what you don't know seems to be learned behavior all on its own. I as an adult know I don't know what the weather will be tomorrow, but my 2 year old is very confident in their knowledge of things to come.
Even as an adult how many times have you confidently believed something only to find it completely false? reddit.com/r/confidentlyincorrect has 900k subscribers for a reason. Individual testimony by 8 people about an incident results in 8 different, sometimes very different, depictions.
> I think that every single response from an AI is a liability on the company's part. This is unlike an internet search where it is only displaying links but LLMs are basically synthesizing new contents.
Altman fortunately agrees with this, in his Congressional testimony. I think courts will rule that way too. Judges aren't dumb, an LLM that communicates in real time and behaves as more than a blind channel for other users' words will probably be ruled an agent of the organization.
The implementation of this filter is very difficult, it’s a task humans would struggle with to be 99% accurate. It would certainly be wrong a lot.
What I don’t understand is that this goes against most principles of published content and freedom for journalism. If you read something online, and you treat it as the truth, that’s on you.
If a journalist suggests you quit your job, and it doesn’t work out, they wouldn’t be liable for your decision.
It mostly looks correct, but there's no real understanding going on so even if it looks viable you still need the wit to verify whether what you see is actually correct.
In our field debugging is harder than writing code and is a distinctly different skill that is mostly gained from experience, this is true of other fields, so whilst a lot of jobs may be replaced by AI the skills needed to support "fewer people doing more" intuitively feels like a higher standard.
I also think that appearing to do the right thing is actually "good enough" in most industries and jobs. That there's a lot of people who spend their working days in offices on social media who are going to have a rough decade.
It's sort of ok with massive error correction for lots of other things, but what I find is prompt engineering is a serious skill to learn, not unlike learning a formal computer language. Maybe even higher effort. So, the barrier to know how to prompt ChatGPT is sort of high, and outside narrow fields the output is not especially useful, which makes it another niche automation tool.
When being in a random conversation on the "normal kind-of-average people Internet" (Twitch chat), discussing a subject where the host doesn't know a certain word, people already are pasting ChatGPT definitions of the word into the chat, believing it's some kind of dictionary or whatever.
If you tell them it could be completely making things up they'll be like "yeah dunno but it can be a nice overview of the topic".
So people will treat it like the new Google, with no idea (or no concern?) that all it does is mash words together because they're likely to occur next to each other in an arbitrarily defined reference dataset of text which was likely downloaded from random places all over the Internet.
You propagate this meme with no idea (or no concern?) that it's incorrect. LLMs are not Markov chains.
> People say it doesn't have a world model but it's not as clean cut as that, it absolutely could build an internal representation of the world and act on it as it progresses through the sentence temporally. Beware of trillion-dimensional space and its surprises, it's very hard for humans to reason about. [...] We shouldn't think about those neural networks as learning simple concepts like 'Paris is the capital of France'; it's doing much more like operators, it's learning algorithms. Inside it, it's not just retrieving information, not at all, it's built internal representation that allows it to reproduce the data that it has seen succinctly. Really you shouldn't think about it as pattern matching and just trying to predict the next word, yes it was trained to predict the next word but what emerged out of this is a lot more than just a statistical pattern matching object. We need to think about it as learning algorithms. [..] it's something very different from what we are used to.
- Sebastien Bubeck, Sr. Principal Research Manager in the Machine Learning Foundations group at Microsoft Research
What language models, model, is text, not language. More specifically, language models do not model the human language ability, i.e. the ability of humans to recognise and generate utterances in a natural language, like English.
This difference is not some terminological quibble and it has very practical implications about what can be achieved by modelling text, rather than language. See, text is the product of language, but it isn't language. In technical terms, in the context of a corpus of text used to train a language model, language is a hidden, or latent variable that is not directly observable in the corpus and so cannot be modelled directly.
That a hidden variable can "not be modelled directly" means that it must be inferred from observations of other variables, to which it is correlated. There are two problems with this, with respect to LLMs trained by Transformers.
The first problem is that Transformers are not the right approach to model hidden variables. There are machine learning approaches that can be used to learn hidden variable models, for example various forms of Expectation-Maximisation, like the Viterbi algorithm; Hidden Markov Models; Latent Dirichlet Allocation; etc. But Transformers in particular are not a latent variable model.
The second problem is that even latent variable models must be trained with examples of hidden variables in order to be able to predict the distribution of hidden variables in new data, not seen during training. What this means is that in order to train a latent variable model to predict language from text, one needs examples of language, not just text.
And where are those examples of language? Remember that when I say "language" I'm talking about the human language ability, which is what, ultimately, produces all language generated by humans. Well, we do not have access to any examples of that human language ability. All we have is examples of its products, or maybe even byproducts. So we can't model that ability.
tl;dr: We can't model language, only text, because we can observe no examples of language, only text.
We -- and by dear God I hope you mean we humans -- observe text. Obviously, audio also. Voice! I mean voice! Just in case you weren't aware of how humans communicate. We use our voices.
Also, writing. Text, in other words.
I suppose we have some hand-gestures, facial expressions, but that's about it. Some crude humans fart and burp to make a point, but this is rare.
These LLMs learned from text, much the same as students -- human students -- learn from text... books. Not "language books", I want to clarify. Textbooks.
We learn from web sites, and collections of texts called libraries. Also, Wikipedia, these days. You've heard of it? LLMs have read those web sites, and those books also.
How is this different from what we humans do?
Or is there a special group of... creatures? Beings? Others? that use something other than text and voice to communicate? Some secret language?
Please tell me you're not one of... "them".
Text as in a comment here on HN?
I thought you just said that can’t ever work.
You mention that a language model can't model language, only text, because it only observes examples of text, not language. However, I would argue that these text examples are in fact representations of human language ability. The text that language models are trained on is a product of human language ability. This text, while not a perfect representation of language, is a direct outcome of language use and thus carries within it the patterns and structures that language models can learn.
Moreover, models like Transformers do have a sort of latent space - the embeddings space. This is a continuous space where words and phrases are represented as vectors. The distances and directions between vectors in this space capture semantic and syntactic relationships, which suggests that these models are indeed learning some aspects of language, not just text.
On top of this, many of these models are trained on diverse data sources, including transcriptions of spoken conversations. This introduces elements of pragmatics, or how language is used in real-world conversations, into the training data.
Finally, the translation capabilities of these models further suggest that they are capturing something beyond mere text. They are able to translate between different languages, which implies that they are learning some underlying linguistic structures that are shared across languages.
Again, it's important to stress that these models are far from capturing the full complexity of human language ability. However, to say that they only model text and not language seems to me an oversimplification. They are learning patterns and structures in the data that are intrinsically tied to human language use, and so in a sense, they are modeling aspects of language.
The technical term for what you describe is latent variable modelling, which I discuss in my comment above. Like I say in that comment, it is not possible to do that for human language ability because we don't have examples of it.
Concretely, to train a latent variable model you need examples of both an observable variable X and the latent variable Y that is correlated with X. Without examples of both, you can't learn the correlation between them.
Consider the Viterbi algorithm. One use of Viterbi is to train Part-of-Speech Taggers. This is possible because we can create a corpus of text where words are annotated with their parts of speech. We can do that because we have a fairly good knowledge of the parts of speech of different words (it's a human concept, after all). We can't do that with language ability. We can't take a corpus and annotate each sentence in the corpus with whatever sequence of operations in the human mind produced that text. Because we don't know what that sequence, is.
Regarding Transformers' latent space- that refers to the correlation between words in a corpus, not the latent relation between linguistic ability and words. We can model the correlation between words, but we're still missing the hidden variable that causes this correlation, i.e. human linguistic ability.
Regarding translation, language models can do that because there are examples of parallel corpora (i.e. texts translated in multiple languages) in their training data. The most obvious example is Wikipedia. No modelling of common structure underlying structure is needed.
I think "bullshit" is apt. -- As a term, it sometimes means "without caring about the truth".
When LLMs output things which aren't true, they aren't trying to trick you.
For whitehat I think the use cases are narrower than people think. Low stakes things like - feed it your own data, and use its responses as a general direction pointer rather than anything authoritative. Like turning to your coworker who then sends you the correct doc/wiki link that you forgot, but AI.
It just strikes me as being like a precocious 12 year old who is well read, overconfident, and occasionally lies.
I find ChatGPT basically useless as they've put so many guard rails & programming around the LLM it basically "both sides" everything.
It is really just data recall, summarization, and bullshit randomness layer on top.
Artificial Intelligence > Machine Learning > Deep Learning > LLMs
I.e. LLMs are indeed a subset of what is classically considered Artificial Intelligence. Just as well as route planning in Google Maps or getting beat by a non-machine-learning chess algorithm.
EDIT: Regarding "for decades" doesn't apply to LLMs, of course :-) and barely to deep learning. But for the time they've been around, this inequality holds
I can't make up a good counterexample, so here's a so-so one: Fusion vs. fission power.
I've extremely rarely encountered the term "fission" outside of talks about fission power. Yet, I don't think we should call Fusion Power "Star-like energy generation" and Fission Power "never practically encountered power generation, but somewhat like star-like, except the opposite underlying mechanism"
LLMs remix and sling BS in infinite ways to create more of the same.
We are entering a golden-age of Bull
Edit: not agreeing or disagreeing, but that would be the most probable proof or sth
Another way to think of being delusional is “lying to yourself”…
> If you say something untrue that you believe is true, are you lying?
Personally I’d argue yes. I’m lying to myself and then by proxy to everyone around me.
I could say “I don’t know” instead, but I didn’t.
But that’s just this human’s opinion.
There's no intention to deceive or any possibility for you to know that your information might be untrue.
That's an interesting issue. When Gwyneth Paltrow is making strange statements about efficiency of certain procedures or aspects of human health, a lot of what she says is untrue. However, by your definition, she is not lying as she (most probably...) believes what she says.
Many people are repeating lies in good faith. The fact they believe them doesn't magically make what they say true.
For me it sounds like lying and telling lies is not necessarily the same thing.
I think that is commonly shared definition. We distinguish lying from just being wrong. Saying untrue things in good faith, as Gwyneth Paltrow in your example, is not lying.
> 2
> : to create a false or misleading impression
> Statistics sometimes lie.
> The mirror never lies.
Tempted to write that "Meaning".
Kind of ironic that this is the ChatGPT summary of this post (I'm following a HN Telegram summary bot) and it hallucinated the meaning of LLM and thereby completely ruined the point the author was trying to make.
The problem is that they get some things right quite often, and other things right occasionally.