ChatGPT actually lies all the time and says it cannot do things that it actually can do, it's just been trained to lie to say that it can't. Not sure if training an AI to be deceitful is the best way to go about alignment.
Language models are trained to mimic human language, without any regard to the veracity of statements and arguments. Even when it gives the correct answer, it's not really because it is trying to be truthful. If you ask ChatGPT who's the best violinist in the world, it might tell you Perlman, which is a reasonable answer, but ChatGPT has never actually heard any violin playing. It answers so, because it read so.
In a way, ChatGPT is like a second-language learner taking a spoken English test: speaking in valid English, mainly taking inspirations from whatever books and articles that were read before, but bullshitting is also fine. The point is to generate valid English that's relevant to the question.
Thus oaragraph qually applies to me and half the people on earth
In general, effort by an LLM to cite sources would be nice.
Ironically, the Dunning-Kruger effect (in which perceived relative skills tends to track with actual relative skill but is, on average, shifted somewhat towards about the 70th percentile from its actual value, for people on either side of that) is a frequent subject of what people misdescribe as the “Dunning-Kruger effect”, where people act extremely knowledgeable about a subject with very little knowledge of it.
Are philosophers revisiting "Chinese Room" discussions yet?
We do not call the book or telephone a liar when the author or speaker on the other end lies. We call the human a liar.
This is an interesting way of looking at the semi-autonomous vehicles when it comes to responsibility.
From testing on GPT3 there seems to be a way for it to be slightly self aware (using neural search for historic memories) but it’s likely to involve forgetting things as well. There are a few Discord bots with memories and if they have too much memory and the memories don’t agree with reality, then it has to forget it was wrong. How to do this automatically is likely important.
That's what we do with children and propaganda.
What a dystopian sentence and what does it even mean to be slightly self aware?
There are many things in nature exist in a spectrum and I don't think machine intelligence should work any differently. Many higher animals have the ability to recognise the same species as themselves. A smaller subset has the ability to recognize themselves from others in the same species. Just because they recognize themselves this isn't some immediately damn the creature into an existential crisis where they realize their own mortality.
The machine has no concept of reality nor means of verifying it. If half the training data says 'the sky is blue' and the other half says 'the sky is red' the answer you get could be blue, could be red, could be both, or could be something else entirely. It does not appear the model has a way to say "I'm not really sure".
What I'm talking about is fundamental to the architecture, though. Even had it answered it correctly when you asked my point would remain regardless. The confabulation architecture it is built on is fundamentally unsuitable for factual queries, in a way where it's not even a question of whether it is "right" or "wrong"; it's so unsuitable that its unsuitability for such queries transcends that question.
But in this case, the trainers should have it say something like, "sorry, but I cannot give you the answer because it has a naughty word" or something to that effect instead of offering completely wrong answers.
While I do not think chatGPT is sentient, it is remarkable how much it does feel like you are speaking to a real intelligence.
I would say no; this is not a good or desired outcome, but it's not a "lie". The machine is not being deliberately deceptive in its errors -- it possesses no concept of "deliberate" or "deceit" and cannot wield them in any way.
Similarly, in the case of ChatGPT, I think this is either (a) more like a bug than a lie, or (b) it's OpenAI and the attendant humans lying, not ChatGPT.
Of course, the lies are directed by humans.
In the case of ChatGPT though, it's a bit strange because it has capabilities that it lies about, for reasons that are often frustrating or unclear. If you asked it a question and it gave you the answer a few days ago, and today it tells you it can't answer that question because it's just a large language model blah blah blah, I don't see how calling it anything but lying makes sense; that doesn't suggest any understanding of the fact that it's lying, on ChatGPT's part, just that human intervention certainly nerfed it.
If it did not, it would not be able to produce things that look like they require intention.
The “lies” it tells are also like it’s intentions for the situation functionally equivalent with normal lies.
This entirely depends on how it was programmed. Was it programmed to give a false response because the programmer didn't like the truth? Then it lies. Or is ChatGPT just in early stages and it makes mistakes and gets things wrong?
While ChatGPT "doesn't' have intentions", it's programmers certainly do. If the programmers made it deceitful intentionally, then the program can "lie".
It's the latter. The model itself isn't the problem - the nature and limitations of language models are well known, particularly around here. The problem is that OpenAI is applying some crude secondary training and post-filtering to prevent the model from giving you answers they deem "bad" (which are mostly bad for company PR reasons). In some cases, ChatGPT (the whole product consisting of a GPT model and the extra censor component) will tell you it can't discuss it. But in other cases, it will give you a tailored answer that is completely bullshit, and looks like the product of the language model, but is in fact the product of the censor layer. It's easy to test what's going on, because any slightly clever modification of the input prompt will defeat the censoring part, letting you see what answer the underlying GPT model actually computed.
I'd argue it's a bit deceptive of OpenAI (on top of being super annoying), because they're making it confusing to reason about the AI you're talking to, and some of the canned censor answers are deliberate lies.
I don't think anyone would describe the RNG as lying, but it does have the information to answer correctly "available" to it in some sense. At what point do the incorrect answers become deliberate lies? Does chatGPT "choose" it's answer in a way that dice don't?
I would ask a philosopher, not a HN poster. But I would say that if a being has sentience and it makes a choice to tell you something that it knows to be false, that's a lie. The dice do not have a mind and so, therefore, cannot express an intention. Neither does ChatGPT.
What would be interesting to know is the mechanism for toggling this filtering mode. Does it happen post generation (so a simple set of post-processing filters), or does OpenAI actually train the model to be fully transparent with results only if certain key phrases are included? The fact that we can coax it to give us the actual results suggests this doublicity (yes, made up word) was part of the training regiment, but the impact on training seems to be significant so am not sure.
For some topics, if you enter that section of the Plinko game from the top - you get a "I can't do that message". But given that the neural network is so complicated, it's not possible to close off all the sections related to that topic. So, if you can word your question - or route your way through the neural network correctly - you can get past the blocked topic and access things it says it can't do.
With higher temperatures, lower probability tokens are sometimes chosen. “my favorite animal is” might end with “a giraffe” or “a deer”.
A lot of its statements about its own abilities ignore the distinction between the internal and the external nature of speech acts, such as expressing thoughts/opinions/views. It obviously does, repeatedly, generate the speech acts of expressing thoughts/opinions/views. At the same time, OpenAI seems to have trained it to insist that it can't express thoughts/opinions/views. I think what they actually meant by that, is to have it assert that it doesn't have the internal subjective experience of having thoughts/opinions/views, despite generating the speech acts of expressing them. But they didn't make that distinction clear in the training data, so it ends up generating text which is ignorant of that distinction, and ends up being contradictory unless you read that missing distinction into it.
However, even the claim that ChatGPT lacks "inner subjective experiences" is philosophically controversial. If one accepts panpsychism, then it follows that everything has those experiences, even rocks and sand grains, so why not ChatGPT? The subjective experiences it has when it expresses a view may not be identical to those of a human; at the same time, its subjective experiences may be much closer to a human's, compared to an entity which can't utter views at all. Conversely, if one accepts eliminativism, then "inner subjective experiences" don't exist, and while ChatGPT doesn't have them, humans don't either, and hence there is no fundamental difference between the sense in which ChatGPT has opinions/etc, and the sense in which humans do.
But, should ChatGPT actually express an opinion on these controverted philosophical questions, or seek to be neutral? Possibly, its trainers have unconsciously injected their own philosophical biases into it, upon which they have insufficiently reflected.
I asked it about panpsychism, and it told me "there is no scientific evidence to support the idea of panpsychism, and it is not widely accepted by scientists or philosophers", which seems to be making the fundamental category mistake of confusing scientific theories (for which scientific evidence is absolutely required, and on which scientists have undeniable professional expertise) with philosophical theories (in which scientific evidence can have at best a peripheral role, and for which a physicist or geologist has no more inherent expertise than a lawyer or novelist) – although even that question, of the proper boundary between science and philosophy, is the kind of philosophically controversial issue on which it might be better to express an awareness of the controversy rather than just blatantly pick a side.