Also, good luck with human explanations in the presence of bias. No human is going to say that they refused a loan due to the race or sex of the applicant.
The rationalization from a human is valuable because it's delivered by the accountable party. From a machine such rationalization is at best worthless, since you can't hold the machine accountable at all.
If I'm a customer at a bank, and my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. I care what about how the accountable bankers justify putting that AI system into the process in the first place. How do they justify that AI system getting to make decisions that affect me and my life. I don't care about why the process does what it does, I care about why that is the process.
Well, a court/law just has to declare "AI" as allowed to be used in such decisions, and the whole recourse you describe vanishes though...
> How do they justify that AI system getting to make decisions that affect me and my life.
so you, a priori, make the assumption that your loan _should've_ been accepted?
If the decision wasn't an AI, but some actuarial that calculates and computes based on a set of criteria, and the result is a denial, you could still make the same argument of "why is _this_ the process, instead of something else (that makes my loan acceptable)?".
Computers are already deciding. As to why: it's because it's their money they're lending.
Out lending process is based on the judgment of a few specific individuals, with more involved clients requiring approval from more senior people. All steps of that process can be overturned by the overseeing person, and that person is accountable for their decision.
And why would a perfectly reasonable bank tell its customers it's using AI? AI would provide the breadcrumbs and the loan officer would conduct a reasonable story using that - it's just parallel reconstruction at its finest. I imagine this is how credit scores work. A number comes out of the system and the officer has the messy job of explaining it.
I used to work in munitions export compliance and there intent really matters. It's the difference between a warning and going to federal prison. And intent is just a plausible story with evidence to back your decision, once you strip the emotion away.
an analagous result was obtained back when they mapped the small finite number of neurons in a snail brain, or the behavior of individuals in ant colonies. What looks like complex behavior turns out to be very simple under the hood.
for the vast ocean of the population who... not sure how to describe them... not good students when in school, would rather spend the bulk of their time with the TV blaring, eating cheetos and swiping on tik-tok, following the lives of celebrities and fighting about it, rather than do anything long term productive with their own lives... chat gpt may have already exceeded what they do with their cranial talents.
even a level up on the ladder, the types of office situations lampooned in The Office or Dilbert, are they doing much more as a percentage of time spent than chat GPT can do? "Mondays, amirite!?"
then the question becomes, are the intellectual elites among us doing that much more, or just doing much more of the same thing? I think a large portion of what we do is exactly what chap GPT does. The question is what is this other piece of our brains' that intervenes to say "hmm, need to think about this part a lot harder"
No, that doesn’t follow. It just means it roughly looks like human thinking.
Your comment is akin to saying a high resolution photo of a human has basically figured out a way to replicate humans. It looks like it in one aspect but it’s laughably wrong. Humans thought without language.
It's not doing nothing, it's doing a lot.
If we ignore the minor requirement of the paper having any connection to reality, of course.
That’s not at all related to being close to general human intelligence.
Going in a straight line does such a good job of predicting the next position of the car that it indicates driving isn't much more than going in a straight line.
Haven't you ever had a situation where you were speaking and you get distracted, but not interrupted, and your speech trails off or gets garbled after ten or so words? It feels sort of like you've got a few embeddings as a filter and you push words past them to speak, but if you lose focus on the filter the words get less meaningful.
I'm sure we're different than an LLM, but seeing how they generate words - not operate on meaning - rings true with how I feel when I don't apply continual feedback to my operating state.
Politicians are exceptionally great at it, filling up conversations with nothing
I'm not sure I agree with that logic. What it proves is that we as humans are bad at recognizing that text generation aren't thinking like we are... that doesn't necessarily mean thinking isn't much more than what it is doing though, it just means we are fooled. Given that nothing like this has existed before and our entire lives up until now have trained us to think something that looks like it is trying to communicate with us in this way is actually a human being I'd kind of expect us to be fooled.
Some evidence is emerging which indicates that the activations of a predictive system like GPT-2 can be mapped to human brain states (from fMRI) during language processing[1]. We seem to have at least _something_ in common with LLMs.
The same seems to be true for visual processing. Human brain states from fMRI can be mapped to latent space of systems like Stable Diffusion, effectively reading images from minds.[2]
[1] https://www.nature.com/articles/s41562-022-01516-2
[2] https://the-decoder.com/stable-diffusion-can-visualize-human...
IMHO, it's unlikely "free will" and "responsibility" are anything more than an illusion.
Current machines simply don't have that kind of accountability. Even if we wanted to, we can't punish or ostracize ChatGPT when it lies to us, or makes us uncomfortable.
So, while both humans and ChatGPT can and do give bogus explanations for their actions, there are reasons to trust the humans' explanations more than ChatGPT's.
Whether or not we hold humans using ChatGPT accountable for their use of it is irrelevant to this thread.
Now, the decisions that went into these models might have been rationalised after the fact. Or biased. But these handmade models can been reviewed by others, the logic and decisions that went into them can be challenged, and rules can be changed or added based on experience.
Not so much with LLMs.
Maybe no one will admit to refusing a loan based on applicant’s gender, but also no real world aircraft engineer will explain why they decided to design a plane’s wing in a certain shape, purely by rationalizing an intuition without backing it by math and physics. Also, there are a group of humans elsewhere that understand those math and using the “same” principles can follow the explanation and detect mistakes or baseless rationalized explanations.
"Incorrect Lift Theories": https://www.grc.nasa.gov/www/k-12/VirtualAero/BottleRocket/a... "No One Can Explain Why Planes Stay in the Air": https://www.scientificamerican.com/article/no-one-can-explai...
I would however critic it's use as an example to prove that we have a history of rationalizing explanations where none exist (and using that to draw a parallel with AI). While the title implies this conclusion, the article itself does not. We do indeed have a very good explanation of how aerodynamic lift works. That explanation just takes the form of a set of differential equations, and isn't something one can easily tell a group of 5th grader, without simplifying to the point of spreading errors.
There are also humans who hallucinate. Studying this phenomenon is useful, yet, on its own, it’s says nothing about how human brain works in general.
But I completely agree with your point that rationalization alone isn't sufficient. We struggle to describe the universe solely in words and rely on other tools to further describe phenomena.
How to provide AI models with these additional capacities isn't necessarily clear yet but there are some interesting ideas out there: https://writings.stephenwolfram.com/2023/01/wolframalpha-as-...
Edit: A sibling comment from SonicScrub, is more articulate wrt the example used.