Given that this is 'research' carried out (and seemingly published) by a company with a direct interest in selling you a product (or, rather, getting investors excited/panicked), can we trust it?
I think Anthropic genuinely cares about model welfare and wants to make sure they aren't spawning consciousness, torturing it, and then killing it.
They say it doesn't have that much to do with the kind of consciousness you're talking about:
> One distinction that is commonly made in the philosophical literature is the idea of “phenomenal consciousness,” referring to raw subjective experience, and “access consciousness,” the set of information that is available to the brain for use in reasoning, verbal report, and deliberate decision-making. Phenomenal consciousness is the form of consciousness most commonly considered relevant to moral status, and its relationship to access consciousness is a disputed philosophical question. Our experiments do not directly speak to the question of phenomenal consciousness. They could be interpreted to suggest a rudimentary form of access consciousness in language models. However, even this is unclear.
Not much but it likely has something to do with it, so experiments on access consciousness can still be useful to that question. You seem to be making an implication about their motivations which is clearly wrong, when they've been saying for years that they do care about (phenomenal) consciousness, as bobbylarrybobb said.
Language models are a novel/alien form of algorithmic intelligence with scant relation to biological life, except in their use of language.
They go further on their model welfare page, saying "There’s no scientific consensus on whether current or future AI systems could be conscious, or could have experiences that deserve consideration. There’s no scientific consensus on how to even approach these questions or make progress on them."
I've grown too cynical to believe for-profit entities have the capacity to care. Individual researchers, yes - commercial organisations, unlikely.
For anyone having paid attention, it has been clear for the past two years that Dario Amodei is lobbying for strict regulation on LLMs to prevent new entrants on the market, and the core of its argument is that LLMs are fundamentally intelligent and dangerous.
So this kind of “research” isn't targeted towards their customers but towards the legislators.
- it's a threat for young graduates' jobs.
- it's a threat to the school system, undermining its ability to teach through exercises.
- it's a threat to the internet given how easily it can create tons of fake content.
- it's a threat to mental health of fragile people.
- it's a gigantic threat to a competitive economy if all the productivity gains are being grabbed by the AI editors through a monopolistic position.
The terminator threat is pure fantasy and it's just here to distract from the very real threats that are already doing harm today.
The mechanism which causes job less is that when your competitors automate, all the business goes to them because they're more productive.
Automation of field labour has decreased the worker count by a factor 20 or something.
Same for the mining sector.
It's not necessarily a bad thing as working in the fields or in coal mines wasn't pleasant, but pretending automation doesn't reduce employment is nonsense.
They mostly stopped working in those fields because everyone hates farming and mining and quits the first chance they get.
Here's recent evidence from Canada, Japan and Spain showing automation caused employment increases:
https://pubsonline.informs.org/doi/10.1287/mnsc.2020.3812
Either unemployed or forced to work in even less desirable places, yes.
> They mostly stopped working in those fields because everyone hates farming and mining
https://en.wikipedia.org/wiki/1984%E2%80%931985_United_Kingd...
No matter how hard the work conditions are, people don't usually accept its disappearance.
Automatization reducing work can actually be a good thing, as it is the reason why we can have vacations, retirement and long studies: because the society's need for work is lower than before.
It basically just shows you're looking for a way to dismiss something that doesn't require you to understand it or check their work.
It seems completely obvious that AI companies benefit massively from (and in many cases likely only continue to stay afloat because of) 'research papers' like this.
I also don't think a scientist purely interested in the truth would be claiming anything about concepts like 'introspection' that are nebulous and only really serve to capture the imagination of the general public (and, of course, investors).
The difference between AI and the pharmaceutical industry should be clear: one produces products of undeniable value, and the other is largely built on hype and endless dreaming of what might come next, but so far hasn't.
It's relevant if it's not preregistered. I agree this one is not preregistered and they should release their model weights instead of doing random tinkering on it themselves.