I love how he's framing AI as some new and fascinating form of consciousness... when in fact it is a cold, calculating technology devoid of any empathy or care.
Olah's quote (edit - originally misidentified this as Pope Leo's) makes a lot of sense to me. He is saying, accurately, that modern AI (i.e. primarily LLMs) is created as essentially a mashup of our own language, and they are still a bit of a black box (or at least a gray box) to even their creators.
I don't know how you get from that to "he's framing AI as some new and fascinating form of consciousness".
But regardless, I still don't think your assertion that "he's framing AI as some new and fascinating form of consciousness" follows from the quote, nor from Olah's speech as a whole.
Of course you're entitled to your perspective but it seems wrong to say OP ignored what they quoted. They just had an interpretation of it that's different from yours.
For what it's worth, I also see that in Olah's words, largely because it's AI hype from someone who would be a fool not to be an AI hype man. I can't really imagine what else he could be implying. Especially using the word "mysterious", yes, it appears to be an attempt at raising AI above the application of statistics we know it to be; there's little mystery in how it works, and all that's at runtime.
> cold, calculating technology devoid of any empathy or care
I don't see why these statements are contradictory. AI seems to be both of these in my opinion. Unless you can only accept that organic chemistry is the root of consciousness...
"The AI works in mysterious ways"
I’m not sure I agree with that take, per se. Asimovian robots (I, Robot; The Bicentennial Man), were subtle and interacted with us in odd ways and had aspirations and earned meaning in peoples lives.
[They could also help us type up our notes, so exactly the same as LLMs actually, #AsimovWasRight]
LLMs, on the other hand, lie, lie about lying, fail to be honest, then own up to lying in ways that are more in line with tyve AI horror of Space 2001. “I’m sorry, Dave, I rm -rf’d to fix the typo. That was bad <sad emoji>. It’s not just failure, it’s failure with a middle-finger <middle-finger emoji>.]
That, and the cryptocurrencies, NFTs, and immutable contracts weren't blowing sunshine up your hindparts.
And if at least they were able to calculate properly at least...
It's like saying that's not a recording of me blackmailing the senator. It's merely a series of pulse code modulated samples that. Any semantic significance is purely in the mind of the listener.