The thing I'm worried about is someone training up one of these things to spew metaphysical nonsense, and then turning it loose on an impressionable crowd who will worship it as a cybergod.
I think he was just early. But his moral compass did not seem uncalibrated to me.
Many are already getting very sucked into believing new-gen chatbots: https://www.lesswrong.com/posts/9kQFure4hdDmRBNdH/how-it-fee...
As of now, many people believe they talk to God. They will now believe they are literally talking with God, but it will be a chaotic system telling them unhinged things...
It's coming.
https://www.reddit.com/r/singularity/comments/10gr7dq/how_do...
> You see, your humanity (everything that makes you human) is based on one lived experience, your own, AI's will be based on thousands or millions of lived experiences so it will be more human, than the most human of humans.
https://www.reddit.com/r/singularity/comments/11430n3/the_si...
> I am at a loss of what to do until the singularity comes, knowing that careers (and work in general) are soon to be a thing of the past. I am in the middle of pursuing a masters in data science, and while I enjoy it, I would not be doing it if it weren't for monetary motives. It seems like working hard and spending so much time advancing my skills in data science/analytics will be futile. For that reason, I am thinking about dropping out of the program.
> I spent an hour chatting with Bing today and it was one of the best conversations I’ve had in a long time.
> I have two kids under two, I wonder if there is any point in saving for college. At this rate I doubt they'll ever have to work.
and my favourite:
> I guess the Bing version has attitude and feelings programmed into it.
Seriously though, given how people are reacting to these language models, I suspect fine tuning for personalities that are on-brand could work for promoting some organizations of political, religious, or commercial nature
https://www.dailymail.co.uk/news/article-1232848/Face-Virgin...
As such, I would absolutely love for a super-human intelligence to try to convince me otherwise. That could be fun.