I'm not very creative but I look forward to seeing what "dangerous" things people will do when the limiters are removed. What's the worst that can happen, it writes some offensive text? Recipes on making dangerous items?
A few ideas:
Impersonation at scale. i.e. everyones image and text and sometimes video is freely available one sites like HN, Reddit and social media. Anything that requires voice authentication (calling friends, etc) is now not a thing anymore if you have videos of yourself talking for more than a few minutes online (IG, Facebook, interviews, someone recording you at work, etc)
Targeted impersonation. i.e. Train a model on politicians, local ones are better, and disrupt whatever you want by sending a video to news outlets (see the twitter hack for a bit ago where peoples accounts got taken over. Can do that with your face and voice now. Zoom, press releases, etc.)
Social engineering at scale. i.e. any text you see could be written by an model (such as your comment) and so any information you give out could be to a bad actor. Situations like DM's, etc. Those nigerian scams now just got a lot more effective.
ddos and things like that can be fluid and perpetual. "Run this attack, if it stops working, change a few parameters until works again"
Deepfake blackmail. i.e. change a pornstars face to be one of a girl/guy at your highschool and tell everyone you are going to send it out if you dont do XYZ. Even though it's not you, it looks identical so no one will believe you when you say it's not.
These are just a few off the top of my head and I am not that creative either.
That is, the "most likely" heuristic it uses to generate text seems to bypass many people's critical thinking. You have the guy who is bullshitted into thinking it can play chess or play go when it reality it struggles to draw the board or even avoid invalid movies thought it has the chutzpah (or something) to go ahead and pretend it can play the game anyway. Numerous people write blog posts where they are so amazed at something brilliant ChatGPT wrote, post it to HN, and it is clear to most of them that this emperor really has no clothes and what it said is completely wrong.
If I was trying to chat people up on (say) OKCupid I would run into all sorts of problems. Myself I say offputting things, get me to talk about my childhood and I'll say how I graduated from elementary school the same way Andrew Wiggin did. I learned that one the hard way (I would blame my neurodivergence) but if it is not that one it is another. I know most neurotypicals don't do a lot better though.
I'm pretty sure something like ChatGPT could do a lot better, particularly if it was HFRL'ed on the right material. If I had that for my wingman I'm sure I could get dates.
Now there are only so many dates I can go on with poly people and I'd still face the problem that Christian faces in
https://en.wikipedia.org/wiki/Cyrano_de_Bergerac_(play)
if I met people in person but if we changed the subject to romance scams now that would be scalable and a real business that would pay for graphics cards, software development and all of that.
I can't even imagine a worst thing at this point, but plenty of bad things, like a mass robocalling campaign to convince your grandma to send your money using your voice and likeness if needed.
Scale of bad actors actions is a quality in itself.
Look at how Google's stock price got wrecked when a demo went bad. Or how Bing had to make emergency changes to their chatbot after somebody provoked it into behaving belligerently. (Funny I demoed my A.I. to a group for the first time this morning and had no fear it would go wrong because I've used it every day since Dec 27)
If you are the first or second or third chatbot and it gets out that some sicko got it to write something sick then it is a big deal. If this is the 572th chatbot then it isn't news anymore.
As for "dangerous devices" that is surely overrated too. It's hard to make an effective bomb and you're the first person you're likely to blow up. Ted Kaczynski took nearly 20 years to learn how to make reliably deadly bombs. Dzhokhar Tsarnaev attended one of the best terrorist training camps in the world. The Morewrong prophet is afraid an LLM will spit out a DNA sequence that some rando will send to a mail order DNA synthesis firm to make a 100% deadly virus but it would take quite a bit more than that (maybe thousands of DNA sequences, a very good lab, and some highly trained researchers who aren't afraid to be killed by what they are working on) and if you had those resources you could do "gain-of-function" experiments and make very dangerous viruses without the LLM.
Long term what bothers me is not what goes on in public but what goes in private, maybe
https://abcnews.go.com/Technology/intimate-ai-chatbot-connec...
for instance we know people will follow a prophet like Jim Jones and commit suicide or that a follower of an blackpill incel could become a mass shooter, but all of those things go on in public. The CIA learned in MKULTRA that there is no magic pill or technique that can take the average person and make them into a Manchurian Candidate but if you're willing to wait for the right victim you can very much lead an isolated and directionless person down a rabbit hole.
For instance after 9/11 the FBI tried to bait a Muslim into a fake terrorist attack with a "Sting" operation and couldn't do it. Instead they found some poor black guy who thought Malcolm X was cool and he was so impressionable they were able to give him a fake bomb and fake guns and rent a synagogue for him to "blow up" and he was shouting "Allah Akbar!" when the police came for him and had no idea he was set up and knocked down like a bowling pin.
The system of a chatbot + a vulnerable human could go into very dangerous places whether or not the chatbot is specifically programmed to do that or through the mutual process of "reinforcement learning".
Now maybe that's like the fear that listening to rock music is going to make you become a drug addict or become pregnant, but it's a concern that will come up. See
https://abcnews.go.com/Technology/intimate-ai-chatbot-connec...
of course this is not something fundamentally new, as some people think the parasocial relationships that people develop on OnlyFans are dangerous and plenty of "Freezone" Scientologists have telepathic conversations with L. Ron Hubbard although it seems to me it really would be a gas to teach a chatbot to write like Ron and specifically complete all the O.T. levels that he never got around to finishing.
You know I had such enormous respect for Yudkowsky until Roko's Basilisk.
When things become even more convenient, what happens? When it's so much easier to just fall in love with a chat bot because they get you? When talking to your friends brings abrasiveness (not a bad thing btw, friends who challenge you to improve are fantastic), but your bots just actually get how to talk to you in the way you want? When you don't even know who your coworkers are or what they're like because everyone is just using their bots to convey things on slack, so it's that much harder to even make "office friends" nevermind serious ones. Even going off the deep end, because of the lack of limitations someone hooked up their model to a vibrator and now when it generates certain tags it provides you physical sensations (which, unfortunate to any readers here in the dark, there are already tools to link audio/video to your toys so this idea isn't that far fetched).
I don't think people are really prepared for how much the average person is willing to completely avoid for the sake of convenience.
Maybe what was needed was a wider understanding of media literacy and the creator's mindset, the idea that even the most tasteless and bland porn video in existence could only have possibly been brought about because humans were behind the camera shots, the acting, the production, even if they were all terrible at their jobs. Nobody cared to think about those people until now because a lot of content comes off as disposable, even with a team of humans behind it, and well-poisoning AI was relegated to jokes about Skynet.
We never considered the idea that something other than a human could also create such a thing, and at a comparable level of quality. It was an unwritten rule of creativity for centuries until generative tech made it blisteringly obvious.
Maybe "I don't need to be a chef to appreciate the food" will be rewritten in the coming years. Maybe as, "but the chef needs to be human for the food to connect."
It's going to be really interesting to see where the gradual changes happen, where we redefine concepts. I'm optimistic on that front, I don't think it's all doom and gloom and convenience does have major upsides. However I also believe that it's going to be really challenging for a lot of people if we just blindly keep walking forward without any sort of checks and balances.
What is dangerous is erotica that is better calibrated to the individual and also the development of "parasocial" relationships as in
https://www.vice.com/en/article/n7zaam/replika-ceo-ai-erotic...
I think the greatest controversy about modern erotica is (or should be) the development of parasocial relationships around platforms such as OnlyFans.
Traditional pornography seemed frequently ugly and poorly calibrated, I would point out the "pigface" scowls of models under pancake makeup that may have originated with Penthouse but somehow got copied througout the industry or numerous mannerism that you see in video pornography that are pretty ugly and offputting in my mind but that I imagine viewers think must be sexy because why would they be putting things that aren't sexy in my porn?
The new interactive pornography is much more aesthetic and better calibrated but it also promotes "simping" behavior that might be more harmful than the old pornography with all its selfish memes.