I'm not Catholic, but this podcast presents a very interesting argument against talking to AI as if they were human: https://newpolity.com/podcasts-hub/debate-chatbots
After the damage is done, the mega corps would simply shrug and will say "Well, we were just responding to our business competition" The business knows no human-ness, because it is not a human. Businesses and machines are creatures that see humans as their fodder. And humans created these, assuming it is progress, to have businesses and machines. We call it progress because it required our mind power and it helped us to dominate other species. Dolphins are laughing at us.
Like saying I no longer walk because I can drive. I no longer cook because Doordash exists. I no longer play piano because midi exists.
I mean i guess, but it seems like you didn't LIKE crafting the text or coding from scratch, you just wanted the outcome. If we are talking purely about work, I understand that its about being productive and it sucks to have a job shift to something you enjoy less.
But for daily life? I dont see how it changes, maybe its a tech thing where people think about making their daily lives more productive, but most people dont.
You can CHOOSE to let AI think for you or achieve for you, but you dont have to. You only HAVE to in a work context (for some people), which I admit sucks (or is great depending on who you are).
This feels like a touch grass moment.
But when I am with friends, out for a mtn bike ride or playing golf, we are just hanging out like we did 20 years ago. I dont see any cars and I am sure there are tons of AI apps for both sports that would make me "better" but I really don't care. I don't do it for the outcome.
The bonding, happiness, sharing, laughing, are all there just like it always has been and always will be.
It's like trying to have meaningful conversations on Twitter. You don't go to Twitter to do things like that. Can you? Sure. It's just not what the format and the conventions (and the people) lend themselves to.
I don't think there's much merit in pretending that human activities are only shaped by hard limits.
I think the parent is saying now that that is attempting to be applied to "creativity" directly, as opposed to something like a shift of medium, that it threatens many peoples' ability to maintain creative capabilities.
Anecdotally I've already experienced this at work where post-AI we had a junior completely stagnate and a senior with over a decade of experience in the bay atrophy to the point that he had to be let go.
Our devices have now increased the distance _between_ us -- it's not about _you_ being able to "do X" -- talking to others is not _you_ doing it, it's you _and the other person_ doing it _together_. You can't be doing anything together consentually when the other person is in the habit of talking with their AI, or doomscrolling for that matter.
I spent a few weeks at a hostel last year. It was always kind of depressing and tense in the shared kitchen, just this heavy silence.
I don't feel comfortable around strangers, so I solved that problem by just saying hi to everyone.
Most people didn't respond much, although most of them smiled and the tension was eased. But a few of them struck up conversation and we ended up making friends.
I ended up making like, ten new friends in two weeks. And then a bunch of them ended up becoming friends with each other as well.
Even if you do it, you are still swimming against the current.
I.e. it's only a problem if you're not willing to go and strike up conversations with people. Which is not AI or mobile phones' fault. Expecting society to come up with "systemic situations" to your personal problems is a fast path to a lifetime of disappointment.
It is not a problem if there are other people willing to also go and strike up conversations with people. That is why GP wrote that it is a systemic problem that can be solved on a personal level.
> Expecting society to come up with "systemic situations" to your personal problems is a fast path to a lifetime of disappointment.
It is not a personal problem. It is a systemic problem. Are you conflating on purpose or do you not understand the difference?
And it is a systemic problem, all you need to do is look at the data.
Uhm, those lonely people need to get out and start talking. How is this going to help society? This is going to make it worse.
Oh that kid is kinda quite and sad? Throw him an iPad. Oh that adult is kinda bored and wandering aimlessly? Throw him into a casino. Oh that adult is kinda lonely and feels like they don't have anyone they can talk to about their life? Give them LLM companionship.
Yep, it is over for humanity. People simply don't understand externalities.
Great, what is being done to help that happen?
Even if all I am saying is bullshit, "what is being done to help"? What about what is being done to make it way, way, way, way, way worse. If I had LLM companionship when I was alone, yea, I would have never gotten out of my shell. I would be stuck in there forever. Hell, why should I even talk to you? I should just argue with a robot instead.
For some of these people even talking to a robot would likely be a huge improvement in quality of life, and that's just talking. If said robots could also help them out in real life, sort of like a personal assistant, that would be even better.
I mentioned in another comment that I totally get it for these people. You are using the extreme cases to justify opioid for the masses. The moment you "solved" human loneliness and interacting by giving them drugged up with chatbots, its over. I am telling you, it is over.
AI psychosis is bad enough as it is with really poor imitation. The moment it can imitate and fool the average person, I just don't see why most people need real friends anymore. We will NEVER be able to get it out of kids hand, the cats out of the bag. You think kids growing up with social media and ipad is bad? Lets see what happens if they live in actual fantasy land with their robotic friends.
I don't even know if I want to live in that world anymore.
Also, your solution to lonely people with disabilities or old age is to create robotic "Friends" for them? This is just sickening. You might as well just drug them. The _actual_ solution is to create a compassionate society where this doesn't happen. The solution to overworked society isn't more opium to relieve the pain.
> Social people will be fine
Thing is, "social" is not only a scale, it's not even a line for scale. It's this multidimensional space of different social preferences and different preferences as to _how_ to socialise, not to mention with whom and for what reason. In short, it's much more complicated than as to permit a "binary" division into the "social" and "other" people, but even if you did, I would wager the division puts it at 50% for either group, meaning you're basically describing a cataclysmic event where 50% of the population suddenly has no access to a fundamental trait of humanity that in known and unknown ways has assured our feeling of happiness and warded off a plethora of conditions of misery and worse that psychology has no names for (and won't ever have names for because that's like classifying different gradations of drinkable water, I imagine).
But yeah, humanity has always had to evolve, it's just that these changes come too fast for our adaptability to adapt to, I believe.
This is almost certainly going to bite us in the ass long term, because eventually without human creativity you're just training AI's on other AI slop, or limiting the possible catalogue of styles to "things that existed before AI ruined every creative job". I guess the question is, what kind of future do you want to live in? One where we have a massive abundance of easy-to-create but vaguely worthless artifacts in a society that's completely devalued being good at something? It just sounds really dystopian to me.
AI doesn't stop creative people from creating. it gives creative people an additional tool with which to create
Exactly, so why make it worse? Why try to automate an industry when the money you make off killing it wouldn't even cover the cost of your research?
> AI doesn't stop creative people from creating. it gives creative people an additional tool with which to create
I can't emphasize enough how much actual creative people hate this shit. Here's a relatively balanced take, although the headline spoils it (How AI Poisons Creativity: https://www.youtube.com/watch?v=WeCSzEtZcUw ). Long story short, the struggle is where creative ideas come from. If you automate away the process you automate away the result.
Ai doesn’t technically stop me writing code. But it’s sucked so much joy and interest out of it, that I can’t scrape together the motivation to work on side projects anymore. It has a “chilling effect” the effects of which are notoriously indirect to observe.
Yup. Except, by God, we ought to make sure the business has unlimited free speech (i.e. campaign contributions).
So thinking corporations and such were created to push human progress is laughable.
I've wanted a good voice mode for precisely this reason. When I take my dog on a walk and I'm thinking about a bunch of problems/ideas, I'd love to have feedback and a record, or perhaps to even kick off research or ask questions to fill in gaps that would otherwise have me debating pulling out my phone to try to get an answer.
I don't think having a child-like sense of wonder at the world, or continued human ingenuity, is a sign that one wants to be a child forever.
I'm genuinely sorry if you've lost that spark.
>You don't need take a dog for a walk, if you live in a normal place that allows dogs to roam freely.
I love my dog and love walking him, even though he can freely roam on my property. I also love walking with my wife, and I don't think her agency has much reason for why I might enjoy it?
>Your mind is a servant of your body. And your body never asked you for that.
...are you okay, friend?
If you think AI is bad, wait until I tell you about the horrors of social media who profit on controversy and division, US health insurance which profits on rejecting claims, and big pharma profiting on the opioid epidemic.
And it's not like this is new, either. Upton Sinclair was writing about this stuff a century ago with books like The Jungle.
The only difference between then and now is we "think" we're not evil today. We've lied to ourselves that "We're so much better than we were back then." Facebook wanted to bring people together originally, but they ended up providing the most toxic social media experience known to man. Facebook forgot to tell us they cared more about profit than people.
Please spare me the argument AI is the straw to break the camel's back here. The system has been broken a long time before that.
Valid, but, I think, you conflate two separate things.
AI voice mode as a human socialization/conversation replacement? Cringe in my book, fully in agreement with you. Though my opinion on that aspect remains the same, regardless of whether it is done through text or voice.
AI voice mode as an alternative interface to interact with AI-as-a-tool? Great idea imo. There were a few instances where I was either too tired to type or wanted to brainstorm things in more of a freeform mode, for which a well-working voice mode would have been great.
Naturally, the current distinction between AI-as-a-personality and AI-as-a-tool exists purely on the user's end. All I know is that I want the latter a lot, and if some people want to use it for the former purpose, that's not my problem. Sadly, I think that it will be judged more on how an average person decides to use it (i.e., in the most degenerate/reductionist ways possible), as opposed to being judged on the merits of what it can actually be used for by someone who just treats it as a tool.
But there are people that treat it like a therapist or a friend.
In my opinion, the 2nd group should be discouraged from using these agents in this way, as it cannot be healthy. But if places like Reddit etc. turn into bot towns where do the AI companies get insight into the general population? Through the eyes of their most mentally unstable users?
So I don't really see things changing, and they'll probably get worse.
don't you dare clump the best fruit in the world with cockroaches!
I think you're mistaking their sarcasm for sincerity... especially considering the emphasis on self-identity ironically juxtaposed as originating from a decidedly non-self activity, which has all the hallmarks of being intentional...
On the other hand, reading their other content leads one to believe that they may, in fact, be serious... hmm...
Very blatantly and obviously not though???
If your answer is "well I'll simply touch grass" I agree. But most people won't which is why this is tech is immiserating and, I would argue, evil.
It's possible that an "AI boyfriend" might make someone less prone to put in the continued effort to keep rolling the dice on dating apps, but the reality is that there's a more fundamental problem driving this.
Also, I want this tool for work. Just because society is fubar and people are using this tool as a crutch for their inability to find a partner, doesn't mean I should lose better tooling that makes my life easier.
Focus on fixing the actual problem.
Things in society don’t exist in a vacuum, and “who cares about them, so long as I get my needs” is quite literally a part of the problem being discussed.
Just because things do not exist in a vacuum doesn't mean they should be blurred and conflated until unrecognizability. There's value in explicitly anchoring the salient aspects, and getting the cause and effect right, to the extent these can be done.
Both intelligence amplification and pathological parasocialness are very real effects of this technology, and a blanket ban, a blanket halt, and similar broad and dull policies are absolutely going to be unfair and unreasonable. And even such policies would not "happen in a vacuum", there's a consequence to prohibiting something not prohibited elsewhere.
Whether we can do better is where the jury is still out. I don't think an example like your dirty needles one particularly helps with this. It's asinine and inflammatory, quite the opposite to nuance-inviting, which I believe was more towards your actual intent.
As someone with a GLP-1 user at home, responsibly depositing their syringes in an approved container that we dispose of safely at a drop site, I'd rather not have bad users inhibit my access to a tool that makes our lives better.
Anti human, corrosive, powerful, uncanny... but not evil?
I am fully encountering this, believe me. My conclusion is that it's evil.
Powerful and uncanny are morally neutral traits.
At root an LLM is a different kind of mind that thinks different kinds of thoughts. (I’m making no claims about consciousness here.)
It’s a novel object for us humans, and we’ll need decades to come to grips with it. In the meantime I think a bit of humility is due.
I use these tools at work. If I talked to other people instead of a model every time (or even just some of the time) I talk to a model, I'd be tanking everyone's productivity by constantly disrupting them, and would get worse quality responses pretty much guaranteed, if I wouldn't be just ignored outright.
This isn't to say I stopped talking to colleagues either. I reach out to them about the same amount. The time budget I took away from was the one I used to spend manually doing things, reading docs, and so on. So your entire mental model of this is just outright false there.
On a personal level, I traded off entertainment time (though for me, interacting with ChatGPT is also just a form of entertainment). Instead of staring at YouTube, I pitch silly thought experiments, explore topics, ask difficult-to-search-for questions. So once again, I did not trade off social time, meaning what you propose simply does not apply whatsoever. I do not share memes or catch up on life with it, which is what friends are for. It has no sense of actual humor, has no life of its own, and I'm not into roleplay like that. Would be just weird.
There are people who engage pathologically with LLMs, but that is neither the premiere use nor the goal. I'm sure there are also situations where people will actively approach a model over a person, but I do believe people should be able to self-regulate that decision. It gives people leverage to choose the extent and venues of their social engagement. If a topic has a community you do not wish to interact with, you can finally choose not to, while still reaping any prospective benefits (information). This can be a bad thing, but I think it's easy to see that it is also very much good.
For example, there were times where I felt more compelled to interact with ChatGPT than I felt leaving a comment here on HN. And frankly, I'm pretty sure I was way better off, and maybe the people here were too. Or a connected but different example, there were HN comments I took to heart pretty bad. Relitigating them with ChatGPT enabled me to get over them in-depth, and even consider different viewpoints properly, without having to be the resident punching bag of some asshole. It's difficult for me to see these as anything but a net improvement.
There is something to be said about interactions in one's life becoming overly structured. I think a reasonable case could be made that in less structured lives, where things just kind of happen, you get more random interactions, and it is through those random interactions that you gain valuable experiences and relationships, even if there's a lot of cruft in addition [0]. That in our effort at optimization, we're making our lives inhuman to live, for the same reason people are crapping on LLMs being statistical (unreliable, imperfect) rather than hardwired (reliable, perfect). But that implies there is an optimal life, which I'm not sure exists: specifically because of this local vs global optimality tension. In complex stateful systems, you're guaranteed to hit scenarios where locally correct and globally correct behaviors are irreconcilable [1], unless you meticulously construct them such that this cannot happen, after all.
[0] like https://en.wikipedia.org/wiki/Small-world_network
[1] like https://en.wikipedia.org/wiki/Tragedy_of_the_commons
Read the GPT Live announcement. Why do they want to make it more life-like, if not to encourage the pathological engagement? Why does the model say "thank you for that insightful question"?
It is not "just a tool", that is not the goal. It is designed to be a replacement for people. Even in the most charitable "LLMs are just a tool" reading. The value proposition is to have fewer people (dramatically fewer: the "one person unicorn" with fleets of AI agents!) and for those people to spend more time talking to bots. Replace your junior employees with bots. Replace your customer service people with bots. Replace a bored teenager behind the Wendy's counter with a servile golem whose smooth plastic face can easily be wiped down when spit on.
Their metric is engagement, it is how engaged are you and how long do you spend using the application. They will do absolutely anything to get that number up. Remember, after the GPT-4O debacle, when Altman talked about implementing sex bots? That's a shark, smelling blood. Something like 30% of under 18's have used AI chatbots for "companionship," a big subset of those for ERP.
> Would be just weird.
I was a weird kid who spent 10-12 hours a day on the computer, back when computers were primarily low-res screens and boring text. Now everyone is spending 10-12 hours a day on the computer. These things change in a single generation and I'm sure you know as well as I do that boy does it happen fast.
You sound like me: an older, healthy, well-adjusted person who can understand what's going on when the model says "that's a really insightful question!" Most people are not. They will not understand when it's an AI video clone of Chris Helmsworth.
> there were HN comments I took to heart pretty bad
I bet it made you feel alive.
In the video example with the grannies, the knitter is essentially wanting a PA. Regular folks don't have PAs. Even when that became a thing in the aughts they were all outsourced.
When I've used voice chat, it has often turns into rabbit holes on very niche topics. For example, I had one start about the 1996 performance of Rage Against the Machine in Portland, Oregon that was supposed to feature Wu Tang Clan. (already outside most human's knowledge) that dove into details of the club scene in Los Angeles at the time of RATM's signing to Epic Records.
Was anyone else here at that '96 show in Portland? It seems like it might be challenging to find a person on the internet able to engage on the topic.
The person may exist, but not during my fleeting interest in the subject while walking to the park.
Human beings tend not to be available (results vary by culture).
Also, imagine you're 82 years old and living alone (e.g. widower). It is believed that lack of interaction is a significant driver of cognitive decline (which is why being hard of hearing accelerates the onset of dementia). I wonder if having an LLM to talk to under those circumstances will decelerate cognitive decline?
Is this sarcasm?
Otherwise it's kind of like being manipulated by a psychopath
I am open to the possibility that this is a future that is coming, but as far as I’m concerned, we’re years away from that tech. Is it actually here for you?
Each time I hear the bot emote, it's like a subtle suggestion to just play along and anthropomorphize it - it feels weird, silly and annoying
There is a difference in expression / emotionality with speaking vs writing. Speaking tends to carry more emotion while writing is generally more deliberate/less-emotional*. Having a voice conversation will be more likely to get a human to engage in an emotional based expression mode, which could increase the chance of "false connections", believing the AI "gets" them or "understands" or "listens". This happens with text too, as some headlines show.
The issue is that while some people are going to "connect" with their AI in text and voice, some who do not make the connection via text may do so via voice because it tends to change a persons expression mode.
> I definitely talk to AI as if it were human
Do you talk to it as if it were a friend or family? or Do you just use natural language to give directives? The distinction, I believe, is in the kind of way we express the "talking".
I talk to AI as if it were a tool that understands human commands and then executes those commands and relays them in a human understandable format. This includes commands to provide options that I may not have covered and explain the options. If I talked to a human this way, they wouldn't be around much longer -- unless they were an employee and even then they would probably be looking for a new job
After reading some of the psychotic break headlines from AI chats, I see some people really do talk to AI as if it were human. Which I would guess includes seeking broad "thoughts and feelings" on a persons situation or asking the AI if their view/side of things is the "right or wrong" side. Basically begging the AI to be responsible for their own thoughts, or simply offloading them and taking what comes back -- which is going to be what they wanted to hear because the entire context would be full of emotion based prompts.
*I forget which books Ive read about this in. It's not an obscure concept, quick search brought this up: https://kellercenter.hankamer.baylor.edu/news/story/2023/spe...
I just want voice assistants to reliably understand what I say and do what I mean
So hopefully you can turn that off.
There are plenty of applications for that more human conversational style though (from mock interviews, improv practice, learning languages, etc) - I just don't want that for most things.
- nobody else with the skills - talk to machine - nobody develops the skills to have a conversation - go to step 1. - bonus step: your own skills atrophy too.
There’s plenty of evidence that these models worsen these issues, and we know from prior experience that companies like FB will happily exploit vulnerable users for money.
Besides, I do not think there is anything inherently immoral with not being social, or not having the ability to be. Consider for example people who do not naturally have the social network to interact with people they want to (e.g. some gifted children).
I am not convinced this model has enough empathy to satisfy most users on an emotional level. A bond is not merely an exchange of words, but prolonged and deep contemplation of the other being. We cannot introspect into these machines, and they certainly cannot yet do the same to us.
There is no moral obligation, in any domain, to refuse to make a product that adults, with full informed consent, find useful and purchase. Who are you to say you know better than the market?
> Who are you to say you know better than the market?
You really don't think the scientists & engineers making these tools know some things better than the market?
There is no meaningful competition in a marathon if I can drive you over with a car at the first 100m.
And thanks fkin God that we have regulations and meaningful laws and some asshole for-profit company can't just put drugs into food.. the "informed buyer" is bullshit. Humans are faulty, and there is a billion dollar industry meant to take advantage of said faults: it's called marketing.
Pretending that everything’s totally fine and ok because “it’s adults” really just disregards reality.
Disclaimer: I am from Scandinavian.
I'd be more worried about the inevitable robo-nannies who could end up talking more to young children than actual people.
“Driving to work or running a 10km on the weekend sharing the same surface goal: from A to B.
You may have less autonomy whether you should drive to work, but you definitely have more decision power about whether to go for that run or not”
I don't understand the counterfactual. What's the opposite direction of this direction that's desirable? Less capable voice models are obviously not it, so I am curious what direction you mean or if it's just vague indirection.
Moreover I think aggresive robocalls and advertising will derail the entire trend in a few years, people will become tired of fake talk.
I personally think this is a moralistic regulatory overreach. And they definitely do that due to political pressure too, since there are various bills around the world in various legislatures that want to regulate AIs giving useful advice and being too personal to talk to.
So you can rest assured, I think, at least in that regard. The AI disempowerment will come to us anyway, just in a more sanitized corporate form.
"The way that I formulate the argument, just to get us started here, is that acts, the things we do, have ends. We act for an end. They have a purpose. And that it is wrong to deliberately frustrate an act from attaining its end, from attaining its purpose."
I will draw your attention to the three instances of "end", first we have the noble, tautological, "have ends", actions have ends; then, following close behind we see "an end", we act for an end, those ends have been made into one now, singular end; and finally, "its end", the act, as an entity unto itself, has an end you mustn't interrupt.