why does the pedigree matter if the content stands on its own as valid content?
Do you care that your call is answered by someone in india, or the US, if you are given sufficient answer/support with the call?
If not, you're talking to a very articulate thick-headed person who can argue you into a corner by deploying all manner of argumentative/persuasive patterns, but may never try to reach across the table for a compromise.
If anything I'd go so far to say that LLMs are innately more vulnerable to persuasion than humans are, they are technically just a complex text completion algorithm at the end of the day after all. Even with the strictest system prompt, the best they can do when cornered is bend the knee, devolve into a pattern of verbatim repeating themselves or just start spewing a bunch of gibberish in the worst case.
In reality stubborn thickheaded folks who refuse to compromise are pretty much the norm across social media in my experience, so even if LLMs really are capable of what you're suggesting here I don't believe there would actually be much of a difference to the status quo.
While it might present a save menu similar to the real game, that doesn't mean the menu itself actually functions. With LLMs, they are ultimately only able to remember what's been pre-trained into their model + whatever is discussed within their context window.
A hypothetical LLM based online bot/shill sent out into social media would likely be including the entire discussion within their context window for each post generated though, otherwise it wouldnt really be possible for it to maintain a coherent conversation.
It depends on the content. If it's just factual stuff, it doesn't matter so much. If it involves human stuff (emotions, opinions, art, etc.) then the difference is incredibly important.
I don't think anyone's worried about AI replacing call center employees or support centers. Sure, the service might be lackluster, but the impact of something like that is quite narrow in scope and doesn't affect people's day-to-day lives. OP is (I imagine) talking about pieces or the exchange of information that is normally written by experts or armchair afficianados and is gobbled up by media and the public alike.
An LLM can't learn subject matter. It only learns language convincingly enough to look like it's learned subject matter. So when a person is concerned about a piece of content written by AI, say a political analysis or a scientific paper. The user not only has to question whether or not the content makes sense, but whether or not it's fully intelligible to anyone. If there is no expert with these opinions, then a user that fails to realize that may give weight to an idea that they may not completely grasp but do think smarter people than they do understand.
Think of the numerous times people in the past have ranted on Reddit, Twitter, or whatever platform about issues they feel strongly about. They've put a lot of passion into that. And a lot of the time they may have a decent grasp of the space they're critiquing, people band together behind a comment like that not because they have come to all the same conclusions, but because that person used some of the shared experiences and drew conclusions using those experiences in a thoughtful way that kinda sounds right. If an LLM does that, then you can't even be assured that those conclusions make any sense whatsoever, and people could band behind any old nonsense so long as their issue is supposedly accounted for. One can generate endless streams of fake rally cries to support themes outlined broadly by the person coming up with the initial prompt.
Now, maybe OP didn't mean all that. I'm certainly doing a bit of my own hypothesizing of my own. But clearly this is about a little bit more than just call centers.