There are many ways AIs differ from real people and any conclusions you can draw from them are limited at best -- we've had enough bad experiments done with real people
https://en.wikipedia.org/wiki/Stanford_prison_experiment#Int...
It's so weird to live in a time when what you just said needs to be said.
For us layman, the flaw of using AI trained on people for surveys is, human. Humans have a unique tendency to be spontaneous, wouldn’t you say?
How would a focus group research team approach this when they’re bombarded by AI solutions that want their research funds?
>unless they involve Donald Trump
A sense of shame perhaps. If you ask someone "how often do you brush your teeth" and compare it to more pragmatic testing, you see people have some sense of wanting to give the "right" answer. Even in a zero risk anonymous survey.
A YC company just launched doing exactly that.
I would assume there is so much in the corpus based on behavior optimized for the actual existing social media we have that the behavior of the bots is not going to change because the bot isn’t responding to incentives like a person would it’s mimicking the behavior it’s been trained on and if there isn’t enough training data of behavior under the different inputs you’re trying to test you’re not actually applying the “treatment” you would think you are.
But i have a footnote of "yes" because as you said, decision makers are just not interested in having this discussion about "focus on making fun games". So it will unfortunately affect my job in the short and even medium terms. Because so much of big money in games these days is in fact not focused on making a game, but on trying to either generate a gambling simulator, an engagement trap, or (you guessed it) AI hype. Both to try and claim you can just poof up assets, and to try and replace labor.
Knowing this, I do have long term plans to break out into my own indie route.