It will be interesting to see. People are already involved in business-critical processes, and they have a far-from-100% success rate. You can see this with self-driving cars; 0.00001% of the time, they decide to swerve into oncoming traffic for no reason. But they can't get drunk! So whether or not that's a win is something that only time can tell. (People will be outraged, because no person would ever do what the AI did, but what matters are the numbers in aggregate. One could also argue; why should I have to put up with this when I never drink and drive? It's a good point and involves philosophical discussions about free will and all that.)

I guarantee people will be mad when an AI chatbot denies their insurance or customer service claim. But I have a friend who works in customer service and it feels like half her job is correcting the mistakes other customer service agents made. If the first line support was AI, would it change much, other than not paying people to make a bunch of mistakes?

Again, it's going to be painful while we try it out and get the data. It feels like it's going to be worse, and individual anecdotes will make each fuckup seem even worse than it is. But it seems inevitable that we're going to get the data. The only way to get VC writing software right now seems to be by tying your company to AI. That means a lot of experiments are about to leak out into the world. We'll learn a lot.