Comment by PaulHoule | Hacker News Reader

Comment by PaulHoule | Hacker News Reader

Parent Full thread

There seem to be two kinds of people: ones who see the mistakes ChatGPT makes and the ones who don’t.

If I was a scam artist today I would be very happy to find people of the second kind to victimize.

Secondly I’d be very concerned that the second kind of person is working at the hospital dispensing drugs or on the beat as a cop or doing any kind of job where mistakes could have consequences. They're not just going to be making mistakes and letting other people's mistakes go by, they are going to be denying it and making excuses about it.

If ChatGPT has a core competence it is that people give it the deference that they give to high status people, who are used to just saying whatever comes into their head and having people just nod. I think of the story of “the Emperor’s new clothes” where the Emperor pulls off something that not everybody could do.

These concerns converge in the 419 scams which deliberately filter for the second kind of person with deliberately poor grammar and spelling. Amazingly there are people who have $10 million dollars to lose who fall for this! I wonder if there is somebody evil, bright, hard-working and ambitious who is HFRL training a model right now to do romance scams.

This is a false dichotomy.

I'm acutely aware of ChatGPT's mistakes, have spent lots of time investigating them, and still find immense value in the tool.

While it's true that many less technical people will end up treating it as an oracle, let's not conclude that it's impossible to understand what it is and use it productively despite its flaws.

I’ve frequently been the ‘final assembly programmer’ for projects that were started by some fresher that got to what looked like 90% done by management but it was unsound underneath and that last 10% took 50% or more of the work.

There is nothing I find more fatiguing than pushing a bubble around underneath a rug in a system that eventually has to be carefully taken and inspected bit by bot and a huge amount of work put into figuring out what exactly is wrong with it and fixing it.

I’m afraid the ChatGPT enthusiast is going to walk away from a looming disaster thinking they and ChaptGPT is so brilliant, probably never realize the harm they did, and if they do they’ll be contemptuous of the low-status ‘grinds’ who take so long to make things right.

> There is nothing I find more fatiguing than pushing a bubble around underneath a rug in a system that eventually has to be carefully taken and inspected bit by bot

Couldn't agree more

> I’m afraid the ChatGPT enthusiast is going to walk away from a looming disaster thinking they and ChaptGPT is so brilliant

Also agree.

But none of this detracts from ChatGPT's value or incredible power.

It just means that it will also cause many problems. Maybe once you net everything out the balance sheet is negative. I think you could make that argument for social media, for example. But the point is moot, because it's here, and its influence is only beginning.

> There seem to be two kinds of people: ones who see the mistakes ChatGPT makes and the ones who don’t

Agreed; it quickly became apparent once I found myself entering the 'seeing its rough edges' camp.

Once you've _tried_ breaking it and can see the limits of an LLM you'll find yourself wishing for an AI assistant, but knowing what is and isn't possible makes managing one's expectations with new technologies easier.

I'm still generally an optimist with these new techs because they're still very cool and potentially useful interfaces to existing technologies, but I agree with you in that it's too easy to get caught up deferring to it like it's some techie oracle (and that the tendency for people to want to do so is concerning).

I used to think people wouldn't "simply accept" the types of systems in _Minority Report_ or _Psycho-pass_, to the extent that I found it to be immersion-breaking. But, concerningly, it seems folks are happy and willing to give up that deference (in what's probably a well-studied sociological observation I'm simply unaware of the term for). Scary stuff.

I love Psycho-Pass, it's the best police sci-fi anime since Patlabor.

I'm one of the ones who sees ChatGPT's mistakes. It makes lots of them. Some are very dumb.

But the idea that it's a fraud or a scam or whatever label the naysayers choose to put on it is just wrong. It makes mistakes, but it also provides a lot of value when used correctly.

Consider the many use cases where you don't need to rely on its "knowledge" at all - just this morning, I used it to write some marketing emails. I've got a paragraph about the company that I give it, plus some specifics about the particular email, and it knocks out something that's entirely useable in seconds.

There are plenty of tasks like this, in which it's creating something based on your input, in which it is immediately and clearly useful. Just because there are use cases where it fails doesn't make it a scam.

I would wager that the recipients of marketing emails do not consider them valuable. The prospect that in the future, these will not even have human creativity flowing into them just adds insult to injury.

I've got a few thousand people who have signed up for them and haven't yet unsubscribed. Whenever I send out a marketing email to that list, a number of people immediately make purchases.

Why would that be the case if people didn't consider them valuable?

Once in a blue moon I make a purchase after seeing an ad, yet I do not value watching ads. How would that be the case? You tell me.

When my "genius" boss first heard about ChatGPT etc., he was convinced that this would end up replacing all the developers in our org. I told him that the more likely COA was that it would replace all the middle managers who just collate TPS reports to push to the VPs.

He was not amused.

Seems like it will be perfect tool to generate power points, reports and so on that no one cares or reads...

Who has the belief that every answer ChatGPT gives will be accurate? It's hard to imagine many people reaching that belief.

Is there any evidence of the existence of scams which deliberate use poor grammar to filter respondents?

Somehow i believe the critiques and opponents of the current AI wave are as much prone to hallucinating their critique as is ChatGPT with facts.

Whatever. Despite its occasional mistakes, I have found it to be incredibly useful.

Reply on news.ycombinator.com