Hacker News Reader

Comment by Gasp0de | Hacker News Reader

I think this is really dangerous. People often take what they get as a search result for granted, and if you ask for facts, ChatGPT will straight out lie to you but make it sound very plausible.

For example, I asked chatGPT what percentage of electricity in Germany was produced by renewables, and it said 85.6% and quoted the federal environmental agency. When I went to the FEA website, it clearly said that only 41% are generated by renewables.

View on HN

"Really dangerous" is a bit dramatic. Most search already has an info pane for many queries, and it's often wrong or incomplete as well. The world hasn't ended. This just adds more noise

This seems like a tractable problem in the near future, by rewarding the model for factual accuracy.

A reward is not a constraint. In the language of modern ml, rewards 'encourage' models to produce certain constrained outputs. The actual outputs during inference can be arbitrarily poor in spite of the added 'reward' during training.

It's really dangerous for AIs and their proponents, actually. Once people start mistrusting these results, it'll be a slippery slope. I've generally just stopped using ChatGPT entirely because of this.

It also seems to be 120% sure that the word 'air' ends with the letter 's'.

Still I would imagine that you could build a dedicated postprocessor that fact checks the text and another that fixes any math errors.

Given the vast number of possible mistakes the model can potentially produce, i doubt such an error correcting program would be easier to implement, than a LLM that does not make those mistakes in the first place.

Sounds like politicians may be the first to be replaced by AI.

Reply on news.ycombinator.com