Comment by hammyhavoc | Hacker News Reader

Comment by hammyhavoc | Hacker News Reader

Absolutely. This can be applied to anything and everything in its output, and this is why I take a very dim view of how quickly this has been trotted out, yet how little understanding the average person has of the maths behind it.

You should try asking LLMs about politics, specifically policies of each party, ask it to help you decide who to vote for et cetera. You'll very quickly see it spout absolute bollocks, especially if you're talking politics beyond the USA or Canada. Extra nonsensical if you start using languages other than English.

Even asking it questions about COVID can yield some nonsensical responses in regards to politics.

Having it gaslight you about the NHS and policies of each party is also amazing, especially when you tell it that it's wrong and it tells you that it isn't. With elections looming, these things are the next Brexit bus or Cambridge Analytica waiting to happen.

>especially if you're talking politics beyond the USA or Canada

I think globally we've very quickly hit LLM limits because they were simply not trained on as much information as we think. Or at least, not nearly as much region-specific information in many countries.

I've hit the same issues trying to ask any LLM to give me tax advice for tax law in specific countries, and even beyond the "ask a professional" type restrictions eg OpenAI has, it's clear that it just does not have the information.

Sometimes that's just because of the US-centric view of these companies training the models, and sometimes it's also because information sources are tucked away on botched government Wordpress sites and random PDFs.

This is why you have to do things like RAG to make a "talk to your doc" kind of bot even if you're feeding it public info.

The extra danger is when we don't know it comes from an LLM. If we think a human wrote it, we're less skeptical.

I've had a few less tech-savvy friends tell me dubious things this past year and asked where they heard it—usually "this neat app I found called ChatGPT". eyeroll

Reply on news.ycombinator.com