How do you know the answers are correct?
More than once I got eloquent answer that are completely wrong.
More than once I got eloquent answer that are completely wrong.
I'm asking it a question about social dynamics in the USSR, what's the worst thing that'll happen?! I'll get the wrong impression?
What are people using this for? are you building a nuclear reactor where every mistake is catastrophic?
Almost none of my interactions with LLMs "Matter", they are things I'm curious about, if 10 out of 100 things I learnt from it are false, then I learned 90 new things. And these are things which mostly I'd have no way to learn about otherwise (without spending significant money on books/classes etc.)
As a simple example, LLMs will happily incorporate "facts" learned from marketing material into it's knowledgebase and then regurgitate it as part of a summary on the topic.
1. You ask someone you can trust for facts and opinions on topics, but you keep in mind that the answer might only be right in 90% of the cases. Also people tend to tell you if the are not sure.
2. For answers you need to rely on you ask people who are legally or professionally responsible if they give you wrong advice: doctors, lawyers, car mechanics, the police etc.
ChatGPT can‘t lose it‘s job if it informs you incorrectly.
I have had language models tell me it doesn't know. Usually when using a RAG-based system like Perplexity, but they can say they don't know when prompted properly.
https://danluu.com/nothing-works/
The idea that competition is effective, is often in spherical cow territory.
There’s tons of real world conditions which can easily let a firm be terrible at their core competency, and still survive.
If you had written "notaries don't even get insurance because giving bad advice is not something you can be sued for" you would be consistent.
If they lie on purpose I don’t ask them anything anymore.
The real experts give reliable answers, LLMs don’t.
The same question can yield different results.
Most people don't even know any experts personally, much less have one they could call for help on demand. Meanwhile, the unreliable, occasionally tripping pseudo-experts named GPT-4 and Claude are equally unreliably-expert in every domain of interest known to humanity, and don't mind me shoving a random 100-pages long PDF in their face in the middle of the night - they'll still happily answer within seconds, and the whole session costs me fractions of a cent, so I can ask for a second, and third, and tenth opinion, and then a meta-opinion, and then compare&contrast with search results, and they don't mind that either.
There's lots to LLMs that more than compensates for their inherent unreliability.
Most people can read original sources.
Kurzgesagt had something along the same lines: https://www.youtube.com/watch?v=bgo7rm5Maqg
(Incidentally, not that different from LLMs, once again.)
I trust some LLMs more than most people because their BS rate is much much lower than most people I know.
For my work, that is easy to verify. Just try out the code, try out the tool or read more about the scientific topic. Ask more questions around it if needed. In the end it all just works and that's an amazing accomplishment. There's no way back.
You ask an actual expert.
I don’t treat any water cooler conversation as accurate. It’s for fun and socializing.
This is also not related to the problem being trivialized in the presented solution.
Lack of access to experts, doesn’t improve the quality of water cooler conversations.
Think this thread has gone from "how to Trust AI", to "how do we Trust Anything".
I can't do this with an LLM because it does not have identity and may make random mistakes.
LLMs also lack the ability to say "I don't know", which my fellow humans have.
There’s no reason a source has to be authoritative, just because it’s a computer.
It is a bit of an adjustment, though. We are used to our machines being accurate, or failing loudly.
But, looks like the future is opinionated machines.
What a shift from twenty years ago when optimism over “information superhighways” on the “world wide web” would end knowledge gatekeeping and educate the masses, to now— worries of AI slop and finely tuned ML algorithms frying older and younger generations’ brains, while information of human value gets buried, siloed, and paywalled, with no way to verify anything at all.
Usually, stasis is the enemy that is much great than false information. If people with 90% truth can take a step forward in the world, even if they mistakenly think they have 100% truth, what does it matter? They're learning more and acting more for that step taken. If the mistaken ground truth is false and importantly enough false, they'll learn it bc their experience is grounded in the reality the navigate anyhow. If they don't learn it, it's of no consequence.
This is on my mind because I work in democratic reform, and I am acutely aware (from books like "Democracy for Realists", that eviscerate common assumptions about "how democracy works") that it often doesn't matter if we understand how democracy is working, so long as we feel like we do, enough to take steps forward and keep trying and learning. We literally don't even know how democracy works, and yet we've been living under it for centuries, to decent enough ends.
I think often about the research of Donald Hoffman. His lab runs evolutionary simulations, putting "creatures" that see "reality" (of the simulation) against creatures that see only "fitness" (the abstraction, but also the lie, that is more about seeing what gets the creature living to the next click of the engine, whether that's truth or falsehood about the reality). https://www.youtube.com/watch?v=oYp5XuGYqqY
Basically, creatures that see only fitness (that see only the lie), they drive to extinction every creature that insists on seeing "reality as it is".
I take this to mean truth is in no way, shape, or form favoured in the universe. This is just a convinient lie we tell ourselves, to motivate our current cultural work and preferences.
So tl;dr -- better to move forward and feel high agency with imperfect information, than to wait for a full truthful solution that might never come, or might be such high cost as to arrive too late. Those moving forward rapidly with imperfect information will perhaps drive to extinction those methods that insist on full grounding in reality.
Maybe this is always the way the world has worked... I mean, does any mammal before us have any idea how any of reality worked? No, they just used their senses to detect the gist of reality (often heuristics and lies), and operated in the world as such. Maybe the human sphere of language and thought will settle on similar ruthlessness.
We've come too far since the age of enlightenment to just give it all up.
Also, the vision heuristics that brains use to help us monitor motion as another obvious example. They lie. They work. They won.
https://x.com/foone/status/1014267515696922624?s=46
> Objective truth is crucial to science
Agreed. We define science and science is truth about base reality.
> Objective truth is crucial to [...] progress.
More contentious imho. Depends if progress is some abstract human ideal that we pursue, or simply "survival". If it's the former, maybe objective truth is required. If it's the latter, I find the simulation evidence to be that over-adherence to objective truth (at least information-theoretically) is in fact detrimental to our survival.
Frank Herbert, Dune
The industrialization of content generation, misinformation, and inauthentic behavior are very problematic.
I’ve hit on an analogy that’s proving very resilient at framing the crossroads we seem to be at - namely the move to fiat money from the gold standard.
The gold standard is easy to understand, and fiat money honestly seems like madness.
This is really similar to what we seem to be doing with genAI, as it vastly outstrips humanity’s capacity to verify.
There’s a few studies out there that show that people have different modes of content consumption. A large chunk of content consumption is for casual purposes, and without any desire to get mired into questions of accuracy. About 10% of the time (some small %, I don’t remember the exact) people care about the content being accurate.