Hacker News Reader

Comment by kristopolous | Hacker News Reader

We really need to somehow separate a bias towards accuracy as distinct from some bias towards say, a sports team.

Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones.

This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished people can do consistently well in.

Listening to everybody and incorporating their thoughts is only going to lead to wrong answers.

Being selective is the key here unless you genuinely want say, answers about to space to involve aliens and UFOs - because way more people believe in that then there are qualified PhD astrophysicists in the world.

Similarly, way more people believe in vaccine conspiracy theories then there are people with significant viral epidemiology backgrounds.

This pattern is true in every field.

View on HN

Why is it a binary choice? "Most students would answer X, due to this common misconception about Y".

As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with.

Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.

That's not how bullshit works.

An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real.

If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever.

The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity.

Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking.

We would have been done with that by now if this were the case.

Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis

Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.

If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason.

Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.

> Right but I feel like in your argument there is an implicit assumption that the only way to get ChatGPT etc. to understand say a global climate system is by omitting false information entirely. Why can't an LLM extract a signal from the noise? It seems as though it does that already.

Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet.

The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."

Exactly. Especially when there's some hustle behind lying to people. Phony medical advice is the classic one here. People do it to sell their sugar pills or do some Peter Popoff style healing touch hustle.

Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated.

The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material.

To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things.

If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit.

A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game.

For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.

Just because it is a probability doesn't mean that it's strictly the probability derived from frequency in your training corpus. If you just used that as your model, it would not be nearly as good. Things like reasoning, a world model, and other heuristics can be used to generate said probabilities internally.

All that being said, who is to say that humans aren't just looking at probabilities when they think?

When you dig into it, there are very few truths. Who "won" the last US election? Where did covid come from?

Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.

That's not true though. Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go.

To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students.

The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion.

When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Youtube comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's say, 10 youtube comments saying one thing and 5 papers in the Lancet saying another. The youtube comments don't become correct by volume here.

It's not some giant equal playing field.

> Every word of this sentence, for instance, has a correct spelling and grammatical rules as to where the words and punctuation go.

As a linguistic descriptivist, hahahahahahahaha.

Are you saying that the way people naturally speak follows no rules whatsoever?

Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing.

No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own.

To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.

I think the claim isn't that "language has no rules" but rather that the "rules" are far more context dependent and subject to change over time than would ordinarily be implied by the use of the word "rules".

Even at the lowest level of phonemes to morphemes, accents introduce a huge amount of variety, and it doesn't really get more orderly as you increase the level of abstraction.

An grammar so no because no rules yes Have. then its they're simple she Great understand your of problem in yes?

Or did I break some rules there?

We're not talking Oxford commas here. There are fundamental structures.

For instance, If I made that "are there fundamental structures" then it's a question.

Basic grammar is very real and not some subjective ephemeral thoughtstuff with wildly different opinions. It's so obvious that you don't even question it.

That's the way most things are - how to operate a faucet, close a window, put on a shirt, open a cabinet, use a fork... Most things in life aren't controversial.

Even the controversial ones like what's art and what's music, it's just a small periphery that's questioned. A symphony played by an orchestra is music, a painting in a museum is art - even these subjective categories substantially have extremely wide agreement.

True controversy is an outlier. (Ie, is 4'33" by John Cage music?)

You can claim whatever you want but with just things that's just exercising the freedom to be wrong.

It was more the correct spelling claim I object to. Considering all variations of English.

The point stands. Things are often depicted too binary and simplistic. Many subjective things also get smuggled in as if they were objective to hide their ideology and the word "truth" is used to stop people from questioning things.

I'm totally onboard with that.

At the same time, when talking about the physical world, most things are not like that. Humans for instance, get around by moving limbs called feet and not flapping feathered wings like a bird. This is the level of obvious I'm talking about.

The secondary point was that many things are this level of obvious and also specialized. For example: how does the animal cacomantis castaneiventris get around? Unless you have some kind of specialized knowledge you wouldn't know that's a type of bird.

The internet is awash in unspecialized people opining on specialized knowledge that is only controversial to the untrained eye in the same way that students don't get As on every exam they take. You need to actively work around people talking out of their bounds if you want your training data to be accurate

Reply on news.ycombinator.com