Hacker News Reader

Comment by ozten | Hacker News Reader

> That humans also do this all the time is “interesting”, “dangerous” etc., but it is also why trying to move from superstition (this is actually what “reasoning by correlation” amounts to) to more scientific methods is critical for anything like civilization to be created.

"reasoning by correlation" as superstition is a brutal insight.

View on HN

It’s great that he recognizes that most people aren’t to be trusted either. Most people outside the field are talking about “safety” as though humans never lie or make up bullshit, whereas in reality I trust GPT4 much more than I’d trust an average human, just not 100%. Literally people keep hyperventilating about “fake news” generation even though their entire news consumption already consists solely of pre approved narratives, many of which are fake, by omission or otherwise.

I can’t wait for the day when ultra long context LLMs are able to point out omissions and factual contradictions in human produced fake news for any given event

Are context windows not long enough to do this for most news articles and forum posts nowadays? I'd've thought the issue would be finding the relevant and reliable sources of truth for comparison would be the difficult part.

That is difficult, yes. The way the game is played is a handful of large outlets promote themselves as “trusted” and then drown out everything else as “untrustworthy”. But the fly in the ointment for them is that even these large/sophisticated opinion manipulators (they shouldn’t be called news sources at this point) are inconsistent both over time and sometimes also in the moment. Today’s public short attention span prevents it from seeing such inconsistencies or from paying attention to the broader picture. But guess what today’s models easily outperform humans on? The ability to pay attention to more than a small handful of things for longer than a few minutes.

The comparison is not between an LLM and an average human but an LLM and an expert on a subject.

> "reasoning by correlation" as superstition is a brutal insight.

I don’t think he is right on that one, though. Reasoning by correlation is a kind of empiricism, and it can be tested. “These things happen together” or “if I do this, then that happens” can be disproven with statistical analysis even without any understanding of the underlying mechanisms at play their causes.

Superstition is beyond that; it is a belief that cannot be proven and that is not based on logic. The problem with superstition is not that there is no causal link between the supposed cause and the effect, it’s that there is not even a correlation.

Superstition is assuming a causation when you identify a correlation.

Often followed by applying confirmation bias to see correlations when there are none.

i think this is the part of superstition. Reproductive, effective, correlation is the start of a scientific path to true discoveries. Superstition only begins when the correlation has never truly never been observed, but only repeated from generations to generations without any other kind of justification other than "that's what i heard".

There is also, fashionable on some message boards: assuming a lack of causation after only seeing the title of a report of correlation..

(One may also link fashion/accusations of fashionability etc etc to superstition..)

Yes, some people confuse contrarianism for cynicism and both for wisdom.

A road trip by random walk is also a kind of empiricism that can be tested. This does not imply it is worth the energy to actually do.

A road trip by random walk is a valuable experience, the kind most people on HN lack.

Call me crazy, but I don't think the goal of AI should be to give the rock with electricity in it "a valuable experience".

What are you even talking about?

The right way to build AI. You know, the topic of this thread. I've been wondering what you're talking about, though.

There are plenty of superstitions that are perfectly falsifiable. People continue to believe in them for comfort. For example, wearing odd socks to a sports match helps your team win. Or, leaving a fan on overnight might kill you.

I think the author and you are using different meanings for "reasoning".

We do know that many people even in professional fields confuse correlation with causation. And even when that doesn't happen, when only doing measurments, correlation is often considered good enough without giving any thought to the underlying mechanism. This may be the root of Goodhart's law stating that when a measure becomes a target it stops being a good measure. The measure was only correlated with the behaviour the person doing the measuring thought was measuring.

Superstition is similar, but drops any shred of statistical rigour, relying on mere anecdotal evidence.

The problem here is that correlation is not a hard term for the people doing drawing the lines. People who take homeopathics for example will assure you they always get healthy when they took the sugar pills. What they fail to see is that they would have gotten healthy without it as well.

If you always rub your lucky penny before playing a sports game a superstitious person would account wins for the penny and excuse losses by not having rubbed the penny enough, the right way, not concentrated enough. This also feeds into correlation.

We are not talking about things actually being correlated, we are talking about people believing they are. Those are two wildly different things one is physical reality and doesn't change when you ignore it, the other is bound to individuals and tbeir world view and can take on absolutely ridiculous forms.

It seems like the other part of this is the ability of LLMs to rationalize even answers that are not rational (he says piling BS on BS). That has previously been a very human (and quite distracting/confusing) activity and it's very effective on most people most of the time.

Here's an excellent recent comment along the same lines (in response to whether our intelligence is statistically based like LLMs):

> mjburgess 9 days ago

Suppose you touch a fireplace once, do you touch it again? No.

OK, here's something much stranger. Suppose you see your friend touch the fireplace, he recoils in pain. Do you touch it? No.

Hmm... whence statistics? There is no frequency association here, in either case. And in the second, even no experience of the fireplace.

The entire history of science is supposed to be about the failure of statistics to produce explanations. It is a great sin that we have allowed pseudosciences to flourish in which this lesson isnt even understood; and worse, to allow statistical showmen with their magic lanterns to preach on the scientific method. To a point where it seems, almost, science as an ideal has been completely lost.

The entire point was to throw away entirely our reliance on frequency and association -- this is ancient superstition. And instead, to explain the world by necessary mechanisms born of causal properties which interact in complex ways that can never uniquely reveal themselves by direct measurement.

https://news.ycombinator.com/item?id=39661963

Causation is just correlation, and a good story. :-)

Reply on news.ycombinator.com