Hacker News Reader

Comment by ziofill | Hacker News Reader

A direction that is 99.99% accurate survives this argument completely. For all practical purposes one does not need totality.

View on HN

For a one time coin flip, sure. For a lot of them, depending on the stakes, that can be very, very wrong.

https://www.google.com/search?q=why+99.99+accuracy+is+not+en...

It doesn't take a lot of thought experiments to realize this. Imagine if every bite of food we eat had a 0.01% chance to turn into something instantly lethal in our mouth. Average lifespans would be reduced measurably, and apart from anxiety, we'd develop all sorts of strategies and laws around that. E.g. absolutely NO eating for airplane pilots. You wouldn't go on a date to have dinner, dancing and sex, you'd go dancing and have sex, and then have breakfast. People would modify their jaws and stomachs so they could eat less, but bigger chunks of food. It would be a whole thing!

And that's not even talking about water changing on us, or a tiny chance of getting sucked into the toilet whenever we use it, and a lot of other things where going from damn near 100% to 99.99% would change everything for the worse, by so much.

ziofill's claim was that "A direction [in an LLM's embedding vector space] that is 99.99% accurate" is fine for practical purposes, not that 99.99% is fine for the chance of any given bite of food not killing you or similar hypotheticals - you'd want a few more 9s there.

To justify relevance of inability to correctly answer liars-paradox-type questions ("what won't your response to this be?"), the article suggested the way LLMs are used in practice is dependant on them being entirely accurate truth oracles:

> > as a truth-oracle [...] is how these things will be used practically by the vast majority of people. They are already replacing standard Google search results

But for the replacement to make sense they just need to be more accurate than what they're replacing (ignoring other factors like convenience and cost) - in this case standard Google search results and knowledge box which were obviously not 100.0% accurate.

I think you are making a lot of assumptions on what "practical purposes" even mean in that case.

Please think carefully and then try to tell me whether or not some Trump-administration government agency would not in the "99.99% reliable" circumstance just plug an LLM into the nukes and funnel worldstate input into it and have it make the decision "is it time to fire the nukes?" over and over again each second.

I argue that that would require far more nines than even "will food turn to poison in my mouth" would.

> plug an LLM into the nukes and funnel worldstate input into it and have it make the decision "is it time to fire the nukes?" over and over again each second [...] I argue that that would require far more nines than even "will food turn to poison in my mouth" would.

Sure - but (even assuming that's a practical purpose) the point is it that it doesn't need to be a 100% accurate truth oracle, which is all the article's argument prohibits. If the current human chain of command has 99.99999994% accuracy, then 99.99999995% accuracy is an improvement and not ruled out by the argument.

A basic course in statistics will inform you of why a 99.99% accurate test should be looked at with skepticism when diagnosing a rare disease. Yet we see the fancy 9s and think somehow this many 9s is enough.

Sure, but that’s not what we are talking about

Diagnosis of rare diseases is a subset of "true" statements, no?

he's referring to frequency. a 2.9T parameter processing a 1M token data will inevitably produce wrong "false" tokens, guaranteed.

So even if you have a 99.99% pure date set of the "true" variety, the errors will aggregate to some very less than 99.99% characteristics.

Realm of big numbers problem.

Really? Laws of probability works against your argument.

Such a basic course in statistics needs to be scrutinized.

Every breath you take, there is a 99.99% chance that everything is normal, and a 0.01% chance that you breathe mild acid which horribly burns and causes a massive coughing fit.

It probably don’t cause long term damage unless that breath happened to be more important than normal, like while driving right as a child runs into the road.

Would you act differently knowing that you had one of these occasional acid breaths? Even if it only happened once per day on average (0.001%)

And your point is? Laws of probability doesn't discriminate.

There is always a probability a planet killer GRB hits us and we all die. Should we take precautions in our everyday life. Or should we wear helmet while walking on the road pray a coconut not falling on our head is just 99.999%.

You are correct, but this is a made up example, not a practical matter. You’re falling into the same trap as the author.

You must have come to rescue him :) from the "incorrect" trap.

If 1 in 10,000 people have a disease, then a "test" which always reports the patient doesn't have the disease will be correct 99.99% of the time. "99.99% accuracy" should be "looked at with skepticism" in that it doesn't tell you what you need to know to understand the quality of a a test for a rare disease (a classifier under conditions of severe class imbalance); at a minimum, you would want to understand it's false positive and false negative rate, not (just) it's overall error rate.

See example "A": https://en.wikipedia.org/wiki/Base_rate_fallacy

If only stating the obvious was a fallacy :)

You appear not to have understood probability theory my friend. You will never get 100% in this universe for anything. What if "its a simulation" or "a dream" arguments ensures you never acheive 100%.

Bayes probability theory will be a good start for you.

https://en.wikipedia.org/wiki/Bayes%27_theorem

Well, I do agree that all measurements contain error, but the point wasn't that the error rate would be greater than 0% but that a single headline summary of error can't always distinguish between good and bad tests.

Ironically, some users took the “all” in my first post too literally. It should have read “for most practical purposes”

Reply on news.ycombinator.com