AdieuToLogic

> So far, your definitions of understanding seem to be whatever vibe you are going for in the statement and I would urge you to think about what a sensible mathematical definition of understanding is so that it can sensibly be assessed on neural networks.

Any reasonable definition of understanding is not dependent upon "whatever vibe you are going for", but instead must include at least an English dictionary definition of "understand" such as:

  to grasp the meaning of[0]

And, for further clarification, "grasp" can be defined as:

  to lay hold of with the mind[1]

Which makes an equivalent term-expanded definition of "understand" to be:

  to lay hold of with the mind the meaning of

As such, there is no "sensible mathematical definition of understanding", unless you possess a complete mathematical model of the human mind.

>> Understanding is a state of mind

> This is meaningless, it is a circular definition at best.

See above to as to why there is meaning in what I wrote.

>> It exists entirely within the individual and nowhere else

> Then why are we talking about it? What is the point if it is something that can only be defined per individual?

I like to think analyzing fundamental premises, often implicit, explicitly can help to identify fallacious positions.

> We are still not getting anywhere because you have not prescribed criteria to determine whether [an LLM] understands.

My apologies for being opaque. Let me clarify:

  LLMs do not "understand".  People interpreting LLM output
  are the only entities involved which can "understand",
  because "understanding" exists strictly within each
  person who possesses it.

0 - https://www.merriam-webster.com/dictionary/understand

1 - https://www.merriam-webster.com/dictionary/grasp

> These days if you're arguing from a position you took even a week ago, you already might be out of sync.

"Might be out of sync" is a unique spelling of "crucified online". :-D

> Pattern recognition becomes "understanding" once individually recognized concepts become sufficiently sparsified [sic] and compactified [sic].

Understanding is a state of mind. As such, it exists entirely within an individual and nowhere else.

For example, take any two university professors who teach the same subject where one only speaks Arabic and the other only speaks Vietnamese. Each will not be able to understand what the other says, regardless their understanding of the shared topic.

> I argue that for any proper definition [of understanding] you provide which humans satisfy, a strong LLM is very likely to satisfy that as well.

This is demonstrably incorrect as detailed above. There is no "understanding" LLMs can satisfy as we know it, since to certify said "understanding", it requires interpretation by a person to "know" an LLM "understands."

> I also would not argue that humans are "simple token generators". That is not what I said.

That is the essence of what you wrote, unless you object to my use of "simple" instead of "statistical". In this context, I postulate this is a distinction without difference.

> I said that just about everything can fall under the classification of "statistical token generators" at an abstract level, so it isn't a useful distinction.

This only holds if one subscribes to statistical token generators being a/the fundamental underpinning of "everything". Here is a proof by contradiction:

  If everything can be classified as a derivative of
  statistical token generation, how does one explain
  quantum physics?

> Then disprove the physical Church Turing hypothesis in regard to the human brain.

The onus is not mine to disprove a hypothesis you have chosen to mention in passing. The responsibility is yours to prove said hypothesis or at least contribute meaningfully with some amount of credible research.

Or try to learn from Proverbs 17:28[0]:

  Even fools are thought wise if they keep silent, and 
  discerning if they hold their tongues.

Either works for me.

0 - https://www.biblegateway.com/passage/?search=proverbs%2017:2...

>> Again, LLMs do not "hallucinate." They are statistical token generators whose results are dependent upon their training data set and involve a degree of randomness. >> Nothing more.

> You can say that about everything in a human brain.

> What's the most significant, strongmanned, core difference that makes silicon doing number crunching "nothing more" and brains "something more"?

The fact that you formulated this question, in and of yourself, without "prompting" from me or anyone else.

Cogito, ergo sum.[0]

0 - https://en.wikipedia.org/wiki/Cogito,_ergo_sum

For context, in response to my original statement:

  [LLMs] are statistical token generators whose results are 
  dependent upon their training data set and involve a degree 
  of randomness.

This is literally what was written:

  During conversation, we are statistical token generators 
  whose results are dependent upon our training set.

>> If you do not see a difference between humans conversing (known consciousness as defined by humans) and the output of an LLM (known algorithms as defined by humans), I don't know what to say.

> That’s not what they said.

How did I misquote and/or mischaracterize any the above?

> To be fair, they also have “adieu” (farewell) to logic.

It is always a joy when a person, such as yourself, finds the irony in my moniker.

Thank you for this.

> You ever heard of something called a metaphor, guy?

In this media (comments in HN threads), all I can do is interpret what people write. ;-)

> And "hallucinating" to mean "outputs plausible sounding gibberish that doesn't hold together consistently". Of course there's no actual hallucination going on.

This may very well be what you know to be true and I have no reason nor desire to assume otherwise. The problem is... Many people use the word "hallucinating" in this context literally and not metaphorically.

Since I do not know you, how am I to tell the difference?

>> If you do not see a difference between humans conversing (known consciousness as defined by humans) and the output of an LLM (known algorithms as defined by humans), I don't know what to say.

> Whatever you might think about your own abilities, most individuals can't tell the difference.

I have yet to see an LLM say "hello" to a neighbor. I have done so and can definitively assure you "most individuals" can tell the difference.

>>> I'm not anthropomorphizing anything ...

Yes you are, regarding LLMs at least. Here's why:

  just for fun I asked a reasonably smart LLM to ...
  [be] capable of understanding if it's gone off on
  a hallucinatory path ...

"Smart" in this context is a subjective value judgement. "Hallucinations" are only experienced by living organisms.

You then went on to state:

> If you know something rare and the LLM does not, you'll immediately see when it's hallucinating an answer or answering factually.

Again, "hallucinating" is not something an algorithm can do. Also, determining factuality is again subjective based on the person assessing the information.

>>> How is this different from a human using an algorithm they have memorized ...

>> Humans identify which "algorithm they have memorized" to use beforehand, due to the problem to be solved being defined by other humans ...

> This doesn't make any sense at all. Was this supposed to be a gotcha?

No, it was meant to be an explanation as to the difference between "memorization" and "understanding." In this context, people pick the algorithm they determine applicable and then the question of memorization is relevant.

> An LLM is trained on problems defined by other humans, and identifies which algorithm it must use based on pattern recognition.

Funny that you make this argument here, where when I wrote elsewhere in this thread:

  [LLMs] are statistical token generators whose results are
  dependent upon their training data set and involve a
  degree of randomness.

  Nothing more.

  ...

  It is pattern recognition, a task in which ANNs excel.

To which you replied to the above with:

  During conversation, we are statistical token generators 
  whose results are dependent upon our training set. 
  Seriously, write that definition out rigorously. It 
  encompasses virtually everything. It is totally 
  meaningless. So to say "nothing more" is effectively also a 
  tautology.
  
  This argument was asinine in 2024. It is insane to be 
  saying these things in 2026. Where have you been?

  ...

  It absolutely understands how to do math, by whatever
  reasonable definition you want to provide to the word
  "understand".

So which is it?

Are LLMs ANNs? Which themselves are pattern recognition algorithms (hint: they are)?

OR (setting aside the ad hominems you kindly provided)

Do LLMs possess "understanding" of concepts such as abstract mathematics (defined and interpreted by humans) and we, as simple humans, nothing more than statistical token generators as you assert?

Because it cannot be both.

> During conversation, we are statistical token generators whose results are dependent upon our training set. Seriously, write that definition out rigorously.

If you do not see a difference between humans conversing (known consciousness as defined by humans) and the output of an LLM (known algorithms as defined by humans), I don't know what to say.

> Heck, just for fun I asked a reasonably smart LLM to ...

LLMs are neither smart nor stupid. They are statistical token generators whose results are dependent upon their training data set and involve a degree of randomness.

> You still have to be skeptical of its results and capable of understanding if it's gone off on a hallucinatory path ...

Again, LLMs do not "hallucinate." They are statistical token generators whose results are dependent upon their training data set and involve a degree of randomness.

Nothing more.

See also anthropomorphism[0].

> More precisely it's that [LLMs] can't do the math internally but they're quite capable of producing the tool that does the math.

This still falls under the purvey of statistical token generation. To wit, given enough variations of:

  bc -e '1 + 2'
  bc -e '41 + 1'
  ...

LLMs can identify the addition expression in "What is 4 + 1?" and then emit a `'bc "4 + 1"'` command to produce a response. This is not "doing" or "understanding" math.

It is pattern recognition, a task in which ANNs[1] excel.

0 - https://en.wikipedia.org/wiki/Anthropomorphism

1 - https://en.wikipedia.org/wiki/Neural_network_(machine_learni...

>> they found bits of code that are associated with "math" and the supplied arguments

> How is this different from a human using an algorithm they have memorized, or reading it from a reference site written by a human and then writing the same formula into a custom one off piece of python code?

Humans identify which "algorithm they have memorized" to use beforehand, due to the problem to be solved being defined by other humans, which leads to...

Wait for it...

Understanding.

Right and wrong are forms of judgement, subject to debate, and sources of resentment.

Correct and incorrect are at least possibly quantifiable and do not intrinsically involve subjectivity.

In other words, my being incorrect is something others can help me to understand and rectify. My being "wrong" is a position which can yield second order harm.

A friend of mine used to work at Veritas[0] making enterprise data retention solutions. When I spoke about their product as being "making backups", he corrected me by saying:

  We are not in the backup business.  We are in the restoration
  business.

0 - https://en.wikipedia.org/wiki/Backup_Exec

> Judgemental much?

Not often, but yeah... I did on this occasion.

> Anthropic's goal is to commoditize intelligence.

And Google's original goal was to organize the world's information.

How did that turn out?

> I'm curious what people are sending to Claude that is so secret.

When Claude is used in a professional setting, any or all of:

  Proprietary intellectual property (a.k.a. system code)
  PII[0] of the employee, customers, or both
  HIPAA[1] data known to a system
  Internal communications not meant to be publicized
  Sensitive data, such as SSH keys and the like

Pretty much anything on a machine which uses Anthropic/OpenAI native tools is a candidate to be compromised really.

0 - https://en.wikipedia.org/wiki/Personal_data

1 - https://en.wikipedia.org/wiki/Health_Insurance_Portability_a...

The simulated dialog you have provided is very much representative of what many of us have heard first-hand. However, I disagree with:

  Most businesses don’t care about later risk or any future 
  planning beyond the quarter horizon, they’re not concerned 
  about how it will effect their performance in 3 quarters or 
  lead to instability or issues, those are future problems 
  for a future person and we’re here for money now.

Businesses care about "later risk" and what it implies.

Individuals within an organization do not unless it will specifically affect their bonuses/promotions.

> If your house burns down because of known flaw in Microsoft Windows they are not liable (well as far as EULA let’s them, you can most likely still sue them).

A EULA does not obviate responsibility of a company for its products. Continuing with your example, while it may be very difficult to prove a known flaw in MS Windows was the cause of your house being set afire, if one had said proof, a EULA would not absolve Microsoft.

This reminds me of the old joke[0]:

  There are 10 kinds of people in the world.
  Those who understand binary and those who don't.

0 - https://www.gnu.org/fun/jokes/10-kinds-of-people.html.en

> What is "await" if not a synonym for "suspend"?!?

The `await` keyword in most languages is not a synonym for suspending thread execution so much as it is an effectual attempt to replicate the functionality of `coreturn`[0]. To wit, if an underlying `Future`/`Promise` has completed before the `await` instruction is evaluated, the thread executing same will not be suspended.

0 - https://www.euclideanspace.com/maths/discrete/category/highe...

Like many things in complex projects such as operating systems, accepting PRs is not as simple as "if this one use of a tool works, why not accept it?"

Why? Because the onus of determining fitness of purpose and correctness has to be done by someone. So if a PR is buggy and/or lacking in test coverage, but was made by a contributor making an honest effort, then experienced contributors can help educate the person as to what is missing or needs to be considered. Ultimately, this is a form of apprenticeship.

Contrast that with someone using an LLM to produce the exact same PR. Who can be mentored? Who can be referred to materials which can deepen requisite knowledge?

> has anyone found a good way to improve code quality? just wondering ...

  - keep prompts focused on atomic tasks.
  - use expert prompting[0] when possible.
  - require coding agents to verify changes.
  - require coding agents to create/update unit tests with 100% coverage.
  - use git to commit/revert atomic tasks manually.
  - leverage planning capabilities to review instead of recover.
  - consider using something like Karpathy guidelines[1].
  - leverage Constraint Programming[2] concepts when
    formulating prompts.

0 - https://arxiv.org/pdf/2305.14688

1 - https://github.com/multica-ai/andrej-karpathy-skills

2 - https://en.wikipedia.org/wiki/Constraint_programming

> Optimizing for fewest LOC is probably slightly more bad than more LOC, and both are bad for the same reason - it makes it harder for humans to interpret and understand ...

Concision begets perplexity.

;-)

> It's more like if someone dumped lab-grown diamonds on the side of the road that are virtually indistinguishable from mined diamonds.

Equating LLM generated code to "lab-grown diamonds" is disingenuous at best.

> Someone would be happy to take them and [be] none the wiser.

If the quality of GenAI PRs were equivalent to those authored by experienced contributors, then there would not be as many GenAI PR bans in FOSS operating system repositories.

> The only thing I got out of this article is that both parties + the F500 need to be destroyed.

Destruction is easy, as we are all witnessing.

Making things better is hard. It takes commitment to incremental change, often having to overcome setbacks, and accepting that the product of democracy is inherently nonlinear.

Democracy also demands that its participants understand that most of the time what each individual wants to be, will not be exactly as they envision, since that would be autocracy.

> A separate document explaining how a system works can go out of date.

Good documentation does not explain "the how", it captures "the why."

> I am convinced that there are paid posters on here.

Maybe. More likely is there are AI bots used to proliferate messages much cheaper than "paid posters."

> Why would anyone defend a billion dollar corporation so vehemently and not earn a single dime?

Because some like the products made by said billion dollar corporation?

Because others like to be contrarian?

Because not everyone is motivated by being paid?

Maybe, just maybe, people are different and have differing motivations not easily reduced into being "am I getting paid for this or not."