Comment by perching_aix | Hacker News Reader

Comment by perching_aix | Hacker News Reader

Parent Full thread

I largely agree, and upon reading this article is sadly also in that camp of applying this perspective to be dismissive.

However, I find it incredibly valuable generally to know things aren't magic, and that there's a method to the madness.

For example, I had a bit of a spat with a colleague who was 100% certain that AI models are not only unreliable because from a human perspective, insignificant changes to their inputs can cause significant changes to their outputs, but because, in their idea, they were actually random, in the nondeterministic sense. That I was speaking in hypotheticals when I took an issue with this, as he recalled my beliefs about superdeterminism, and inferred that "yeah if you know where every atom is in your processor and the state they're in, then sure, maybe they're deterministic, but that's not a useful definition of deterministic".

Me "knowing" that they're not only not any more special than any other program, but that it's just a bunch of matrix math, provided me with the confidence and resiliency necessary to reason my colleague out of his position, including busting out a local model to demonstrate the reproducibility of model interactions first hand, that he was then able to replicate on his end on a completely different hardware. Even learned a bit about the "magic" involved myself along the way (that different versions of ollama may give different results, although not necessarily).

I also had to argue with a lawyer on the same point - he held a firm belief that “Modern GenAI systems” are different from older ML systems in that they are non-deterministic and random. And that this inherent randomness is what makes them both unexplainable (you can’t guarantee what it would type) and useful (they can be creative).

I honestly find this kinda stuff more terrifying than the models themselves.

> [The] article is sadly also in that camp of applying this perspective to be dismissive.

TFA literally and unironically includes such phrases as "AI is awesome".

It characterizes AI as "useful", "impressive" and capable of "genuine technological marvels".

In what sense is the article dismissive? What, exactly, is it dismissive of?

> TFA literally and unironically includes such phrases as "AI is awesome". It characterizes AI as "useful", "impressive" and capable of "genuine technological marvels".

This does not contradict what I said.

> In what sense is the article dismissive? What, exactly, is it dismissive of?

Consider the following direct quotes:

> It’s like having the world’s most educated parrot: it has heard everything, and now it can mimic a convincing answer.

or

> they generate responses using the same principle: predicting likely answers from huge amounts of training text. They don’t understand the request like a human would; they just know statistically which words tend to follow which. The result can be very useful and surprisingly coherent, but it’s coming from calculation, not comprehension

I believe these examples are self-evidently dismissive, but to further put it into words, the article - ironically - rides on the idea that there's more to understanding then just pattern recognition at a large scale, something mystical and magical, something beyond the frameworks of mathematics and computing, and thus these models are no true scotsmans. I wholeheartedly disagree with this idea; I find the sheer capability of higher level semantic information extraction and manipulation to be already a clear and undeniable evidence of an understanding. This is one thing the article is dismissive of (in my view).

They even put it into words:

> As impressive as the output is, there’s no mystical intelligence at play – just a lot of number crunching and clever programming.

Implying that real intelligence is mystical, not even just in the epistemological but in the ontological sense, too.

> But here at Zero Fluff, we don’t do magic – we do reality.

Please.

It also blatantly contradicts very easily accessible information on how a typical modern LLM works; no, they are not just spouting off a likely series of words (or tokens) in order, as if they were reciting from somewhere. This is also a common lie that this article just propagates further. If that's really how they worked, they'd be even less useful than they presently are. This is another thing the article is dismissive of (in my view).

I agree, real intelligence may also potentially be explained as all "math and probability", whether it's neurons or atoms. A key difference between our brains and LLMs is that the underlying math behind LLMs is still substantially more comprehensible to us, for now.

It's common to believe that we have a more mystical quality, a consciousness, due to a soul, or just being vastly more complex, but few can draw a line clearly.

That said, this article certainly gives a more accurate understanding of LLMs compared to thinking of them as if they had human-like intelligence, but I think it goes too far in insinuating that they'll always be limited due to being "just math".

On a side note, this article seems pretty obviously the product of AI generation, even if human edited, and I think it has lots of fluff, contrary to the name.

Oh, so the issue isn't that the article is dismissive of AI as a technology, but that its author doesn't share your radical views in philosophy of mind (eliminative materialism/illusionism).

Okay, I guess. But I wouldn't characterize that as "being dismissive of AI".

Well I did. I do not recognize dismissiveness as something objective or formally defined (it's a natural language property after all), which leaves us with subjectivity or at best shared subjectivity. It's also plain context dependent, at that much I hope we do agree on, so indeed if you do not accept my philosophical view underpinning why I'd find it dismissive, clearly you won't appreciate it as dismissive. It's a lot like with Gödel's "proof of god". Only holds as long as you subscribe to the axioms being built upon.

I'd imagine we do not share the same subjective perspective on this (e.g. I don't think my views are particularly radical), so you wouldn't characterize it that way, whereas I do. Makes sense to me. Didn't intend to mislead you into thinking this wasn't one of these cases, apologies if this is not what you expected. I wrote under the assumption that you did.

I feel a lot of disagreements are just this, it's just that most often people need 30 comments to get here, if they even manage to without getting sidetracked, and without getting too emotionally invested / worked up.

Sometimes it's useful, imo, to pick apart the differences in intellectual temperament, intuition, foundational beliefs, etc., that drive disagreements. But I agree, sometimes it's a lot of clumsy fumbling.

I'll say this against your perspective (or perhaps just use of language), though: it seems to leave little room for skepticism of the greatest general (not product-specific) claims made today in the AI industry. You either buy into the notion that the path we're on is contiguous with "AGI", or you're dismissive of AI! This is nearly as deflationary as your view of consciousness. ;)

I would expect "dismissive" to describe more categorically dismissive views, and not to extend, e.g., to views which admit that AI is in principle possible, non-eliminativist materialism (e.g., functionalists who say we just don't have good reason to say LLMs or other neural networks have the requisite structure for consciousness), etc.

Since you brought him up, Gödel himself seemed to have a much more "miraculous" notion of human cognition that came out in (IIRC) letters in which he explains why he doesn't think human mathematicians' work is hindered by his second incompleteness theorem. That, I would say is dismissive of AI.

But if any view not grounded in illusionism is dismissive of AI, what can a non-dismissive person possibly identify as AI hype? Just particular marketing claims about the concrete capabilities of particular models? If that's true, than rather than characterizing extreme or marginal views, a view is "dismissive" just for refusing to buy into the highest degree of contemporary AI hype.

> I'll say this against your perspective (or perhaps just use of language), though: it seems to leave little room for skepticism of the greatest general (not product-specific) claims made today in the AI industry.

Maybe I can alleviate this to an extent by expanding on my views, since I believe that's not the case.

I tried alluding to this by saying that, in my view, models have an understanding [of things], but to put it in more explicit terms, for me "understanding" on its own is a fairly weak term. Like I personally consider the semantic diffing tool I use to diff YAMLs to have an understanding of YAML. Not in a metaphorical sense, but in a literal sense. The understanding is hardwired, sure, but to me that makes no difference. It may also not even be completely YAML standard-compliant, which is what would be the "equivalent" of an AI model understanding something to an extent but not fully or not right.

This leaves a lot of room for criticism and skepticism, as that means models can have elementary understandings of things, that while are understandings, are nevertheless not e.g. meaningfully useful in practice, or fail to live up to the claims and hype vendors spout. Which is sometimes exactly how I view a lot of the models available today. They are not capable of fully understanding what I write, and to the extent they are, they do not necessarily understand it the way I'd expect them to (i.e. as a human). But instead of me classifying this as them not understanding, I still decidedly consider these tools to be just often on the immature, beginning side of a longer spectrum that to me is understanding. I hope this makes sense, even if you still do not find this view agreeable or relatable.

You may argue that my definition is too wide, and that then everything can "understand", but that's not necessarily how I think of this either. A "more rigorous" way of putting my thoughts would be that I think things can understand to the extent they can hold representations of some specific thing and manipulate them while keeping to that representation (pretty much what happens when you traverse a model's latent space in a specific axis). But I'm not sure I spent enough time thinking about this thoroughly to be able to confidently tell you that this is a complete and consistent description, fully reflective of my understanding of understanding (pun intended).

Like when in an image model you can quite literally manipulate the gender, hairstyle, clothing of characters depicted by moving along specific directions, to me that is a clear evidence of that model having an understanding of these concepts, and in the literal sense.

Last time I checked, modern LLMs could give you a different answer for the same sequence of prompts each time. Did that change?

They can, but won't necessarily. If you use a managed service, they likely will, due to batched inference. Otherwise, it's simply a matter of configuring the seed to a fixed value and the temperature to 0.

At least that's what I did, and then as long as the prompts were exactly the same, the responses remained exactly the same too. Tested with a quantized gemma3 using ollama, I'd say that's modern enough (barely a month or so old). Maybe lowering the temp is not even necessary as long as you keep the seed stickied, didn't test that.

Just a note for everyone, people do these fixed prompts with fixed seed and 0-temp tests to track model versions. A cool application of the technique

Small update: now that I brought it up, I figured I should just test it. And indeed, as long as the seed is pinned, the replies are bit-for-bit identical run to run (note that the conversation context was cleared before each attempt, of course).

So even setting the temp to 0 is not actually needed. This is handy in case somebody makes the claim that the randomness (nonzero temp parameter) makes a model perform better.

The devil really is in the "insignificant for humans but significant for the model" details, basically. Not in the computational determinism.

A random number generator also gives deterministic results when you fix the seed. Does that make it not a random number generator?

Yes, it does. They're called pseudo-random number generators (PRNGs) in literature for that very reason instead.

Reply on news.ycombinator.com