LeroyRaz

This reads like something AI generated...

Examples: tptacek gets an 'A' for his comment on DF which the LLM claiming that the user "captured DF's unforgiving nature, where 'can't do x or it crashes is just another feature to learn' which remained true until it was fixed on ..."

Link to LLM review: https://karpathy.ai/hncapsule/2015-12-02/index.html#article-....

So the LLM is praising a comment as describing DF as unforgiving (a characterization of the present then, not a statement about the future). And worse, it seems like tptacek may in fact be implying the opposite of the future (e.g., x will continue to crash when it was eventually fixed.)

Here is the original comment: " tptacek on Dec 2, 2015 | root | parent | next [–]

If you're not the kind of person who can take flaws like crashes or game-stopping frame-rate issues and work them into your gameplay, DF is not the game for you. It isn't a friendly game. It can take hours just to figure out how to do core game tasks. "Don't do this thing that crashes the game" is just another task to learn."

Note: I am paraphrasing the LLM review, as the website is also poorly designed, with one unable to select the text of the LLM review!

N.b., this choice of comment review is not overly cherry picked. I just scanned the "best commentators" and tptacek was number two, with this particular egregiously unrelated-to-prediction LLM summary given as justifying his #2 rating.

I am surprised the author thought the project passed quality control. The LLM reviews seem mostly false.

Looking at the comment reviews on the actual website, the LLM seems to have mostly judged whether it agreed with the takes, not whether they came true, and it seems to have an incredibly poor grasp of it's actual task of accessing whether the comments were predictive or not.

The LLM's comment reviews are of often statements like "correctly characterized [program language] as [opinion]."

This dynamic means the website mostly grades people on having the most confirmist take (the take most likely to dominate the training data, and be selected for in the LLM RL tuning process of pleasing the average user).

Hacker News readers, and especially commenters, are number 1!

Have you read the literature? Do you have a background in machine learning or statistics?

Yes. We know that LLMs can be trained by predicting the next token. This is a fact. You can look up the research papers, and open source training code.

I can't work it out, are you advocating a conspiracy theory that these models are trained with some elusive secret and that the researchers are lying to you?

Being trained by predicting one token at a time is also not a criticism??! It is just a factually correct description...

I mean, yeah, statistics works. It's not that surprising that super amazing statistical modelling can approximate a distribution. Of course, thoughts, words, arguments are distributions, and with a powerful enough model you can simulate them.

None of this is surprising? Like, I think you just lack a good statistical intuition. The amazing thing is that we have these extremely capable models, and methods to learn them. That process is an active area of research (as is much of statistics), but it is just all statistics...

I'm not sure what you mean?

Saying we understand the training process of LLMs does not mean that LLMs are not super impressive. They are shining testiments to the power of statistical modelling / machine learning. Arbitrarily reclassifying them as something else is not useful. It is simply untrue.

There is nothing wrong with being impressed by statistics... You seem to be saying that statistics is interesting and there for to say that LLMs are statistics dismissed them. I think perhaps you are just implicitly biased against statistics! :p

I think the person you are responding to is using a strange definition of "know."

I think they mean "do we understand how they process information to produce their outputs" (i.e., do we have an analytical description of the function they are trying to approximate).

You and I mean, we understand the training process that produces their behaviour (and this training process is mainly standard statistical modelling / ML).

In short, both sides are talking past each other.

What do you mean? what do you think statistical modelling is?

I am very confused by your stance.

The aim of the function approximation is to maximize the likelihood of the observed data (this is standard statistical modelling), using machine learning (e.g., stochastic gradient decent) on a class of universal function approximators is a standard approach to fitting such a model.

What do you think statistical modelling involves?

How is that a misconception? LLMs are just advanced statistical modelling (unsupervised machine learning) with small tweaks (e.g., some fine-tuning for human preference).

At the core, they are just statistical modelling. The fact that statistical modelling can produce coherent thoughts is impressive (and basically vindicates materialism) but that doesn't change the fact it is all based on statistical modelling. ...? What is your view?

You realize that they will gladly hallucinate science...

You should check the papers it claims to reference as see if the claims it makes are actually backed up.

In my experience, it can completely mischaracterize scientific literature. For example, I asked it if a codebase was a faithful implementation of an algorithm described in a CS paper, and is said "no" and then proceeded to list a dozen small changes. Every single change was incorrect. The codebase was in fact a completely faithful implementation.

I don't think things get better over time. What is your source for that? Here's an article (with sources) describing a massive down trend in literacy and reading comprehension: https://jmarriott.substack.com/p/the-dawn-of-the-post-litera...

In short, college students nowdays have lower reading comprehension than young children in the 1850s. That is not what I would call progress.

Speaking personally, I believe I would potentially have significantly worse critical reasoning abilities if I had grown up using LLMs. It is very clear to me the temptation of using them as an ersatz for engagement and thought.

I think you are perhaps conflating technological progress (yes technology has improved) with demographic progress. Demographic progress is far from monotonically increasing (reading comprehension is newly plummeting, maths scores are dropping in America, science per scientist is stalling compared to 50 years ago, etc...)

Be careful. The models easily hallucinate problems and misdiagnose. For example, I had an issue with some GPU code, and it assured me, with utter conviction that my problem was caused by some subtle race condition ('a known issue') that the model described in great when the real issue was just a trivial typo - no race condition, no subtly or complexity.

You make a completely false equvilance.

A car person would be some kind of car person hybrid if you read it literally. Car person is acting as a short hand for "Car obsessed person." Car is a noun, blind is an adjective, etc...

There's no such thing as capitalism without government. Depending on how the government regulates capitalism, you don't necessarily get wealth inequality.

To take the example of trans: yes. Multiple different groups have genuine concerns about trans issues. For example, you have trans people wanting to be respected and not victimized, you have women who fear men being in their spaces, you have parents who fear their children being harmed and encouraged to transition, you have female athletes concerned about unfair competition from people who went through male puberty, etc... None of these groups innately hate each other or are bigoted, they just have different opinions on what the best society should be like as shaped by their own experiences.

And all of these groups have a point. Obviously you do get examples of trans people being victimized, you do get examples of children regretting transitioning, you do get examples of unfair athletic competition, etc... It is a complete conspiracy theory to label all of the groups as made of hateful fake news spreading bigots. They are all just human beings, muddling through the world!

Chat Control seems unabashedly evil.

This particular type of authoritarianism is primarily from the left in the west. The EU, the UK, Canada, it is the political left implementing these policies, often using them to censor right wing views (e.g., Objecting to immigration gets labelled racist -> justified censorship. Objecting to trans women in women only spaces gets labelled as hateful -> justified censorship, etc...).

Intolerance is the wrong framing. You should be understanding. People who throw the label "bigot" around are themselves bigots (convinced of the superiority of their own beliefs without engaging with others views). A lot of people can have genuine disagreements with certain ideologies (e.g., they might think that transitioning minors is net negative for society) but also be open to dialogue. Such people are not bigots. Those who disagree with the left on any given point may or may not be wrong, but disagreement is not bigotry. There is no paradox there.

There is also a difference between speach and action. As a society, we should allow all speach (e.g., people questioning authority), but supress certain actions (e.g., violence). Currently, the American left seems to believe that people voicing the wrong views justified violence. That belief is abhorrent and fundamentally completely at odds with liberalism and a just and well functioning society.

Specifically, re-tolerance of intolerance, I highly recommend a speach by Rowan Atkinson on exactly that topic. If you Google it you can probably find it. It is worth a watch. He is an incredibly intelligent and eloquent man.

I don't understand how the removing of right wing users is described as a necessary good to create a safe and inclusive space, while the removal of left wing users is decried as censorship. If you dismiss people who have genuine concerns about things as bigoted, you yourself are being bigoted! It is as though people don't understand the definition of the word.

Maybe I haven't worked on tricky enough problems, but I swear people massively hype-up the difficulty of coding. The vast majority of the time there is little difficult problem solving

It confused me sufficiently that I consulted the original paper to check what type of AI algorithm the research used! That was my immediate reaction to reading the sentence.

Out of curiosity, are you familiar with work like "the AI Scientist"? Having an LLM based AI suggest experiments based on parsing scientific literature is not outlandish.

There are two ways to interpret the sentence we are discussing:

A: a grammatically incorrect statement, saying that "the AI used theory", when they mean that "the AI's design can be understood using theory" (or more sloppy "that the design uses the theory").

B: a grammatically valid if contentious-to-you statement about an LLM or knowledge graph based system (e.g., something like the AI Scientist paper) parsing theory and that parsing being used to create the experiment design.

As I have explained, B is a perfectly valid interpretation, given the current state of the art. It is also valid historically, as knowledge graph based systems have been around for a long time. It is also the likely interpretation of a lay person, who is mainly exposed to hype and AI systems like chatGPT.

Regardless, they a) introduce needless ambiguity that is likely to mislead a large proportion of readers. And b) if they are not actively misleading then they have written something grammatically incorrect.

Both findings mean that the article is a sloppy and bad piece of writing.

This particular sentence is also only a particular example of how the article is likely to mislead.

What makes it clear to you that they don't mean what they explicitly write? What are you defending and why?

Philosophical discussions aside, it is entirely possible for current AI to use concepts (but the research they are describing does not employ that kind of AI).

I also think most lay people seeing the term AI are likely to think of something like ChatGPT.

It is a) literally incorrect what they write, and b) highly misleading to a lay person (who will likely think of something like ChatGPT when they read the term AI). Why are you defending their poor writing?

They write the "AI was probably using some esoteric theoretical principles." That is a direct quote of the article.

If it was an LLM based model this could be a correct statement, and it would suggest a groundbreaking achievement: the AI collated esoteric research, interpreting it correctly and used that conceptual understanding to suggest a novel experiment. This might sound far fetched, but we already have LLM based systems doing similar... Their written statement is plausible given the current state of hype (and also a plausible, though ground breaking, given the current state of research).

In reality, the statement is incorrect. The models did not 'use' any concepts (and the only way to know that the article is wrong is to actually bother to consult the original paper, which I did).

The distinction matters: they implied something ground breaking, when the reality is cool, but by no means unprecedented.

Tldr: using concepts is not something classic ML algorithms do. They thus directly erroneously imply (a groundbreaking) foundation model based (or similar) approach. I care because I don't like people being mislead.

The standard term is Machine Learning (ML). It's not artificial intelligence (AI) because there is no sense of attempting to manipulate concepts (e.g., the classic symbolic conceptual manipulation algorithms, or modern foundation models).

The article is misleading and badly written. None of the mentioned works seem to have used language or knowledge based models.

It looks like all the results were driven by optimization algorithms, and yet the writing describes AI 'using' concepts and "tricks". This type of language is entirely inappropriate and misleading when describing these more classical (if advanced) optimization algorithms.

Looking at the paper in the first example, they used an advanced gradient descent based optimization algorithm, yet the article describes "that the AI was probably using some esoteric theoretical principles that Russian physicists had identified decades ago to reduce quantum mechanical noise."

Ridiculous, and highly misleading. There is no conceptual manipulation or intuition being used by the AI algorithm! It's an optimization algorithm searching a human coded space using a human coded simulator.

Isn't the writer themselves using the "insipid bullet points with bold style"?

TikTok is a legitimate national security concern.

Money Extraction: fighting over a slice of pie.

Wealth Creation: creating new pie.

In practice, a lot of things that look extractive (e.g., designing better high frequency trading algorithms) potentially have some marginal utility (e.g., creating market liquidity), but the money high performing people make is likely larger than the utility they add to the system (because most of the money in having the best high frequency trading algorithm comes from beating other people's high frequency trading algorithms).