naasking

That's just vibes then. How is that supposed to be convincing?

> but there are many things which they are not good at which it is not cost effective or meaningful to improve

Can you name a few such things so I can keep an eye on them in the coming years?

> We know exactly how attention layers work and how they produce the next word as well as draw them from larger feature spaces.

This is not what's meant by the statements that we don't know how LLMs work. Explain why LLMs are so good at programming, finding bugs, and developing mathematical proofs. Like, way better than all prior tools specifically designed to be bug finding tools, despite being merely "language models".

The search space is far too large for a mere order of magnitude to make any difference at all.

> we know everything about how LLMs work

No we don't. That we understand the low level mechanics of a system doesn't mean we understand how any high level phenomena emerge from those low level mechanics.

This is as true for quantum mechanics as it is for LLMs.

I don't think that's correct. Increasing parameter count increases capabilities in all domains per the scaling laws. Models are larger than they were years ago, so capabilities in all domains must necessarily be better. This doesn't even account for better training data, which has also much improved.

Bribes are not speech though, and advertising is.

It's interesting to claim that it would be difficult to explain why we punish successful crime more. A successful murder creates more suffering (victim's friends and family), and ostensibly the loss of a productive member of society. These all seem like fairly common justifications for punishing murderers under the banner of retributive justice.

It does matter though. If you want to murder someone by hitting them with a plushie, you're not going to get charged with attempted murder because it's not possible that that would ever work. There must be justification that the choice will have the intended effect.

We should not be gung-ho to give the government more power to regulate speech.

The margins for groceries are objectively thin. The only way to provide food at lower prices is to provide a worse good or service, eg. less variety, less quality, less availability, etc. You will see all of these outcomes in NYC, if anyone even accepts the bids to begin with.

Yes, but there is little evidence this had a meaningful effect on votes.

They're important everywhere of course, but especially on mobile. If AI researchers figure out how to offload knowledge and expertise from reasoning weights, then a core reasoning ASIC linked to the knowledge would totally rock.

> I can't see how the model could include the actual subjective human experience.

People who say LLMs have subjective experience aren't saying they have human-type subjective experience. Nobody who sees an LLM express hunger when role playing as a hungry person thinks that the LLM is actually hungry.

I too can role play as a hungry person despite not being hungry, so there is no reason in either case to conclude that the words produced reflect genuine internal subjective states. The point is that such internal states may still exist.

Even mechanistic models generate interesting discussion. How many years have we discussed Turing machines and the lambda calculus? Almost a century of great work came out of those.

The reason I insist on mechanistic models is because the original post was making a definitive knowledge claim, and in my experience, the knowledge claim is unwarranted.

> Dunno about the parent commenter, but I personally interpret the concept as having a hidden representation of self that is continually tended to

I don't see why an LLM could not have a sense of identity or personality while it's evaluating a specific prompt, or even change self awareness while evaluating a prompt since many outputs model a back and forth conversation. My point is that without a mechanistic model of what "self awareness" means, we have no way of truly evaluating such questions, we're just hand waving vague intuitions about what it could mean.

Making definitive claims about whether LLMs do or do not have specific properties absolutely does require precise definitions of those properties that can be used to evaluate those questions. Merely hand waving that LLMs didn't undergo the same evolutionary process is not a definitive argument.

For example, the Turing machines and the lambda calculus don't look anything alike, but they are fundamentally interconvertible, and so in a real sense they are fundamentally equivalent. Without a model, all of your arguments are completely unconvincing for exactly the same reasons, eg. that there may exist many paths to fundamentally equivalent ends.

> The qualia themselves, even those that are quite abstract, are rooted in our physical presence and evolution.

There is no objective evidence of qualia. All evidence of qualia are vocal or other expressions of belief in qualia. Perceptions clearly exist and are observable, subjective experience and qualia, not so much.

> I see no reason to believe that a neural network built entirely based on the symbolic level of language could have the features needed for the subjective experience itself.

If your objection is to models based on "symbolic level of language" which you think lack semantic understanding of, say, trees, you should ask yourself how our brain, based on physics which also lacks any semantic category for trees, can somehow develop a semantic understanding of trees. All of these appeals to differences with the brain never seem to acknowledge that fundamentally, the brain has the same explanatory gap with physics.

> But if we assume awareness because outputs resemble what we consider meaningful as humans, yet the neural network has had no inputs or evolution that could form the actual basis of human-like experience

This assumes a lot. It seems very possible to me that intelligence inherently develops a map of natural categories (natural kinds), and language naturally develops around such categorical understanding. Semantics are then fundamentally the network of associations between categories, eg. there is no fundamental difference between symbols and semantics, and the latter cam be inferred from the former, and that's exactly what LLMs do, and why the semantic maps between different languages are so similar and how they can translate between languages.

> AI has no self-awareness

What is your mechanistic model of self awareness that yields this conclusion?

> It's a tool

Does your model suggest that tools can't have self awareness?

What sentence from my post implies that?

All of them depend on temporarily suspending concurrency in order to synchronize. Even atomic exchange operations are like this at the hardware level.

> 3) and also by how Microsoft operates (e.g. certainly not using any modern C).

Are you suggesting "modern C" is less prone to memory safety issues, and that if MS used "modern C" then that would meaningfully reduce the 70% of security vulnerabilities it claims are caused by memory safety violations?

You wouldn't have to worry so much about running up to date software if memory safety were pervasive.

An opinion is a particular subject's belief [1]. That's literally the definition. "X believes Y at time T", is a fully specified fact and completely follows from the definitions of all terms involved.

[1] a view, judgment, or appraisal formed in the mind about a particular matter, https://www.merriam-webster.com/dictionary/opinion

This seems like a very long winded way to agree with my statement that opinions can only be considered facts when modelled as time series data.

Opinions can only be seen as facts when modelled as time series data. Verifying an opinion X at time T does not entail X at time T+1.

Comments sometimes help the LLM think through a problem. They are particularly useful if you have thinking disabled, eg. it's not uncommon to run Qwen3.6 locally, but its thinking is extremely verbose, so it's fast if you disable thinking, but doing that will sometimes cause it to think a little in comments, and if you disable that too, then it will produce worse output.

Comments can always be stripped reliably later.

> Can't help to think of a recent HN post about most AI-generated projects being abandoned within months. Why?

That which can be created with little effort can be dismissed with little fanfare, since it can easily be recreated later if there's a need.

The other category of AI abandonware is one-off stuff. It did its job and there's no further use for it.

> If you can do a Rust rewrite with AI, I can create one as well.

Translations are a lot more likely to be error-free and robust, as the original data structures and algorithms have been battle tested.

> Trump called for investigation and arrest on Comey, Cheney, Powell, ... despite the fact that they never crossed the line in expressing their criticism to Trump's decisions.

I'm not a fan of Trump's governance, but none of those people were investigated for "criticizing Trump's decisions".

> This has never happened in Europe.

Poland: The "Lex Tusk" Commission (2023)

Hungary: The Sovereignty Protection Office (2023)

Ukraine: Viktor Yanukovych vs. Yulia Tymoshenko (2011)

Turkey: Erdoğan vs. Ekrem İmamoğlu

Romania: Investigating the Chief Anti-Corruption Prosecutor (2018)

France: The "Clearstream Affair" (2004)

> How is that a bad thing? If your opinions are not stupid, you don't need to insult people to express them.

1. Insults are subjective. If you don't see the problem with criminalizing subjective opinions, then I don't know what to tell you.

2. You're literally trying to restrict how I express my legitimate opinions while simultaneously claiming that speech is freer in Europe.

> Me: "Mr Smith was not arrested because he wears a red t-shirt, he was arrested because he acted like an idiot by deciding to expose himself to children in the street". You: "So you are saying that acting like an idiot is a criminal offence".

You've lost the plot. The original point was that people were arrested for criticizing politicians, your rebuttal was that the way they did it was idiotic, which directly implies that you're ok with criminalizing idiocy even when the criticism is legitimate. Your reply here is a complete red herring.

> Academia are highly international and therefore meritocracy based:

This does not follow. Literally.

> This idea that the whole word is so much into the conspiracy that every universities on Earth are covering the left-wing academia conspiracy is so stupid.

It's not a conspiracy when people are just acting in their own best interests. I don't know where you got the idea that it has to be a conspiracy.

> Maybe another reason is that a lot of right-wing political ideas don't make sense when confronted to a rigorous analysis, and therefore the right-wing positions are falling from natural selection.

This is also delusional, and frankly completely ignorant of studies done on exactly this (not only researcher bias, publication bias, hiring bias, and more).

> When, at the same time, "normal" people have absolutely no problem criticizing politicians in a "normal" way.

So you agree that the US has more free speech because people can deviate from your completely arbitrarily defined "normal" form of critique.

As for examples, the German criminal code has literally criminalized insulting people (section 185), and insulting politicians has even harsher penalties (section 188). There are hundreds of articles covering cases, just ask any AI for a summary.

> So they were not arrested for criticizing politicians, they were arrested for acting like idiots.

In your opinion, acting like an idiot is a criminal offense, even if it does not harm anyone, in the tort sense?

> Well, first, it cuts both way: left-leaning scientists have a left-leaning bias, and right-leaning scientists have a right-leaning bias. So it cancels out.

And what are the political demographics of academia right now? This is a big reason for the replication crisis in the social sciences.

> politicians who have no idea of the subject who ruin a scientist career because they saw the term "Enola Gay" or "financial equity"".

That isn't really happening. No one's career is being ruined by having to rewrite grant proposals to remove DEI language, because previous policy required DEI language. The restructuring of academic funding and incentives is frankly long overdue. Everyone, including scientists, has been complaining about grant funding and the skewed incentives in academia, like publish or perish. You often have to break a system before it can be fixed.