in-silico

In what ways to you think the TESCREAL arguments are wrong?

I've noticed the existence of something like "thought promoting cliches" in my life that when said, usually half-jokingly, cause me to think more about something than I otherwise would.

The prime example is "this says a lot about society"

> 48% Of people believed Griffin was a real person after a one- minute video call.

...I don't think this will have a positive impact on the world

Were people getting their pants tied up in knots like this over names like daemon programs (a term that carries "baggage" of its own) and installation wizards (which, to my knowledge, do not actually invoke in the arcane)?

The public figure viewer would be way better to use if I could inspect entries without opening a new page.

I wonder when work started on this project, and how the public release of Jev played into their timing.

Why can't detractors of rationalism and AI-doom ever just lay out an argument for why they're wrong

It's always "yudkowsky is a weird guy" or "they had an orgie once" but who gives a fuck?

I just want someone to lay out, in impersonal terms, the flaws of their reasoning and why I shouldn't agree with them.

My problem is with the whole genre of article rather than this one specifically.

How do you guys realistically plan to survive long-term against the hyper-scalers as a 2 person lab?

Curious because I'm considering joining a similarly small lab.

A lot of current diffusion LLMs don't convert tokens to continous space before noising them. They add discrete "noise", which is often as simple as replacing some tokens with [MASK].

The real problem is that when using fewer sampling steps than output tokens, diffusion formulations fundamentally cannot represent distributions where output tokens are heavily codependent. Autoregressive formulations don't have this problem, they can represent any distribution (ignoring limitations of the underlying model).

Does the free tier really not have access to reasoning models?

That would explain a lot of the terrible AI/LLM takes online.

This algorithm: sample a bunch of latents, train the model using the one with the lowest error.

IWAE: sample a bunch of latents, weight the loss of training the model using each one by softmax(-error). For images and text where the errors have large variance, those weights become one-hot, yielding this algorithm.

The training method is very similar to Importance Weighted Autoencoders: https://arxiv.org/abs/1509.00519

Did the model really need to hack huggingface to get access to ExploitGym data? I'd imagine that once it had full internet access it could have just used the HF API or website (but the heavy prompting/nudging towards hacking made it do things the hard way).

What would evidence of "intelligence" or "creativity" look like for you?

One "solution" to these problems is to have the probes land on planets instead of asteroids, and build the necessary infrastructure there.

The difference is that artistic sensibility is largely subjective. This means that:

1. It's hard to measure (and people can disagree about it)

2. It can't really be improved using RL without a human in the loop (which is how math is being trained)

I feel like the data should have been generated by a much less predictable policy.

It often feels like the model is ignoring my inputs and just doing what it would expect the bot to do (which is unsurprising if the model could predict what would happen next during training without paying attention to the inputs)

This post misses a very important point: humans aren't 100% correct either. This means that the bar for being useful (at tasks that humans usually do) isn't perfection, it's human-level correctness. If we can create AI models that make fewer mistakes than humans (which is almost certainly possible, even if not easily or soon), then we will actually need to spend less time double-checking for correctness than we do now.

> We see it with self-driving cars. It's really impressive what is possible. But they aren't true self-driving. A person needs to be sitting at the wheel, keeping their attention focussed on traffic as if they would be driving themselves, so they can intervene when the AI makes an inevitable mistake.

There are self-driving cars all over San Fransisco transporting people on public roads with no human at the wheel. This proves my point: those cars are not perfect, but they are human-level (or close), and that's all that's needed.

Why can't a next token predictor do math? Humans aren't calculators either, but we can do math.

If you want proof just look at the benchmarks. Modern frontier models can get basically perfect accuracy on American Invitational Mathematics Examination tests: https://matharena.ai/?comp=aime--aime_2026

If you want an explanation of how they do math, we've found geometric calculators inside their neural networks: https://www.goodfire.ai/research/a-geometric-calculator#

Additionally, maybe it's easier for a model to realize that it doesn't know the answer when the question is easier.

If Opus gets all but the hardest questions right, it might have a higher hallucination rate because the questions it gets wrong are the questions where verification or hallucination detection are the most difficult

Neither of these strike me as particularly groundbreaking.

The first idea (as I understand it as retrieving token ids rather than hidden states) is going to really struggle to do useful compositional reasoning and contextual recall.

The second idea has been been done a million times, with Linear Attention being maybe the first modern example. Hyena, state-space models, DeltaNet, and LaCT also lie in different regions of the performance-parallelizability spectrum of fixed-size models.

> LLMs cannot do math

This is plainly not true anymore

These types of ablation studies are always good. However, I'm not sure how generalizable the language model findings here are.

Their 1.2B model was trained on only 10B tokens, which is less than half of the chinchilla compute optimal number. Modern overtrained 1B LLMs are trained on the order of 10T tokens (1000x more).

This is important because, from my own experience, simplifications and alternatives to standard attention can look fine in the under-trained regime but lag after over-training. This happens because attention has very little out-of-the-gate inductive bias, so it takes a lot of training for the expressiveness to really shine through.

I can't fault the authors since longer training runs cost money, but it warrants pointing out.

I'm also disappointed that they didn't report reasoning benchmark results for the Q=K-V case, since that is by far the most theoretically interesting case (in my eyes).

> The problem is "public schools". The idea itself is wrong, and it can't be made to work.

Do you have an alternative idea in mind?

Care to elaborate?

This is really semantics, but I wouldn't call attending to the KV cache re-reading the context.

The model takes in the context, encodes it into a "memory" (the KV cache), and accesses that memory later. That fact doesn't change just because the KV cache grows in size with the context.

I don't know what memory would look like other than an encode-retrieve loop.

Relevant: Transformers are Multi-State RNNs - https://arxiv.org/abs/2401.06104

> It's not changed by the experience

The entire file is not changed, but the KV cache is.

> It doesn't remember anything

The model definitely remembers previous exchanges within the same conversation.

> Would be interesting whether it is possible to write a LLM-like program just using compression and function interpolation algoritms.

gzip can be used as a (not very good) LLM-like text and image generator: https://arxiv.org/abs/2309.10668

We know how the models are built and trained, but we have a very limited understanding of how the final products work.

That is to say, we don't know why they give the outputs that they do.

If we did know how they worked, AI interpretability would not be an open and growing field.

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers

A bold claim given that the current top post on HN is "An OpenAI model has disproved a central conjecture in discrete geometry": https://news.ycombinator.com/item?id=48212493