ainch

On that note, Terence Tao gave a good interview to Dwarkesh Patel talking about Kepler. He pointed out that the previous geocentric models were actually more accurate than Kepler's at the time, in part because they'd had so much complexity piled on to solve minor errors. Kepler's theory was more elegant, but at the time it wasn't necessarily a better model.

I think important paradigm shifts can often look like this - there's not necessarily a reason to expect them to be instantly optimal. Deep Learning vs 'good old-fashioned AI' is another example of this dichotomy; it took a long time for deep learning to establish itself.

I think it depends whether you can leverage some knowledge. It's possible for a person/LLM to look at a loss curve and say "oh that's undertraining, let's bump the lr" - whereas a Bayesian method doesn't necessarily have deeper understanding, so it'll waste a lot of time exploring the search space on poor options.

If you're resource unconstrained then BO should ofc do very well though.

I'd like see a system like this take more inspiration from the ES literature, similar to AlphaEvolve. Let's see an archive of solutions, novelty scoring and some crossover rather than purely mutating the same file in a linear fashion.

Microsoft released a report with some numbers on Deepseek adoption globally. They say it's got ~90% market share in China, and is growing in popularity across Africa.

https://www.microsoft.com/en-us/corporate-responsibility/top...

I was a little surprised to see a Telegram integration rather than Slack or Teams, given Anthropic's enterprise-first posture. But then I looked it up, and it turns out Telegram dwarfs both, at around 1bn MAUs, vs 50m and 300m respectively! I had no idea - reminds me of the time I found out Snapchat has 2x the userbase of Twitter.

The human genome contains around 1.5GB of information and DeepSeek v3 weighs in at around 800GB, so it's a bit apples-to-oranges. As you say, what's been evolved over hundreds of millions of years is the learning apparatus and architecture, but we largely learn online from there (with some built-in behaviours like reflexes). It's a testament to the robustness of our brains that the overwhelming majority of humans learn pretty effectively. I suspect LLM training runs are substantially more volatile (as well as suffering from the obvious data efficiency issues).

If you'd like an unsolicited recommendation, 'A Brief History of Intelligence' by Max Bennett is a good, accessible book on this topic. It explicitly draws parallels between the brain's evolution and modern AI.

What competition was OpenAI likely to face from a team working on fast Python tooling?

I got Codex to whip me up a Chrome extension that autoswaps back to Pro whenever I reload the page. It's made Gemini significantly less irritating to use.

I don't think it's just an engineering problem - decades of research have failed to produce a convincing, general definition of intelligence, capability or agency. You can try to form proxy metrics by combining benchmarks, but existing benchmarks are flawed, and should be taken with a pinch of salt.

It's evident in the fact that every time AI has historically met certain thresholds (chess-playing, the Turing Test, fluent language), we play with them a little more and find out there's something still lacking.

I use Gemini via its web app, which aggressively autoswitches to the Flash over Pro, but I usually notice quickly because the answers are weird or the logic doesn't quite follow. I feel like, at least for 'daily driver' usage, small models are still a little disappointing. That said, they're getting very good for more automation-y tasks with simple, well-constrained tasks.

That's likely because they're chasing enterprise - see deals with HSBC, ASML, AXA, BNP Paribas etc... Given swelling anti-US sentiment and their status as a French 'national champion', Mistral are probably in a strong position for now regardless of model performance, research quality or consumer uptake.

pass@k means that you run the model k times and give it a pass if any of the answers is correct. I guess Lean is one of the few use cases where pass@k actually makes sense, since you can automatically validate correctness.

The only other pair I've seen treated that way is Nothing's Headphones - although the (maybe niche) musicians I've seen wearing them lean into y2k aesthetics, where Apple's products are more broadly appealing.

Jaxtyping is the best option currently - despite the name it also works for Torch and other libs. That said, I think it still leaves a lot to be desired. It's runtime-only, so unless you wire it into a typechecker it's only a hint. And, for me, the hints aren't parsed by Intellisense, so you don't see shape hints when calling a function - only when directly reading the function definition.

Personally, I also think the syntax is a little verbose: for a generic shape hint you need something like `Shaped[Array, "m n"]`. But 95% of the time I only really care about the shape "m n". It doesn't sound like much, but I recently tried hinting a codebase with jaxtyping and gave up because it was adding so much visual clutter, without clear benefits.

This would be an insta-switch feature for me! Jaxtyping is a great idea, but the runtime-only aspect kills it for me - I just resort to shape assertions + comments, but it's a pretty poor solution.

A follow-up question: Google's old `tensor_annotations` library (RIP) could statically analyse operations - eg. `reduce_sum(Tensor[Time, Batch], axis=0) -> Tensor[Batch]`. I guess that wouldn't come with static analysis for jaxtyping?

I wonder how much this is just a sampling bias. Older media has been repeatedly filtered over time, so you don't see all the bland, derivative ripoffs that were abundant at the time. Likewise, interesting and forward-thinking work produced today may not be widely appreciated for many years - consider that Van Gogh's work was largely ignored during his lifetime.

It's unintuitive to me that architecture doesn't matter - deep learning models, for all their impressive capabilities, are still deficient compared to human learners as far as generalisation, online learning, representational simplicity and data efficiency are concerned.

Just because RNNs and Transformers both work with enormous datasets doesn't mean that architecture/algorithm is irrelevant, it just suggests that they share underlying primitives. But those primitives may not be the right ones for 'AGI'.

LeCun was stubbornly 'wrong and boneheaded' in the 80s, but turned out to be right. His contention now is that LLMs don't truly understand the physical world - I don't think we know enough yet to say whether he is wrong.

The general concern around Taylor Swift's emissions has always struck me as shortsighted. Her Eras tour is estimated to have generated around $5bn in economic uplift in the US, at an estimated 10,000 tonnes CO2e for her personal travel. Even if the total footprint is higher, that is thousands of times lower than the emissions intensity of an industry like fast fashion. From an environmental point of view, attending a Taylor Swift show is a much less carbon-intensive way to spend your money than ordering from Temu.

One way of thinking about diffusion is that you're learning a velocity field from unlikely to likely images in the latent space, and that field changes depending on your conditioning prompt. You start from a known starting point (a noise distribution), and then take small steps following the velocity field, eventually ending up at a stable endpoint (which corresponds to the final image). Because your starting point is a random sample from a noise distribution, if you pick a slightly different starting point (seed), you'll end up at a slightly different endpoint.

You can't jump to the endpoint because you don't know where it is - all you can compute is 'from where I am, which direction should my next step be.' This is also why the results for few-step diffusion are so poor - if you take big jumps over the velocity field you're only going in approximately the right direction, so you won't end up at a properly stable point which corresponds to a "likely" image.

Even then, if you're familiar with NumPy it's pretty easy to switch to Jax's NumPy API, and then you can easily jit in Python as well.

The most chilling thing imo is that Anthropic is the only lab that have said anything about this. Google and OpenAI presumably signed up to all these terms without any protest.

I've heard of journalists using it to try and figure out whether images sent by sources were generated. In their Nano Banana 2 release blogpost, Google mentioned that SynthID has been used ~20 million times, so there's clearly some interest in identifying AI-generated images.

Sorry but this is famously not true! There is no guarantee that statistical models generalise. In your example, whether or not your model generalises depends entirely on what f(x) you use - depending on the complexity of your function class f(x+2) could be 7, 8, or -500.

One of the surprises of deep learning is that it can, sometimes, defy prior statistical learning theory to generalise, but this is still poorly understood. Concepts like grokking, double descent, and the implicit bias of gradient descent are driving a lot of new research into the underlying dynamics of deep learning. But I'd say it is pretty ahistoric to claim that this is obvious or trivial - decades of work studied "overfitting" and related problems where statistical models fail to generalise or even interpolate within the support of their training data.

This understates the possible headroom as technical challenges are addressed - text diffusion is significantly less developed than autoregression with transformers, and Inception are breaking new ground.

I don't think it's a good comparison given Inception work on software and Cerebras/Groq work on hardware. If Inception demonstrate that diffusion LLMs work well at scale (at a reasonable price) then we can probably expect all the other frontier labs to copy them quickly, similarly to OpenAI's reasoning models.

Amodei says yes - each model pays for its training. But they're scaling up investment for each new run, so they're still happily in the red.

And also that may be the case for Anthropic who have fewer free users, a large enterprise business, and less generous rate limits on their subscriptions. I don't know if OpenAI or Google have commented. I suspect OpenAI is in a worse position given their massive non-paying consumer base.

Depends how precisely you define novel - I don't think LLMs are yet capable of posing and solving interesting problems, but they have been used to address known problems, and in doing so have contributed novel work. Examples include Erdos Problem #728[0] (Terence Tao said it was solved "more or less autonomously" by an LLM), IMO problems (Deepmind, OpenAI and Huang 2025), GPT-5.2 Pro contributing a conjecture in particle physics[1], systems like AlphaEvolve leveraging LLMs + evolutionary algorithms to generate new, faster algorithms for certain problems[2].

[0] https://mathstodon.xyz/@tao/115855840223258103

[1] https://huggingface.co/blog/dlouapre/gpt-single-minus-gluons

[2] https://deepmind.google/blog/alphaevolve-a-gemini-powered-co...

It's definitely >0 https://www.independent.co.uk/news/world/americas/us-politic...

Mandarin is weird, because I don't think it's that hard to speak at a passable level, mostly because the grammar is so simple. Many people are spooked by tones, but I think their importance for simple communication can be a little overstated.

But then, learning to read and write requires enormous additional effort. When I learned in Beijing, I'd spend a couple hours a day working on grammar/speaking/listening - and then like 6 hours a day of rote practice to get familiar with characters.