gallerdude

We used to get one annual release which was 2x as good, now we get quarterly releases which are 25% better. So annually, we’re now at 2.4x better.

The weirdest thing about this AI revolution is how smooth and continuous it is. If you look closely at differences between 4.6 and 4.5, it’s hard to see the subtle details.

A year ago today, Sonnet 3.5 (new), was the newest model. A week later, Sonnet 3.7 would be released.

Even 3.7 feels like ancient history! But in the gradient of 3.5 to 3.5 (new) to 3.7 to 4 to 4.1 to 4.5, I can’t think of one moment where I saw everything change. Even with all the noise in the headlines, it’s still been a silent revolution.

Am I just a believer in an emperor with no clothes? Or, somehow, against all probability and plausibility, are we all still early?

I always grew up hearing “competition is good for the consumer.” But I never really internalized how good fierce battles for market share are. The amount of competition in a space is directly proportional to how good the results are for consumers.

Both Opus 4.6 and GPT-5.3 one shot a Gameboy emulator for me. Guess I need a better benchmark.

Both Opus 4.6 and GPT-5.3 one shot a Gameboy emulator for me. Guess I need a better benchmark.

I'm not sure I agree. LLM's have the feel of an alien new technology, and especially did back then. In retrospect, it feels very obvious that small models don't pose much of a threat, but that's only in retrospect.

> the rich are now permitted to ignore copyrights, while the poor remain constrained by them, as before.

Claude Code is $20 a month, and I get a lot of usage out of it. I don't see how cutting edge AI tools are only for the rich. The name OpenAI is often mocked, but they did succeed at bringing the cutting edge of AI to everyone, time and time again.

I'm sure Apple is more than happy to pay the premium for cleanness.

Today I showed Claude Code how to control my lights, and I'm having a blast.

It is interesting that most of our modes of interaction with AI is still just textboxes. The only big UX change in that the last three years has been the introduction of the Claude Code / OpenAI Codex tools. They feel amazing to use, like you're working with another independent mind.

I am curious what the user interfaces of AI in the future will be, I think whoever can crack that will create immense value.

I wonder if they're using reasoning? It usually eliminates these types of errors

> OpenAI researcher Noam Brown on hallucination with the new IMO reasoning model:

> Mathematicians used to comb through model solutions because earlier systems would quietly flip an inequality or tuck in a wrong step, creating hallucinated answers.

> Brown says the updated IMO reasoning model now tends to say “I’m not sure” whenever it lacks a valid proof, which sharply cuts down on those hidden errors.

> TLDR, the model shows a clear shift away from hallucinations and toward reliable, self‑aware reasoning.

Source: https://x.com/chatgpt21/status/1950606890758476264

Local models you can get just the pretrained versions of, no RLHF. IIRC both Llama and Gemma make them available.

I can see both sides of it. There’s a fancy bread bakery by where I live. I go infrequently, the bread is great. But it’s expensive, most of the I just want a cheap loaf from Target, as do most people.

Instead of broad employment of artisan breadsmiths, we have people doing email work, because it’s more economically valuable. If the government mandated a higher quality of bread, we’d be slightly richer and bread and slightly poorer in everything else.

I made a very mediocre platformer in my senior year of high school, published on itch.io. I ended up becoming a software developer, which I enjoy 80% as much, but without any burnout or worrying about the superstar economics of being a game dev. Once the singularity hits, maybe I'll make more games.

https://gallerdude.itch.io/the-journey-east-full

When I was near the end of high school, my family visited London, and I was thinking about being a game dev. So I sent Terry Cavanagh an email, and to my surprise he completely agreed to get lunch.

He was extremely kind, gave me a lot of interesting life advice. I remember him saying that he got most of his ideas just from playing around with mechanics and experimenting a lot, he was never really one to get grand visions.

Anyways, great fellow, glad he opened source V (as he called it).

classic human hallucination

He's awesome.

I listened to Lex Friedman for a long time, and there was a lot of critiques of him (Lex) as an interviewer, but since the guests were amazing, I never really cared.

But after listening to Dwarkesh, my eyes are opened (or maybe my soul). It doesn't matter I've heard of not-many of his guests, because he knows exactly the right questions to ask. He seems to have genuine curiosity for what the guest is saying, and will push back if something doesn't make sense to him. Very much recommend.

For coding, I like the Aider polyglot benchmark, since it covers multiple programming languages.

Gemini 2.5 Pro got 72.9%

o3 high gets 81.3%, o4-mini high gets 68.9%

It's funny, I see myself as basically just a pretty unabashed AI believer, but when I look at your predictions, I don't really have any core disagreements.

I know you as like the #1 AI skeptic (no offense), but like when I see points like "16. Less than 10% of the work force will be replaced by AI. Probably less than 5%.", that's something that seems OPTIMISTIC about AI capabilities to me. 5% of all jobs being automated would be HUGE, and it's something that we're up in the air about.

Same with "AI “Agents” will be endlessly hyped throughout 2025 but far from reliable, except possibly in very narrow use cases." - even the very existence of agents who are reliable in very narrow use cases is crazy impressive! When I was in college 5 years ago for Computer Science, this would sound like something that would take a decade of work for one giant tech conglomerate for ONE agentic task. Now its like a year off for one less giant tech conglomerate, for many possible agentic tasks.

So I guess it's just a matter of perspective of how impressive you see or don't see these advances.

I will say, I do disagree with your comment sentiment right here where you say "Ask yourself how much has really changed in the intervening year?".

I think the o1 paradigm has been crazy impressive. There was much debate over whether scaling up models would be enough. But now we have an entirely new system which has unlocked crazy reasoning capabilities.

I had a very hard time working there, maybe the worst time in my life. I worked with a lot of very smart people, but something about the company culture is doomed in a way I haven't seen before.

Last year I read the book Julia by Sandra Newman, which shows the story of 1984 from Winston's lover's perspective. Spoiler, at the very end of the book, Julia escapes Airstrip One, and we find out that Big Brother has just been captured by the good guys, and he is now a decrepit old man with no understanding of the world.

This implies that all the suffering, hardship, and pain experienced in the dystopian classic happens for no reason at all. Airstrip One is just a machine that gnashes and grinds each individual person within it and outputs... nothing.

This is the closest any book has gotten to describing my Amazon experience. I read headlines like this and wonder how long the machine continue to run for.

Which ones? In my experience, a lot of Apples products have incredible longevity. Notes, Calendar, Pages all just get better and better.

Is there really no standalone app, like ChatGPT/Claude/DeepSeek, available yet for Gemini?

Completely disagree… there are a crazy amount of cases that didn’t work, until the models scaled to a point they magically did.

Best example I can think of is the ARC AGI benchmark. It was seen to measure human-like intelligence through special symmetries and abstract patterns.

From GPT-2 to GPT-4 there was basically had no progress, then o1 got about 20%. Now o3 has basically solved the benchmark.

Humans always hallucinate like this, seeing the original problem instead of the twist.

With some transcribing (using another LLM instance) I’ve even gotten it to solve NYT mini crosswords.

4o-mini: 16% 4o: 50% o1-mini: 97% o1: 100%

* disclaimer - only n=7 on o1. Others are like 100-300 each

Very interesting - have you tried using `o1` yet? I made a program which makes LLM's complete WORDLE puzzles, and the difference between `4o` and `o1` is absolutely astonishing.

New technologies can solve problems better.

"I'm starting to think computers are a solution in the need of a problem. Have we not already solved doing math?"

Sometimes I wonder if in 100 years, it's going to be surprising to people that computers had a use before AI...