futureshock

A really great way to get an idea of the relative cost and performance of these models at their various thinking budgets is to look at the ARC-AGI-2 leaderboard. Opus 4.5 stacks up very well here when you compare to Gemini 3’s score and cost. Gemini 3 Deep Think is still the current leaders but at more than 30x the cost.

The cost curve of achieving these scores is coming down rapidly. In Dec 2024 when OpenAI announced beating human performance on ARC-AGI-1, they spent more than $3k per task. You can get the same performance for pennies to dollars, approximately an 80x reduction in 11 months.

https://arcprize.org/leaderboard

https://arcprize.org/blog/oai-o3-pub-breakthrough

I really love this piece! I relate to it but it also doesn’t describe me. I’m far more intuitive than this person, though still agree that insights have driven a leveling up of how I relate to others. They were different insights, sure but the model holds.

Once my spouse and I worked for the same company and attended many of the same meetings. The opportunity to pick apart our impressions of the subtext really helped me to learn that I should listen to my gut, that everything I needed to know about how other people were feeling was already in my head and i just needed to stop doubting.

Another time I watched a rather ugly and old person have amazing romantic success with a young beautiful person. How could it be? And I realized that authentic confidence is social gold. I had to let go of my insecurities because my flaws were irrelevant in the face of authentic, confident self acceptance.

I think everyone has a different journey and different epiphanies and it is so enjoyable to hear these experiences put into words.

I’ll borrow ideas from investing: financial independence, diversification and optionality. If you have enough money you can free yourself from the labor market, but you are still deeply tied to your home country. A second citizenship gives you geopolitical independence. And just like diverse investments protect you from the failure of a specific asset, diverse countries can protect you from, for example, a collapse in heath care, a housing crisis or a currency crisis. And most importantly, its like an options contract on life. You have the option, not the commitment to take a high value move to a new country. If the fortunes of your current country sink and your second country rise, you can exercise your option.

There’s a reason people are willing to spend so much on golden visas with the pathway to citizenship.

This so awesome. It reminds me mightily of beat poets like Allen Ginsburg. It’s so totally spooky and it does feel like it has the trapped spark. And it seems to hate us “real ones,” we slickborns.

It feels like you could create a cool workflow from low temperature creative association models feeding large numbers of tokens into higher temperature critical reasoning models and finishing with gramatical editing models. The slickborns will make the final judgement.

This is a non-story. This was a hardware event. Apple is releasing many new AI features as part of iOS 26 which will launch along side the new iPhones. AI is software. And yet, a number of the features are clearly powered by AI models such as camera enhancements, health monitoring and live translation. Also GPU performance continues to increase in the A19, with CPU remaining presumably fairly flat since no numbers were given, so that’s a win for on-device inference.

This is working really well in GPT-5! I’ve never seen a prompt change the behavior of Chat quite so much. It’s really excellent at applying logical framework to personal and relationship questions and is so refreshing vs. the constant butt kissing most LLMs do.

I think that sounds very reasonable, but unfortunately these models don’t know what they know and don’t. A small model that knew the exact limits of its knowledge would be very powerful.

You can’t vote the climate out of office. Sure our food supplies may crash, but no one person decided they should crash. No one to blame. No one to punish. This is the political reality. This man made catastrophe will feel sufficiently like an act of god for most people and they will just deal with the reduced carrying capacity of the planet as if it were some divine judgment instead of the tragedy of the commons.

It does seem like that’s our new political reality for now. I think that COVID showed world governments just how little control they have over their populations. You get folks to bend a little, but they quickly break and call for you to be thrown out of power. Getting to carbon zero or negative would be asking for an enormous sacrifice of the global population in the form of lower living standards and slower growth. After how people fought against masks, a shot and social distancing, it’s obvious to those in power that there will be no solution to this problem aside from geo-engineering or cost competitive green energy. Might as well stop talking about it.

Google’s AlphaProof, which got a silver last year, has been using a neural symbolic approach. This gold from OpenAI was pure LLM. We’ll have to see what Google announces, but the LLM approach is interesting because it will likely generalize to all kinds of reasoning problems, not just mathematical proofs.

I think the hard solution is to massively increase expectations. Think Star Trek where the grade schoolers are learning quantum mechanics. If everyone has access to the oracle of all human knowledge, then you should teach and test to the maximum of what a student could do with all that power. Find the frontier where the AI fails and the human adds value and teach there.

I think the fantasy of going back hides the reality that new possibilities are always stretched out ahead. I have lived many lives. New careers, new cities, new countries, new friends, new families. By my count I’ve lived 14 iterations of life and counting. New lives are always beckoning. After awhile you get good a recognizing the ones worth stepping into.

I’d encourage thinking not just about yourself or your immediate circle when it comes to policy like retirement age. State run pension systems affect nearly everyone, not just the extremely fortunate few who can become financially independent at a younger age. And that is also largely an American phenomenon anyway. Salaries are much lower on average than in Europe and there is far less differential between minimum wage and what senior employees get paid. There’s also not the same culture of investing because European companies don’t have the same performance as the US stock market. Lower yield real estate investment is much more common in many European countries. And with strong social safety nets Europeans are generally less pressured to save a huge nest egg.

So this raising of the retirement age will affect a huge portion of workers and 70 is OLD to be working full time for all but the most relaxed jobs. Most older folks I know have know being dealing with multiple chronic age related illnesses well before 70.

An interview is a sales pitch for a product. The product just happens to be you. Set aside whatever negative feelings you have about this previous job or the people you worked with there. The interviewers care if you will do their job well and with consistency and professionalism. Your personal feelings are irrelevant as long as you can keep them to yourself, or maybe tell your dog.

ANY negativity during a job interview is going to work against you. It is expected that you find a way to spin every situation and every project in some kind of positive light. Even when interviewers ask for weaknesses or about conflict, the “right” answer is to be able to talk about that negative thing in a way that lets your true brilliance shine through. Skilled candidates know how to inject just the right amount of humanity and relatability in an otherwise perfect employee.

If you are having trouble separating your feelings from your ability to keep to your talking points, then a good therapist may be able to help you learn better emotional regulation skills.

In the future, keep working to proactively manage your career. Keep yourself in roles where you are learning and thriving. When you feel burnout creeping in, deploy strategies to counter it or at least get yourself into a new situation.

There’s increasing evidence that LLMs are more than that. Especially work by Anthropic has been showing how to trace the internal logic of an LLM as it answers a question. They can in fact reason over facts contained in the model, not just repeat already seen information.

A simple example is how LLMs do math. They are not calculators and have not memorized every sum in existence. Instead they deploy a whole set of mental math techniques that were discovered at training time. For example, Claude uses a special trick for adding 2 digit numbers ending in 6 and 9.

Many more examples in this recent reach report, including evidence of future planning while writing rhyming poetry.

https://www.anthropic.com/research/tracing-thoughts-language...

This is a great moment to go read one of my favorite Neal Stephenson books, Diamond Age. Most of the book’s theme deals with nanotechnology, but a very significant element is The Primer, a book that has enough AI to raise a child. The Primer is benevolent and is a good depiction of what AI driven education might be capable of. Stephenson’s book is a product of the 90’s and there’s a lot of techno-optimism. I always hope for the optimists to win the day.

https://en.m.wikipedia.org/wiki/The_Diamond_Age

Strong worker protections can help in all kinds of situations. Case in point, I have a friend in Switzerland. He burned out and had a panic attack at work. The reasons were plausibly personal, but work stress always takes its toll as well. Switzerland has very strong worker protections and with a doctor’s approval he was able to take 6 paid months off while he recovered from the burnout. That was years ago and he has been back to work productively ever since.

We recognize physical impairment such as a broken leg but seem to have very little sympathy for the mental and emotional wellbeing of others.

This is worth a good giggle, as intended, but is just part of the normal space agency PR of doing relatable things like talking about using the toilet and video chatting with elementary students.

Because it’s hard to talk about the actual science being done and this is the only thing the public will pay attention to.

Poland has one of the bedrock economies of Europe and is the source of many excellent engineers and scientists.

The simple truth is that Europe lags because its common market is still quite fragmented so winner take all tech companies can scale much faster in the gigantic consumer markets of the US and China.

I know what you are saying, but I very much disagree. There are also better chess engines. That’s not the point.

It’s all about the “G” in AGI. This is a nice demonstration of how LLMs are a generalizable intelligence. It was not designed to play Pokémon, Pokémon was no special part of its training set, Pokémon was not part of its evaluation criteria. And yet, it plays Pokémon, and rather well!

And to see each iteration of Claude be able to progress further and faster in Pokémon helps demonstrate that each generation of the LLM is getting smarter in general, not just better fitted to standard benchmarks.

The point is to build the universal hammer that can hammer every nail, just as the human mind is the universal hammer.

I think in a lot of ways we are already there. Users are clearly already having difficulty seeing which model is better or if new models are improving over old models. People go back to the same gotcha questions and get different answers based on the random seed. Even the benchmarks are getting very saturated.

These models already do an excellent job with your homework, your corporate PowerPoints and your idle questions. At some point only experts would be able to decide if one response was really better than another.

Our biggest challenge is going to be finding problem domains with low performance that we can still scale up to human performance. And those will be so niche that no one will care.

Agents on the other hand still have a lot of potential. If you can get a model to stay on task with long context and remain grounded then you can start firing your staff.

Wait no, there is actually PLENTY of evidence that performance continues to scale with more compute. The entire point of the o3 announcement and benchmark results of throwing a million bucks of test time compute at ARC-AGI is that the ceiling is really really high. We have 3 verified scaling laws of pre-training corpus size, parameter count, and test time compute. More efficiency is fantastic progress, but we will always be able to get more intelligence by spending more. Scale is all you need. DeepSeek did not disprove that.

Actually it means we will potentially get 100x the economic value out of those datacenters. If we get a million digital PHD researchers for the investment then that’s a lot better than 10,000.

It seems an interesting fine-tuning idea. Drawing from reasoning models, I wonder if it’s effective to 10x or 100x the fine-tune dataset by having a larger reasoning model create documentation and reasoning COTs about the code base’s current state and speculation about future state updates. Maybe have it output some verbose execution flow analysis.

I think you have the right idea. China has yet to truly flex its muscle. They prefer to quietly grow stronger. Their response to Covid with the largely successful zero covid strategy gives a clue about the power of its government. Silly, you can’t become the champion without stepping into the ring.

When they do it, it’s “censorship.” When we do it it’s “safety.” From a technical standpoint it’s the same. Don’t say certain things, respond to certain questions with refusals or with certain answers.

I had the impression that he was ADHD, not autistic.

Someone pointed out on Reddit that DeekSeek v3 is 53x cheaper to inference than Claude Sonnet which it trades blows with in the benchmarks. As we saw with o3, compute cost to hit a certain benchmark score will become an important number now that we are in the era that you can throw an arbitrary amount of test time compute to hit an arbitrary benchmark number.

https://old.reddit.com/r/LocalLLaMA/comments/1hmm8v9/psa_dee...

“Simply brute-forcing”

That’s the thing that’s interesting to me though and I had the same first reaction. It’s a very different problem than brute-forcing chess. It has one chance to come to the correct answer. Running through thousands or millions of options means nothing if the model can’t determine which is correct. And each of these visual problems involve combinations of different interacting concepts. To solve them requires understanding, not mimicry. So no matter how inefficient and “stupid” these models are, they can be said to understand these novel problems. That’s a direct counter to everyone who ever called these a stochastic parrot and said they were a dead-end to AGI that was only searching an in distribution training set.

The compute costs are currently disappointing, but so was the cost of sequencing the first whole human genome. That went from 3 billion to a few hundred bucks from your local doctor.

That sounds reasonable to me because the compute cost for this level of reasoning performance won’t stay at 10^22 and phones won’t stay at 10^12. This reasoning breakthrough is about 3 months old.

So what? I’m serious. Our current level of progress would have been sci-fi fantasy with the computers we had in 2000. The cost may be astronomical today, but we have proven a method to achieve human performance on tests of reasoning over novel problems. WOW. Who cares what it costs. In 25 years it will run on your phone.