vessenes

I was going to say the reverse - claude has been the less satisfying normalized by benchmark for me in the last year. Both astra and fable have their quirks, but I am 90% codex this year up from 10% last year.

Nice to see this release cadence increasing and some continued improvement in quality. I am guessing these models are basically still outcomes of the cursor team integrating with the massive amount of compute they now own: I’d imagine we will see significant step up improvements with grok 5 later this year as the team gets more experienced and confident with larger training deployments. Here’s hoping for another competitive frontier model!

Agreed. Another difficulty here is there are not good benchmarks for this new architecture yet, so it’s easy to potshot and snipe, where jev seems to be pretty broadly intelligent/at least have had a lot of rl in different domains.

We haven’t seen any of these copy cats play doom or street fighter for instance; just categorize email.

I imagine once the author cools down and evaluates on a broad harness of tasks he may find that his new thing has a lot of engineering work ahead.

A counterpoint - I was told a story by one of my professors in the late 1990s, about one of his professors -- he'd written a thesis, gotten hired somewhere like Princeton, and taught there for a few years as Dr. <Somebody>. One day he received a letter pointing out a construction flaw in his thesis. He brought it to the department head who read the letter, and said "Well, Mr. Somebody, ..." Ultimately he fixed the proof.

Upshot, if there are real errors in published work, I think most mathematicians want to know about them.

Well, duh. If you could do this with Opus 4.8, we would know. When Astra’s successor is 2-3x better at math research, and the internal teams say “we believe we will get there,” I’m inclined to believe the insiders.

Sorry, I don't understand what this means. What does it mean?

I think that implies they're seeing unusual new subscription demand, yes? That's how I'd read it, not least because I worked through four resets this week on Astra, which is un unbelievable amount more inference than I've wanted from openAI really ever, and the highest ratio vs. claude since opus 4 at the very least, probably farther back.

I think there are few moats in the engineering use case, and a single new model can absolutely drive compute demand.

Prices are moving inference to highly profitable, oAI seems to have solved their training problems, and they're currently competing nicely with Anthropic on the coding side. I'd wait, too -- why fight this stuff out in public when you can stay private and have your big competitor deal with all the public company concerns? There's plenty of capital available in the private markets for them right now.

“We” does a lot of work here. You keep using that word. I don’t think that word means what you think it means.

Shenanigans is an inaccurate word; it implies underhanded behavior that's hidden / concealed. I think "to act so aggressively" is more balanced.

Here's my answer: If you think we're getting to AGI in the next 9 months, then you believe, with all your heart, that these problems will fall soon. However, there's an ocean to boil in terms of what you could point your limited clusters at. In the meantime, the market is desperate for any sign your company might be first to AGI. Therefore, news of tractability with current models might focus an organization intensely - internally they have a huge leg up on the public, and therefore it's minimal compute to check - and if they are successful, they get approximately $50 million of free PR, likely adding 10-20% to their valuation.

Likewise someone like Tristan is fighting for his (metaphorical) life right now, hoping to preserve his claims of primacy and have a shot at some of that prize money, despite being only partway to a full solution for N-S.

I don't think we see any behavior at all that isn't simple to understand and well described by the setup here, but tell me what you see differently.

Yep, it's possible. But, we have literally no idea how much a set of prompts would impact training as far as general usefulness. I don't think we even know if Tristan's said he allowed training on his prompting or not. This is about money, ego, primacy, all the usual mathematician priority disputes.

This is speculation right now. The idea would be that if someone used the product and granted training rights, which is the default for many subscription levels, then some knowledge would have been imparted into the general weights of the new model.

oAI has made clear they did not specifically pull in any user data to context for this run.

Thanks! It's just infra, so you could do what you wanted. The clients include a safety reminder, and the datastructures include "unsafe" in the name of the inputs passed around, but that's just a little hygiene.

There really aren't good messaging libraries that provide what I wanted, so I built it. Basically I started with "signal but no need to have a phone number." I've used it to build a group messaging iOS app for friends, and just pass it to an agent all the time if I want them to be able to direct message.

The group API approval is modeled off of multi signature approval mechanics from Ethereum - so, you could use it like you describe: "Only allow this call if it passes safety checks from n reviewers", or you could have human in the loop, or a program that checks business rules + an agent and a human, etc. etc. I just wanted something that let us control API calls properly.

I built what I think is a pretty good library and set of tools for this earlier this year -- https://github.com/corpollc/qntm (or `uvx qntm --help`); it includes a cli, python and typescript libraries, and works out of the box aimed at either a public endpoint, or a private one, depending on environment.

It's end to end encrypted, and has group messaging support, so if you wanted to read what the agents are saying you'd just add them to groups you're in. It also has a web ui. Version 0.6.0 should get pushed this evening pacific time, with some additional agent specific features.

Bug reports welcome! If I did a good job on architecture, you should be able to have your blackboard up tonight.

Not arguing against open science - it's super valuable. I'm saying that pearl clutching by people reading Tao isn't useful, because it misses some long history which tells us that this kind of science has been seen as fundamentally competitive for millennia.

Should it be competitive? Is it more useful to be collaborative? How collaborative can it be when it's fundamentally competitive? Is it only fundamentally competitive because of some common 'quirks' of math types, or are there deeper forces pressuring it to be competitive?

These are all questions that I think are worth discussing, as is the note that the pendulum seems to be swinging away from cooperation in the face of competing for $trillion+ valuations (and a real enthusiasm for proving cool math stuff). The alternative, tweeting complaints on twitter without some context, is mostly a waste of space. I mentioned the history in hopes we could get informed complaints on twitter.

That’s not untrue. But it’s also a misstatement of mathematical history. Many leading mathematicians historically have been highly competitive — Gauss comes to mind. Woe betide the lesser intellect that sent Gauss some ideas. The Newton Leibniz controversy was very serious business at the time in the UK and the continent. It was considered at the least a sin to reveal that sqrt(2) was irrational to those outside Pythagoras circle.

Mathematics has always been highly competitive.

If those researchers did not opt out then training data might go in. I think it’s a courteous acknowledgement; as was reaching out and examining the direction of proofs themselves. At stake here is a particular mathematician dynamic - ego, prize money, and the sense of proprietary ownership that some might feel working on a problem.

All that was just kicked in the teeth by a group with a lot of compute that was like “bro I heard on twitter that Navier stokes could be solved. Let’s try it.” That’s an existential level of engagement that almost no mathematician in history would like.

I can’t read your original comment, but when the chief scientist of a company that just released six month old software that can use Kicad on your computer to design a circuit board and have it created and shipped to you tells you he thinks we will get to automatically self reinforcing improvements, I think it’s wise to take him seriously. It was only eighteen months before Astra was created that LLMs could not count rs in strawberry.

The situations aren’t equivalent - luckily in my opinion because the stakes with nuclear are much higher. von Neumann constructed a multinational game theory approach appropriate for weapons. AGI is a much harder problem to corral because there are so many benefits beyond just blowing up cities. But it’s also a much better thing to have for these very same reasons.

Similarly there have been few positive externalities from nuclear industry, making it easier to make the case to wind down research. This same set of concerns in biotech is much harder to get compliance with, precisely for this reason.

Anyway I’m especially wary of over analogizing to nuclear era concepts: I think they’re a trap.

I’m like a 2(.5?) there - I don’t think ASI will care about my kids better than I will for some definitions of better, for instance, and I feel very fuzzy and vague about what actual differences in qualia between me and ASI would yield in the wild.

I’m not a doomer, although I don’t think doomers are dumb, just wrong. I think you should design your systems around the possibility that people who disagree with you are correct , hence my nod to negative sum. If you have more than 30 years to live, I’d personally rep to the most likely outcomes being very positive. With a lot of disruption in the middle.

No. These parties are composed of people who most definitely think this way, and therefore will have distinct goals and interests when presented with opportunities. That’s reality quite aside from how a game theorist assesses the situation.

This is a good essay, and makes me hopeful.

I’m on the record saying that it is extremely dangerous to slow down because the race for AGI is a zero-trust game — defections pay - and combined with a compounding returns model on defection, if you have any strategic adversaries whatsoever you MUST NOT slow.

For slowing to make sense, you need to believe that you can transform the zero trust game into a cooperative game, or that it’s likely racing will lead to a negative outcome for the ones racing ahead (and not everyone else). I don’t believe either of these outcomes are possible, and so I advocate for racing, acknowledging the entire game might be a negative value game, or at least could be for some time — it’s even worse not to play it.

But, I like hearing what reads to me like very thoughtful and informed (internal) policy considerations is great — the public messaging from Sam and Dario just seems so facile and simplistic I’ve been worried.

Anthropic’s Mechinterp did some very fine work on this. TLDR - you can; you train a decoder on neuralese to english and then add a loss function for a roundtrip of english -> neuralese -> english (or possibly n -> e -> n? I don’t recall), giving a pretty strong indication that you have a good ‘translation’.

They published open weights versions of these interpreters for a number of open models sometime in the last year. Very cool idea.

By the way, they concluded CoT often lied, based on the neuralese interpretation.

EDIT: a comment below linked to https://www.anthropic.com/research/natural-language-autoenco..., which is what I was referring to.

Looked to me like it missed the chain though.

Those tend to be under so-called ‘max’ modes, or ‘ultra’ - where you scale up inference time compute and then choose amongst your answers. Astra is new weights.

It’s already running in Hawthorne, CA

True, .. and. In this case, the original proof is considered rigorously checked, so finding a bug in the kernel would be nice to know about, but in my opinion would not take away from the accomplishment (FLT in lean using agents) nor the many benefits of getting these mathematical objects formalized and usable in Lean in the future.

I'd like to note that we should remember a formalized Lean proof does have value in that it enters the pantheon of true things other Lean proofs can rely on. Agreed that for the humans, descriptions and being able to 'grok' the proof / assess it for new tools and concepts is extremely helpful.

Pretty efficiently, apparently, since it saturated ARC-AGI-3 in half of the predicted time, and according to the Chollet blog post on the fly created dense DSLs to describe and analyze individual games.

Weave robotics (no affiliation) claims they are folding over 1,000 lbs of laundry a week in homes right now: https://www.weaverobotics.com. I have no idea how well these work.