motoboi

This is how anthropic captured the code agent so fast. You need training data, users are giving it to you.

Being a terminal application, all interaction is trainable signal (unlike, say, cursor, which is an IDE and let users freely explore, edit the files, move the mouse. Model sees nothing of it, nothing to train upon).

So meta is doing the obvious, we want to train a computer use model, we need training data. Better to capture from employee than buying low quality data.

Well, the model that a lot of people have been given access to and are reporting about on twitter?

With the previous generation? Yes. With 10T mythos-level models? Not even close.

Don't. thinking right now is just text. Chain of though, but just regular tokens and text being output by the model.

Caveman hurt model performance. If you need a dumber model with less token output, just use sonnet-4-6 or other non-reasoning model.

The reasoning is the secret sauce. They don't output that. But to let you have some feedback about what is going on, they pass this reasoning through another model that generates a human friendly summary (that actively destroys the signal, which could be copied by competition).

People say we don’t build cathedrals anymore.

But here it is: JVM is a modern cathedral.

I believe what is missed here is that the brain assimilate things outside of it (not in the physical sense of course). Use a hammer for some time and the brain start to dedicate some networks to simulate the hammer internally and to integrate the hammer as parte of your body. The brain starts using the hammer in the same way it uses your hand. It becomes part of what the higher level processes use to read from and to manifest into the world.

Nothing new here, this is called tool embodiment. A little time after assimilation, you stop consciously thinking about the tool. You are the hammer, the hammer is you.

So what is being missed here is that the brain operates on, well, mental constructs. Ideas and ways of thinking. But those are not the world or the brain, those are tools.

The higher level processes into your mind use ways of thinking the same way it uses the body. Its unconscious (because it has been doing this for enough time) and automatic. The brain just gets guided by the tools. It wants to hammer that nail.

So, what does it have to do with crossing the street and not being able to transmit this knowledge?

You can’t transmit the incorporation. You can describe how to do things, how to think about things, but you can’t reconnect other people’s neurons to establish a way of thinking or a tool as part of the brain image of the self. Yet.

You can’t teach a baby how to embody his spine. You can’t teach someone how to become his thoughts. But you can’t certainly guide then on the use and this usage will build the neural networks. Once established, they’ll get it.

> it was too easy to put YouTube running

That is what is killing your focus and giving ADHD-like symptoms (supposing obviously that you don’t actually have it).

Never ever put something to play on background while you do something. That on the long run will kill your attention span and focus ability.

Im sorry. You are thinking in terms of one time interactions. I’m thinking about the next step in the interaction.

To understand my point, think about a prompt to tell the model “here is a very difficult code problem, answer in a single word.”

It thinks a lot and answer. You send the next prompt. At this moment, you are completely in out of distribution territory.

The reasoning part is not diferente from the part that goes in answer. It’s just that the model is trained to do some magical text generation with back and forth. But when it’s writing the answer part of it, each word is part of its context when generating the next. What that means is that the model does not compute then write, it generates text that guide the next generation in the general direction of the answer.

If you steer it in strange (for it, as in not seen before in training) text, you are now in out-of-distribution, very weak generalization capabilities territory.

The output is part of context. The model reason but also output tokens. Force it to respond in an unfamiliar format and the next token will veer more and more from the training distribution, rendering the model less smart/useful.

If they want to reduce token cost, just use a smaller model instead of dumbing down a more expensive.

Things like this make me sad because they make obvious that most people don’t understand a bit about how LLM work.

The “answer before reasoning” is a good evidence for it. It misses the most fundamental concept of tranaformers: the are autoregressive.

Also, the reinforcement learning is what make the model behave like what you are trying to avoid. So the model output is actually what performs best in the kind of software engineering task you are trying to achieve. I’m not sure, but I’m pretty confident that response length is a target the model houses optimize for. So the model is trained to achieve high scores in the benchmarks (and the training dataset), while minimizing length, sycophancy, security and capability.

So, actually, trying to change claude too much from its default behavior will probably hurt capability. Change it too much and you start veering in the dreaded “out of distribution” territory and soon discover why top researcher talk so much about not-AGI-yet.

I insist on the text generator nature of the thing. It’s just that we built harnesses to activate on certain sequences of text.

Think of it as three people in a room. One (the director), says: you, with the red shirt, you are now a plane copilot. You, with the blue shirt, you are now the captain. You are about to take off from New York to Honolulu. Action.

Red: Fuel checked, captain. Want me to start the engines?

Blue: yes please, let’s follow the procedure. Engines at 80%.

Red: I’m executing: raise the levers to 80%

Director: levers raised.

Red: I’m executing: read engine stats meters.

Director: Stats read engine ok, thrust ok, accelerating to V0.

Now pretend the director, when heard “I’m executing: raise the levers to 80%”, instead of roleplaying, she actually issue a command to raise the engine levers of a plane to 80%. When she hears “I’m executing: read engine stats”, she actually get data from the plane and provide to the actor.

See how text generation for a role play can actually be used to act on the world?

In this mind experiment, the human is the blue shirt, Opus 4-6 is the red and Claude code is the director.

The whole “chat with an AI” paradigm is the culprit here. Priming people to think they are actually having a conversation with something that has a mind model.

It’s just a text generator that generates plausible text for this role play. But the chat paradigm is pretty useful in helping the human. It’s like chat is a natural I/O interface for us.

Working on TRIPA, a internal pack format to save huge amount of small images to cloud storage without dying from the write costs.

Im planning a change that will save 20k a month of storage.

I absolutely could come up with the details and implementation by myself, but that would certainly take a lot of back and forth, probably a month or two.

I’m an api user of Claude code, burning through 2k a month. I just this evening planned the whole thing with its help and actually had to stop it from implementing it already. Will do that tomorrow. Probably in one hour or two, with better code than I could ever write alone myself.

Having that level of intelligence at that price is just bollocks. I’m running out of problems to solve. It’s been six months.

In 2020 I became a full time Java developer, coming from a infrastructure role where I kind of dealt with Java code, but always as artifacts I managed in application servers and whatnot.

So when I first started dealing with the actual code, it scared me that the standard json library was basically in maintenance mode for some years back then. The standard unit test framework and lot of other key pieces too.

I interpreted that as “Java is dying”. But 6 years later I understand: they were are feature complete. And fast as hell, and god knows how many corner cases covered. They were in problem-solved, 1-in-a-billion-edge-cases-covered feature complete state.

Not abandoned or neglected, patches are incorpored in days or hours. Just… stable.

All is quiet now, they are used by millions, but remain stable. Not perfect, but their defects dependable by many. Their known bugs now features.

But it seems that no one truly want that. We want the shiny things. We wrote the same frameworks in Java, then python the go then node the JavaScript the typescript.

There must be something inherently human about changing and rewriting things.

There is indeed change in the Java ecosystem, but people just choose another name and move on. JUnit, the battle tested unit testing framework, had a lot to learn from new ways of doing, like pytest. Instead of perturbing the stableness, they just choose another name, JUnit5 and moved on.

Brazil’s free software initiative in 2000’s was all about technological dependency.

Brazil was hoping to leverage governmental spending to kickstart a national software development industry. Some sort of leap into the future, jumping over first the industrial era and then service-based economy we missed.

It was killed with fire by huge Microsoft (and American, I suppose) lobbying in congress, but then America had a very favorable public view as a nurturing and democratic partner. Some sort of older brother guiding you into adulthood.

Currently, at least in my bubble, the public view of America is more like a predator with Trump as a protodictator. Not necessarily true, understand me, just as that older brother view wasn’t. But it’s public perception.

A good part of that disabling of the Brazil initiative was simply free Google workspace for public universities (which were in the government plan).

I suppose that given the existencial threat level of anxiety caused by current developments will probably make Europe government immune to American lobby (at least in the short term), so I suppose this can actually happen.

Let’s see how it develops when they try to ban Microsoft from the universities. That would be the acid test.

thanks for that!

By typography alone I can now turbopuffer is written in zig.

This is not AI slop, it’s advertise in LLM era.

Let’s be honest. The whole thing is just the prevent Claude from “rm -rf / “.

It’s it someone is trying to avoid the thing talking to the internet or reading your emails, it’s just that it sometimes has the strange itch to change some files outside of the project.

Yeah, but we used to employ hundreds of people per farm, or per plantation, to be exact. Thousands maybe to do the sugar cane work, as an example. Replaced by 5 high tech, GPS driven, human on board to supervise, not even to drive, tractors.

So human doing lawn with mechanized tools: efficiency goes though the roof. Still one per home.

Human doing high volume manual labor job where there were much more job than single human could handle: number of humans doing the job now is amount of work divided by amount of work human can handle.

Of course we get ambitious, like Panama Canal building ambitious. But even that can’t absorb the previous admin of people doing that kind of work.

There are thousands like you now. How many does it take to run the economy? What would the rest do.

Think of it like what a tractor did to agricultural work. The fist guy that used a tractor probably thought: this is not replacing me, I’m just much more productive. Well, turns out you only need one guy per farm now.

gemini-cli being such a crap tells me that Google is not dogfooding it, because how else would they not have the RL trajectories to get a decent agent?

One thousand people using an agent over a month will generate like 30-60k good examples of tool use and nudge the model into good editing.

The only explanation I have is that Google is actually using something else internally.

It’s a wonderful idea. Vibe coded, but not his first rodeo.

Exited on his first company on 110M, then some years of the whole huasca and forest thing, then started creating projects.

Clawdbot (later openclaw) was his 44th try.

This is basically acquihire. Peter seems really a genius and they better poach him before Anthropic do.

Two things to pay attention here. Cerebra’s can only run a small version of GPT5.3, because why else would they run only a smaller model?

Also, where is gpt-5.3-codex on azure? Opus 4.6 is available since the launch in both azure and google vertex. Codex is nowhere to be seen.