As models get better, I think a time will come when it's hard for people to even verify the results. In the end, I think the bottleneck will be people.
If someone hurts another person with a knife, we can't punish the person who sold the knife, but I think someone who sells nuclear weapons needs some level of regulation. I think OpenAI needs some level of prevention for the problems that can be expected.
There may be great researchers who work on several papers in a month, but even with this rule they can just split them and submit to arxiv, so I think it is very reasonable. AI slop is everywhere now (and also increasing), and I think this fast and forceful action by arxiv is effective at stopping this wave of slop.
I'm not sure Google can cut in when Claude and ChatGPT already got. I'm using both, but I'll keep track of whether Google can make it good enough for me to use also this Gemini, or cancel one of the two (Claude and ChaGPT) for it.
Even if the chance of these phenomena is estimated at one in billions, since the number of trials is much higher, and as long as the agent knows these paths, blocking it within the model itself would be impossible I think. If it can be blocked, the only case would be blocking the physical path of the agent's access itself (one the agent cannot intervene), not the model ones.
It sholud definitely not be more capble than Fable. It seems tight comparision but cannot find out its details. I'll use this and check the perceived performance.
I also have set custom harness environment for me in the company (and also for at home), I expect that big companies (OpenAI, or Linear and Snowflake) may release the internal harness setting or functions that would be more comfortable than we already set and may replace the envs we set.
The structured output that Google provides (the function that blocks the disallowed words) may calculate softmax output with assigning -inf value to the other words, then the probability is not the same as the weight the model firstly put on those allowed words. And here, reasoning comes before verdict, so the decision is already made before the last token classification. Resultingly, the last token made seems to not reflect LLM's original intention.
But, I think that the agent may also generate like this approach, not directly generate text, instead first writing the generation code then execute if they are the smart agent.
Since the baseline LLM architecture is similar with each other, maybe already there are relations between their each opinion. Of course, this is an assumption and the explicit training seems effective in this case. But, I'm curious whether the approach would be effective in other cases (in terms of generalization?)
Six months into customizing my own Claude Code harness, I've settled on assuming Anthropic and OpenAI will just handle all of it, except turning my own flows into skills.
Is the attention explanation of why the model tells like this? I've seen that there are many discussions about this. (Image attention visualizations were not that good I think)
Ah, success rate here are scored by an agentic classifier. And uncertain outcomes are excluded from the graph. The thing measured and grading it comes from the same house. In my setup, review agent pass work that an outside critic later rejects
I only get web search from Claude Code when I ask, only one of my last 8 sessions accessed the web at all. But they were the case of Opus here. Curious what Fable 5.1 model does instead of Opus.
I tried to make a memory system in Claude Code (make it behave like a human from the perspective of memory) using markdown files and indexing plus hooks. But, retrieval is not as good as a human and since keywords are ambiguous, it doesn't behave as intended.