samusiam

Hypothesis Driven Development would beg to differ.

I think System 1 is a great term. System 1 is fast, intuitive, and automatic. It describes decision-making that happens without explicit reasoning or deliberation. System 2 is the opposite: it's the more deliberate, "executive functioning" side of cognition -- the part that reasons through a problem before arriving at an answer. That's also what state-of-the-art LLMs do before they respond. Jev doesn’t do that kind of reasoning. It just decides.

> But the moment you build your own agent, you’re on your own. It’s tricky wiring up the right primitives just well enough to match that “it just worked” feeling.

It's wild to me to claim that it's tricky to customize one of these harnesses and for that to be the entire justification for an entirely different harness.

It's really not that hard. If you want to reduce costs then all you need to do is practice delegation: instead of using the strong model, all the time to do everything, instead, you have the stronger model delegate well-defined tasks to a weaker model. Patterns like these are really easy to wire up.

It wasn't

You call it a guess, I call it a hypothesis.

After you sure that isn't because your skin is moist where sunblock was recently applied, and therefore cooler?

To me, one interesting piece of analysis was whether Luna had benefit on top of Astra -- i.e., running both and synthesizing their findings. But even with that, it raises the question whether running a second Astra pass, or Sol, or even a model from another family (GLM? Fable?) would deliver even more additive benefit.

My thoughts exactly. That 22pp gap is massive, and it seems to me Astra is worth every extra penny.

Which is pretty much a useless (i.e., saturated, contaminated) benchmark now.

I'm still using Sync, going on three years since they banned third party apps, and third party apps are still the best way to experience the site.

I don't see anything asking for a credit card.

In my experience this "gaming" behavior is easily caught by just asking another agent (could just be another session of Claude Code) to review the code changes.

For idle sessions I would MUCH rather pay the cost in tokens than reduced quality. Frankly, it's shocking to me that you would make that trade-off for users without their knowledge or consent.

That's true, but the "AI bubble bursts" scenario is usually tied to Western investors getting essentially margin-called. If that happens, the CCP won't suddenly stop their investment; Chinese models will most likely continue developing.

> I have no clue if this claim holds, but alas, just pretending they did not address the obvious criticism, while they did, is at the very least pretty lazy.

But they didn't address the criticism. "cutting ~75% of tokens while keeping full technical accuracy" is an empirical claim for which no evidence was provided.

> Aren't they currently propped up by investor money?

Are Chinese model shops propped up by investor money? Is Google?

Open weights models are only 6 months behind SOTA. If new model development suddenly stopped, and today's SOTA models suddenly disappeared, we would still have access to capable agents.

These OSS model makers need to stop benchmarking against old models. Showing how it performs against Opus 4.5, GLM-5 when we have Opus 4.6 and GLM-5.1 just tells me that it's not comparable to SOTA.

I think there's a lot of methodological expertise that goes into collecting good eval data. For example, in many cases you need human labelers with the right expertise, well designed tasks, well defined constructs, and you need to hit interrater agreement targets and troubleshoot when you don't. Good label data is a prerequisite to the stuff that can probably be automated by the AI agent (improving the system to optimize a metric measured against ground truth labels). Data scientists and research scientists are more likely to have this skillset. And it takes time to pick up and learn the nuances.

Well you're free to disagree but my experience has been counter to your position. I write both code and research / technical documentation. The quality of what the LLM produces is limited by the quality of ideas I give it initially (mind you, this is just a starting point), and the quality of my review of its output.

You're complaining about vibe coding while also complaining about how you "feel" about the code. Do you see the irony in that?

I haven't seen the scrolling glitch in months, where previously it was happening multiple times a day. Also haven't seen anyone complain about it in quite some time. Pretty sure they have resolved that.

I just checked competitors' codebases:

- Opencode (anomalyco/opencode) is about 670k LOC

- Codex (openai/codex) is about 720k LOC

- Gemini (google-gemini/gemini-cli) is about 570k LOC

Claude Code's 500k LOC doesn't seem out of the ordinary.

AI witch-hunters are even more annoying.

> I think the only reason it’s seen as good anywhere is there are a lot of tasteless and talentless people who can pretend they created whatever was curled out. This goes for code as well.

This is an oversimplification.

If you have taste and talent, then the LLM output you get is going to reflect that.

So on the one hand, yes: tasteless and talentless people won't know good output from bad output. On the other hand, people with taste and talent can actually get good output.

"by design, the recommendations will be average"

This couldn't be more wrong. The simplest refutation is just to point out that there are temperature and top-k settings, which by design, generate tokens (and by extension, ideas) that are less probable given the inputs.

"they don't output anything unless prompted"

Unprompted they're not unlike a human sleeping or in a coma. Those states don't preclude consciousness in other states.

Vegan for 15 years. I cook 95% of my own meals, including black bean burgers, tofu, etc... Sometimes I want something that tastes like meat and I reach for a Beyond or Impossible burger. I don't need it. But I can't recreate its texture and flavor profile on my own. It's not "better" than other things I can cook. It's just different.

I can recall reading human-authored text like this for more than a decade.

But literally any decent agent can recommend existing services and help you set them up. And even help you help them set the services up for you. I do this with Claude all the time.

IMO we should all be asking for a raise if our company is making more money. Proportionally, even.