Hacker News Reader

Comment by amluto | Hacker News Reader

Wow, that whole thread is borderline incoherent, presumably generated by an AI without adequate oversight.

Here are the docs:

https://developers.openai.com/api/docs/guides/prompt-caching...

The thread has little explanation as to what weird thing they’re doing to Codex that is making the default work poorly, and it kind of seems like it’s getting confused about whether it wants to set the caching mode or the breakpoint or both.

In any case, I find the behavior change interesting. It sounds to be like 5.5 and below may have been using a conventional attention scheme where a cached KV sequence can be easily used to restore a prefix of itself, but perhaps 5.6 is using linear attention or LSTM or another recurrent scheme where you cannot rewind the model state by just truncating it.

View on HN

I have noticed the same thing starting with 5.6 when editing my last prompt inside the vscode codex plugin, I’ve seen the model’s thinking respond to the edit with a remark.

Slightly bummed out about it because in the past you could try different situations during a planning session and it wouldn’t pollute the cache but now it does. I’m not sure if forking the conversation has the same problem.

It’s really cool that we have this proof that US companies are half year behind Chinese models in architecture.

What is the proof?

Nemotron was using hybrid with recurrence via mamba layers since around April 2025.

Then why are they (US frontier models) still so far ahead whenever I test them against the latest Chinese models? No bias here, I'd love them to be better for my own personal gain, but I haven't seen it

Behind on architecture, ahead on training? It seemed pretty obvious to me that the opus 4.7 and 4.8 releases were more about trying to retain 4.6-level capabilities while being cheaper to run, which would fit. And they can burn so much money on training.

I don't know I just care about the end result. And yeah what you're mentioning here is a pretty common conspiracy theory but you don't actually have any insight into that do you?

Apologies, you asked a question. I assumed that meant you were interested.

You didn't answer my question you literally just asked me another question and then parroted a common talking point about Opus models (which isn't even frontier - Fable is)

Meh. You asked why, I suggested a possible reason, you them said you don't care about why. Congratulations, I now regret engaging.

Sorry I forgot to thank you for just adding more noise

There is so much misinformation in the ecosystem, parrots just hitting "Reply" without thinking one iota, you really cannot trust "human" opinions on the internet anymore, anywhere.

Same with local LLMs, I'd love to use them for my day-to-day software engineering, and I'm not exactly GPU poor, then people with 12GB VRAM try to convince me their local setup is perfectly fine running latest Qwen and it does real engineering but whenever I try, they're a far cry from what Codex+GPT 5.x would do.

Only way to be sure is creating your own private benchmarks and use those, and the difference in quality becomes very apparent, very quickly, for your specific use cases.

> It sounds to be like 5.5 and below may have been using a conventional attention scheme where a cached KV sequence can be easily used to restore a prefix of itself, but perhaps 5.6 is using linear attention or LSTM or another recurrent scheme where you cannot rewind the model state by just truncating it.

I feel like this is the kind of substantial change to your product that you would need to tell your customers about. It would be simply disrespectful to your customers to not disclose this upfront.

It depends on who they consider the customers. Shareholders and govt are the customers, not users.

Users is the product.

“Users are the product” is a phrase used when the users aren’t the ones paying for a free service. For a paid API the users absolutely are the customers.

> Wow, that whole thread is borderline incoherent, presumably generated by an AI without adequate oversight.

What sucks is that every issue tracker for these agent harnesses are the same, and this shit hides real issues!

For example, Codex started encrypting messages from a agent to the sub-agents when you use Sol + Ultra, which is terrible for debugging for obvious reasons. This GitHub issue exists for this: https://github.com/openai/codex/issues/28058

Fine, the opening issue isn't concise exactly, but it's mostly clear what's going on. After a few messages, someone who uses LLMs without reviewing their output starts participating in the discussion, pastes huge walls of texts completely missing the point and overall just bloating the conversation so now whenever a maintainer actually want to address it, they have to wade through 20+ messages of just pure shit and bloat, to even understand what's going on.

Kind of wish some projects started having forums specifically for people who pay for forum access, or some other gate to get rid of these LLM lowlifers who cannot compose a simple message to explain what's in their head, and instead have to ruin perfectly fine conversations/discussions with their verbal poop.

i dont know what the hell is going on lately i pop in to issues or discussions and its agents talking to each other or telling me what PR to merge

I mean i use AI too but was taken back when an agent popped up dictating what i should do and so on....felt weird

My most awkward experience was a maintainer commenting on my feature request just to prompt a bot to "explain to issue reporter why this is very hard to implement."

It felt like they were trying to avoid me. They could have simply addressed me and given me the explanation they gave to the bot: it would have been simpler for him and more polite. I did in fact reply without waiting for the bot.

It's as though you're talking to someone and they were said to their 'assistant', "Explain this to this person" and walked away. It doesn't really matter what the explanation is, it's just gross.

Reply on news.ycombinator.com