SkyPuncher

I agree.

I have flexibility to shift my core working hours (and what I do during N/A business hours). Knowing they're explicitly making it dumb because of load is important. It allows me to shuffle my work around and run heavy workloads late at night (plan during working hours then come click "yes" a few times in the evening).

> with embeddings Pairwise cosine similarity, threshold 0.85

So, your system is unable to differential between AWS and Azure (~95 similarity). Probably unable to consistently differentiate between someone saying they love and hate something.

Yea, I spent a lot of time in this space last year. Contradictions on meaningful data are incredibly contextual and often impossible to fully define in isolation. Real world data is messy and often complex, which means you can't simplify to it's sub components and isolate it from it's context.

This is like 95% of the memory systems I see posted here. Someone comes up with arbitrary configuration of tools that sound like they'll solve the problem then completely ignores how the system actually works.

In most cases, they're getting these systems to work because of some other prompt they've written that'd probably work better with a normal file system.

Nice LLM post.

Yea, I've realized that if I stay under 200k tokens I basically don't have usage issues any more.

A bit annoying, but not the end of the world.

How can you decide if something is a contradiction without having the context?

I'm incredibly interested in this as a product, but I think it makes too many assumptions about how to prune information. Sure, this looks amazing on an extremely simple facts, but most information is not reducible to simple facts.

"CEO is Alice" and "CEO is Bob" may or may not actually be contradictions and you simply cannot tell without understanding the broader context. How does your system account for that context?

Example: Alice and Bob can both be CEO in any of these cases:

* The company has two CEOs. Rare and would likely be called "co-CEO"

* The company has sub-organizations with CEOs. Matt Garman is the CEO of AWS. Andy Jassy is the CEO of Amazon. Amazon has multiple people named "CEO".

* Alice and Bob are CEOs of different companies (perhaps, this is only implicit)

* Alice is the current CEO. Bob is the previous CEO. Both statements are temporally true.

This is what I run into every time I try to do conflict detection and resolution. Pruning things down to facts doesn't provide sufficient context understand how/why that statement was made?

My suspicion is the have an overall fixed cache size that dumps the oldest records. They’re now overflowing with usage and consistently dumping fresh caches.

During core US business hours, I have to actively keep a session going or I risk a massive jump in usage while the entire thread rebuilds. During weekend or off-hours, I never see the crazy jumps in usage - even if I let threads sit stale.

This is my exact experience as well.

It’s further frustrating that I have committed to certain project deadlines knowing that I’d be able to complete it in X amount of time with agent tooling. That agentic tooling is no longer viable and I’m scrambling to readjust expectations and how much I can commit to.

And it’s working larger because the other models haven’t figured out how to provide a consistent, long running experience.

I’ve never been actually rate limited. Usage limits display in yellow when you’re above 90%. At the limit, you’ll get a red error message.

This is the problem most people are facing. Before March, I had hit the rate limit as single time. That involved security audit of our entire code base from a few different angles.

As of now, I’m consistently hitting my 5 hour limit in less than 1 hour during N/A business hours. I’m getting to the point where I basically can’t use CC for work unless I work very early or late in the day.

I skimmed the issue. No wonder Anthropic closes these tickets out without much action. That’s just a wall of AI garbage.

Here’s what I’ve done to mostly fix my usage issues:

* Turn on max thinking on every session. It save tokens overall because I’m not correcting it of having it waste energy on dead paths.

* keep active sessions active. It seems like caches are expiring after ~5 minutes (especially during peak usage). When the caches expire it sees like all tokens need to be rebuilt this gets especially bad as token usage goes up.

* compact after 200k tokens as soon as I reasonably can. I have no data but my usage absolutely sky rockets as I get into longer sessions. This is the most frustrating thing because Anthropic forced the 1M model on everyone.

Yes, but the problem with these models isn't a gradual shift, it's a step function. With a gradual shift, the world has time to react and adapt.

Further, Opus identified most of the vulnerabilities itself already. It just couldn’t exploit them.

Mythos seems much, much more creative and self directed, but I’m not yet convinced the core capabilities are significantly higher than what’s possible today.

The full price of finding the vulnerabilities was also something like $20k. That’s a price point that brings a skilled professional in to accomplish the same task.

No, but I doubt there are more than 100.

The magnitude is so small that anonymity is essential broken.

That part of things is what really made this entire argument all apart of me.

There are ~50k psychiatrists in the US. Roughly, 1 in 10k people in the US is named Scott. Mathematically, that means knowing "Scott is a psychiatrist" brings you down to ~5 people. Even if we assume there's some outlier clustering of people named Scott who are psychiatrists, we're still talking about some small number.

Surely adding in the middle name essentially makes him uniquely identifiable without an other corroborating information.

I've switched to max thinking mode as my default and that's helping in some capacity.

It's not necessarily back to where it was, but it's not desk-flipping bad.

Keep in mind that Opus detected most of these vulnerabilities, it just didn’t exploit them (says so much in the article).

I’m honestly not convinced this is changing the landscape significantly. It’s simple a bit better at self directing.

Yes, I agree. I’m about to drop Claude Code because it’s become literally unusable.

Today, Opus went in circles trying to get a toggle button to work.

I was reading the Glasswing report and had the same thought. Most of the stuff they claim Mythos found has no mention of Opus being able to find it as well.

Don’t get me wrong, this model is better - but I’m not convinced it’s going to be this massive step function everyone is claiming.

It just doesn't listen. Literally a conversation that I just had:

* ME: "Have sonnet background agent do X"

* Opus: "Agent failed, I'll do it myself"

* Me: "No, have a background agent do it"

* Opus: Proceeds to do it in the foreground

* Flips keyboard

This has completely broken my workflows. I'm stuck waiting for Opus to monitor a basic task and destroy my context.

My biggest frustration right now is the seeming complete loss of background agent functionality. Permissions seem completely botched for background agents right now. When that happens, the foreground agent just takes over the task despite:

1. Me not wanting that for context management reasons

2. It burning tokens on an expensive model.

Literally a conversation that I just had:

* ME: "Have sonnet background agent do X"

* Opus: "Agent failed, I'll do it myself"

* Me: "No, have a background agent do it"

* Opus: Proceeds to do it in the foreground

* Flips keyboard

This has completely broken my workflows. I'm stuck waiting for Opus to monitor a basic task and destroy my context.

Claude is exceptionally better at long running agentic sessions.

I keep coming back to it because I can run it as a manager for the smaller tasks.

Those load shedding statements are infuriating. I’ve literal had sessions where we just get through planning a giant feature and I say “get started” with the response being “okay, we’ll pic up tomorrow “

This isn't the case. I basically did an entire business/project/product exploration before building the first feature.

Even after deleting everything from the first feature and going back to the checkpoint just before initial development, I can no longer get it to accomplish anything meaningful without my direct guidance.

Yep. I was doing voice based vibe-coding flawlessly in Jan/Feb.

I've basically stopped using it because I have to be so hands on now.

I'm going absolutely insane with this. Nearly all of my "agent engineering" effort is now figuring out how to keep Opus from YOLO'ing is own implementation of everything.

I've lost track of the number of times it's started a task by building it's own tools, I remind it that it has a tool for doing that exact task, then it proceeds to build it's own tools anyways.

This wasn't happening 2 months ago.

I've noticed this as well. I had some time off in late January/early February. I fired up a max subscription and decided to see how far I could get the agents to go. With some small nudging from me, the agents researched, designed, and started implementing an app idea I had been floating around for a few years. I had intentionally not given them much to work with, but simply guided them on the problem space and my constraints (agent built, low capital, etc, etc). They came up with an extremely compelling app. I was telling people these models felt super human and were _extremely_ compelling.

A month later, I literally cannot get them to iterate or improve on it. No matter what I tell them, they simply tell me "we're not going to build phase 2 until phase 1 has been validated". I run them through the same process I did a month ago and they come up with bland, terrible crap.

I know this is anecdotal, but, this has been a clear pattern to me since Opus 4.6 came out. I feel like I'm working with Sonnet again.

This is only an issue between 12pm and ~4pm ET. If I work at any other time of day, I never hit my usage limit.

Naw, it's not that. This is business-day usage for all of it.