"Cache reads now cost 75% less, or $0.25 per million tokens." For me, at a typical 95% cache hit rate, I think my optimal context window size before autocompaction goes from ~200K to ~400K tokens. Great for longer horizon tasks.
i try to look through the docs, but i didn't find where they said its only for API
is it in the system card?
really hope not, that change the only positive part in this release