Claude for Enterprise

500k context is enormous, but at 3.5 Sonnet prices, also pretty expensive: $1.50 for one query maxing that out. However, if you use their new prompt caching setup and cache most of the big prompt and are hitting it frequently enough, it's more like 15 cents per query. That's still a lot, but it could produce very valuable results, and sticking the whole thing into the prompt instead of trying to narrow it down first with traditional search lets you skip all the pain of indexing and vector DBs and whatever. This makes prototyping much easier because you can get good results right away by jamming your whole corpus into the prompt, and only then decide whether optimization is worth it.

Woah. This is huge! I thought their biggest models only offered 200k context windows?

Is this the first time we've seen an LLM provider put their newest/best models behind a SaaS wall before they make it available through API?