Hacker News Reader

Comment by tikkun | Hacker News Reader

Correct

That would make each API call cost at least $3 ($3 is price per million input tokens). And if you have a 10 message interaction you are looking at $30+ for the interaction. Is that what you would expect?

Maybe they're summarizing/processing the documents in a specific format instead of chatting? If they needed chat, might be easier to build using RAG?

This might be when it's better to not use the API and just pay for the flat-rate subscription.

Gemini 1.5 Pro charges $0.35/million tokens up to the first million tokens or $0.70/million tokens for prompts longer than one million tokens, and it supports a multi-million token context window.

Substantially cheaper than $3/million, but I guess Anthropic’s prices are higher.

It is also much worse.

Is it, though? In my limited tests, Gemini 1.5 Pro (through the API) is very good at tasks involving long context comprehension.

Google's user-facing implementations of Gemini are pretty consistently bad when I try them out, so I understand why people might have a bad impression about the underlying Gemini models.

You're looking at the pricing for Gemini 1.5 Flash. Pro is $3.50 for <128k tokens, else $7.

Ah... oops. For some reason, that page isn't rendering properly on my browser: https://imgur.com/a/XLFBPMI

When I glanced at the pricing earlier, I didn't notice there was a dropdown at all.

So do it locally after predigesting the book, so that you have the entire KV-cache for it.

Then load that KV-cache and add your prompt.

Reply on news.ycombinator.com