Hacker News Reader

Comment by tripplyons | Hacker News Reader

For those who aren't aware, OpenAI has a very similar batch mode (50% discount if you wait up to 24 hours): https://platform.openai.com/docs/api-reference/batch

It's nice to see competition in this space. AI is getting cheaper and cheaper!

View on HN

DeepSeek has gone a bit different route - they give automatic 75% discount between UTC 16:30-00:30

https://api-docs.deepseek.com/quick_start/pricing

Yes, this seems to be a common capability - Anthropic and Mistral have something very similar as do resellers like AWS Bedrock.

I guess it lets them better utilise their hardware in quiet times throughout the day. It's interesting they all picked 50% discount.

Bedrock has a batch mode but only for claude 3.5 which is like one year old, which isn't very useful.

Inference throughout scales really well with larger batch sizes (at the cost of latency) due to rising arithmetic intensity and the fact that it's almost always memory BW limited.

50% is my personal threshold for a discount going from not worth it to worth it.

One open secret is that batch mode generations often take much less than 24 hours. I've done a lot of generations where I get my results within 5ish minutes.

It can depend a lot on the shape of your batch to my understanding. A small batch job can be tasked out a lot quicker than a large batch job waiting for just the right moment where capacity fits.

The latest price increases beg to differ

What price increases?

I guess the Gemini price increase

Ah, 2.5 flash non-thinking price was increased to match the price of 2.5 flash thinking.

No, 2.5 flash non-thinking was replaced with 2.5 flash lite, and 2.5 flash thinking had it's cost rebalanced (input price increased/output price decreased)

2.5 flash non-thinking doesn't exist anymore. People call it a price increase but it's just confusion about what Google did.

They try to frame it as such but 2.5 Flash Lite is not the same as 2.5 Flash without thinking. It's worse.

Only because Flash was mispriced to start with. It was set too cheap compared with its capabilities. They didn't raise the price of Pro.

Reply on news.ycombinator.com