DeepSeek V4 Flash 0731 Intelligence, Performance and Price Analysis
artificialanalysis.ai
artificialanalysis.ai
https://files.parasmittal.com/openai_aa_luna_dsflash.svg
1: https://openai.com/index/advancing-the-price-performance-fro...
High hopes for V4 Pro
So, are they planning to announce an optimized coding agent harness as well ? DSv4 flash is a fantastic model, and my daily driver. With reasonix or pi, I can code all day long and pay a few pennies for it. No token anxiety. Whereas the same model with fireworks/openrouter, with zdr thrown in, token costs ratchet up with no explanation. Likely that the model is subsidized for gathering usage data. I am waiting for the day I can run this locally.
"For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the max reasoning effort level with temperature = 1.0, top_p = 0.95."
I meant something I could download and run.
I am using openrouter with zdr guardrail which routes to any provider that supposedly doesnt train on user data. I also use fireworks (directly, not via openrouter) which is a provider promising zdr and has a bunch of open weights models. My issue is that these zdr providers dont transparently disclose caching/tokens etc and so they end up being far more expensive than directly using DS.
OpenRouter doesn’t list DeepSeek as ZDR combined with Caching. Thanks for the tip of Fireworks as Fireworks has Zero Data Retention by default. [1]
[0] https://openrouter.ai/docs/guides/features/zdr
[1] https://docs.fireworks.ai/guides/security_compliance/data_ha...
Does the file hosting actually cost peanuts when you do it yourself and the cloud has shattered my understanding of what it actually costs to deliver so much data?
CDN costs are pennies compared to inference and training though, HuggingFace will just get another 100 million and be set.
At the scale of Huggingface, that still amounts to a lot load of money. Significantly less than if you did the same in AWS, but still a lot
That said, they do have a deal with AWS to make the data available in AWS ip space. Maybe they got some cheap hosting out of that too
As long as you're smart on caching on the edges and deduplication/overlaying with your content (which structured data types like models, containers and repos typically are fit for) you can get remarkably far for less than you'd think.
The economics of serving models seem to be way more dodgy.
https://artificialanalysis.ai/models/deepseek-v4-flash?intel...
First, your direct comparison, Deepseek V4 Flash 0731 (max effort) $0.03 (rounded up) per task @ index 50.
OpenAI Luna:
* high effort $0.03 (rounded down) @ index 46
* xhigh effort $0.04 @ index 49
* max effort $0.07 @ index 51
So I would say a fair statement would be "OpenAI Luna between 2x and 3x the price of Deepseek Flash, what you get is 2 to 5 times faster inference"
The cheapest OpenAI model that beats it is OpenAI Luna (max effort) $0.07 @ index 51 (if you take the rounding out it summarizes to triple the price for similar performance), but still close to 3x faster.
And can SOMEONE please tell artificialanalysis that using dark blue for both Deepseek AND OpenAI is an especially unfortunate choice of colors, especially today?
For simple tasks, they're already saturated, and you'd prefer the faster model, so that you can have a realtime/interactive-ish experience.
Or to put it bluntly, it's cheaper if you don't value your time. That goes for smaller models in general -- need more handholding, more correcting -- but the Chinese ones are slower on top of that.
As for speed, Sol on Low is faster than Luna on most settings.
uhh openai is dark gray: `rgb(31, 31, 31)` and i'm pretty sure it always has been?
It’s so cost effective I can offer a generous free tier since my goal isn’t to make money with it.
One feature of the app is that all scripture is verified and what’s show to the user doesn’t come from the LLM at all and instead a trusted source.
I think exploring scripture this way does not alleviate you from struggling to learn and apply it. It hasn’t for me.
But there are ways to control and constrain the LLMs and what the user is presented with.
These are all top of mind for me and why I felt there could be a better option than asking ChatGPT directly.
What's difficult and doesn't have to be with philosophy/ spirituality is to find relevant bits off situation, theme etc.
This app does that very well, LLMs are good at entity recognition.
I think it depends, Catholics wouldn't be able to use this because the Magisterium is the ultimate authority on interpreting Scripture, so the personal interpretation isn't really needed. This is not to say that Catholics don't read the Bible, they are encouraged to do so since it deepens their faith
On the other hand, for Protestant it varies, the High Church denominations are closer to Catholics (though none of them accept the Magisterium) in terms of scripture interpretation, but the Low Church ones (like Baptists or Non-Denominational ) are more open to personal interpenetration.
Disclaimer: I'm a Catholic, so if I made a mistake here fellow Protestants, please correct me.
I'm on a team that develops a Bible study app, and we're all relatively content with how the basic models converse regarding scripture. Even as far back as GPT-4 was excellent. They occasionally have minor hallucinations (a dealbreaker for a production app), but they do an excellent job with theology and Bible scholarship, given reasonable guardrails.
I'll admit I'm coming from the perspective of "should we be implementing this?" It seems, on the surface, that a strong embedding-based verse retrieval covers the bases at a microfraction of the cost.
If you're interested, check out the development server where we're working on this. You navigate to the search (magnifying glass) and then hit "Meaning". Sorry for the confusing route; we're still deciding on back-end details and haven't focused on the front yet.
flash is suitable only for a toy apps, not for production environments :)
So it’s less about model choice and more about governance of scripture.
I will check out the link you sent for sure!
Anyways, impressive app! We haven't tackled such an ambitious project just for it being daunting.
https://api-docs.deepseek.com/quick_start/agent_integrations...
Deepseek v4 Pro prices with Opus 5 perf would be freaking unbelievable!!
This is probably a dream.
Plus a size you can genuinely run at home: Unsloth lossless Q8 at 162GB.
But it makes little to no sense as long as API prices are what they are. Except for maybe privacy reasons.
owning a few GPUs is a lot cheaper than supercars.
I don't use it for local inference so much. I use it to learn.
I also use it as my daily driving Aarch64 development system.
Aside it's also very cool what else can be done with unified GPU memory, once you realize you have it...
Going local has as opened up a world of use-cases I never would have entertained the idea of on metered/cloud usage. Privacy is a large part of it but, I also no longer think twice about whether to send a prompt or not based on the psychology of it costing money.
Cached input tokens on local inference are free, so I don’t care about running sessions up to 500k tokens and hundreds of turns (it’s rarely useful, but DSv4 remains surprisingly coherent up there)
maybe it is Fable level
If the full non-flash model follows up with the expected improvements, and at the price point they've been keeping, it puts the frontier labs in a tough position and it feels to me like like OpenAI is reaching deep into their pockets to try to head that off.
TFA link is a 404 though. I'm reading through https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 instead
It’s also so inefficient, when they release the full performance numbers it’s not going to be good.
One example, it takes about 3.6x more tokens to finish the same work as Gemini Flash 3.6.
All else being equal passing triple the amount of tokens through a model to solve the same problem makes it slower.
Doesn’t mean this model is bad, and it has to be considered how cheap it is, but it’s a factor.
Mind you, until the recent price cuts to Luna - Gemini 3.6 Flash wasn't even egregiously priced (but oh how things change in just 1 week).
That depends. Is it also more reliable?
If two books, one big one slim, prove the same thesis, what I would be interested in is the quality of the content, not the size. There can be a measure of efficiency in "have you really thought it through", but it is clearly complex - it requires measuring how solid the reasoning is.
It use the stock model, no new models requires.
Worth spend a few hours to try.
The DGX Spark requires a small hack to ignore the difference between sm120 vs sm121, but it does run on sm121.
You learn something every day. Today, it was the term "systolic array": A systolic array is a specialized grid of simple, interconnected processing units designed to execute parallel data operations—like matrix multiplication—by rhythmically passing data directly from cell to neighboring cell without writing intermediate results back to main memory.
The term comes from the biological word systole (the contraction of the heart pumping blood through the body). In a systolic array, data "pulses" through a network of processing elements on every clock cycle, driven by a global clock beat.
As for sibling comments, huawei ascend is more of an NPU-style architecture where you can easily have much bigger MMAs as primitive. But you usually don't anyways for many reasons.
The subsidized subscription model won't last, API pricing "feels" closer to a true sustainable business model.
I've used it in some open source code though, and loved how fast it was.
My mind is changing on how valuable my code actually is though... it's the complete picture, how it's put together, the design, the UI, the attention to detail that's the real value.
Claude is very spendy.
Of course you can get more bang for your buck by being more deliberate. But that's equally true with US frontier models. You can optimize your work by choosing between Opus, Fable, Sonnet, Sol, Luna and Terra for each task. Some people seem to prefer to let Opus code and Sol review, for example. And then there is the whole debate whether $current_version is actually better (Some people stay on Claude 4.8 because they dislike how 5.0 is sometimes doing stupid things, just as many opted out of dynamic reasoning when they still could)
I don't find it suitable for everything, but there's some tasks it crushes for what feels like almost free.
The official release of the DeepSeek-V4-Flash API is now in public beta. The API calling method remains unchanged — simply set the model name to deepseek-v4-flash to use the latest version.
The specific agent is focused on getting precise and on point answers about a codebase.
The starting point was nowhere near. E.g. asked why was X implemented in a certain way it would give bogus answers when the real answer was that there was no reason at all.
The benchmark included more than 50 questions or different difficulty.
But when the agent was improved in its prompt and rooting it was impossible to have it perform worse than closed source sota.
Just to say that the quality of the harness is as important as agents intelligence.
But a REALLY good pelican on reasoning mode high (via OpenRouter): https://static.simonwillison.net/static/2026/deepseek-flash-...
It's good.
> who is the customer
If no one else, then definitely those that can only budget $1/mo to $5/mo. Probably 100s of millions, if not billions, of the Global South.
[0] Free access to DeepSeek v4 Flash is indeed available from providers like OpenRouter, OpenCode, and Freebuff (to name a few), but no ZDR.
DS has a lot more than mere 80% of Claude's capability, and the price is a lot less than 20%.
Am I just using it on tasks that makes it go on forever vs these benchmarks that are short&sweet, or something like that? I've been throwing bunch of identical prompts at different models at the same time, and when comparing hy3 and K3 I've never once had K3 reason less than hy3, as just one anecdotal data point.
Similar price? Doesn't make sense. Maybe they meant power, capability or speed?
Why do the cache hit rates seem to vary so much between harnesses?
I use pi, which is very minimalist, and I get a hit rate of ~99%. Paying like $1 a day for Flash. Yet, the hit rate mentioned on OpenRouter is only ~79%.
BTW, this is one of the things that I really like about Pi. It’s very simple and thus very predictable.
What am I missing?
[0] https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
create a plan with SOTA, execute with this.
Or a benchmark to benchmark benchmarks?
benchmark website benchmark is indeed a benchmark that benchmarks websites with benchmarks (but it can be shown outside websites as well, it's not picky)
...touching myself RN.
The ban on these open models is coming within weeks, if not days. As usual, the excuse will be "national security".
For example: no government contract to any company who uses even one vendor in it's entire chain of dependencies, who uses such open models.
They can extend this further by laying more conditions, such as: any company dealing in this-this field can only use models "officially" approved as "safe". Rest you can guess how easy it would be to get that "safe" rating for such open models.
I'm not sure the outcome would be beneficial for the US as a whole here. But perhaps that is not their priority.
- the US strictly regulate cryptography https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...
- some prime numbers are considered illegal https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes
I don’t think banning open weight is that far fetch compared to those
People on HN downvote objectively correct information because they don't like it 24/7. There's a reason the creator of Zig left and gave the computer version of a middle finger on the way out to HN!
commenting about voting is also something the HN guidelines warns against:
> Please don't comment about the voting on comments. It never does any good, and it makes boring reading.
Since when have hackernews started to become toxic like stackoverflow used to be?
I claim the CCP will wise up within 2 years, possibly much much sooner, and ban their own companies from open sourcing to prevent the Americans from acquiring the capabilities.
Despite all the nonsense claims of China distilling US models, the reality is that the Americans absolutely do distill these free Chinese models, and distillation when full logprobs are available (i.e. you have access to the weights of the model) is an order of magnitude better than when you don't.
Yes, Chinese open weight models in the short term harm US closed source model providers bottom line. In the slightly longer term, "showing your hand" and publishing both the architecture innovations and the models weights will be too dangerous for the CCP to allow. This is triply true if they can release a model that beats the Americans on most benchmarks.
I've already warned investors that this is probably the closest open weight models will ever get to closed access.
https://www.businessinsider.com/xi-jinping-open-source-ai-us...
They can't compete, they have bills. By the end of the year, if they can't react, it's game over.
Maybe US clients could be a little patriotic here, but money is money. They won't give them free money forever.
All the good talent moved to other countries due to this.
The fact that China could catch up in what, two years, shows that this is a commodity.
But if models will be commodities, maybe that won't be such a terrible thing longterm?
Retaining world-class talent and home-growing industries is no small feat. In Europe, we have been struggling with this, and I am not even entirely sure why.
But imagine if Chinese labs would lose. Then there will be no open models, and the prices for closed models would be raised to the maximum.
Sanctions and trade policies that hurt ordinary people? Cultural dominance through Hollywood (Holy Wood, again with the sick ruling elite and their witchcraft crap), social media, brands, the poisoning of food and water on global scale?
Etc? Etc?
Most people don't hate "Americans" or their companies, they are just against your ruling elite. Live one day outside the united states and understand how the planet works for the rest of mankind and you might start to grasp on reality.
And yeah, China will catch up with hardware pretty damn soon. Jesus, you guys bought your radios from Japan not 50 years ago. Sorry, can't dominate the world's economy forever. Never happened in history.
No matter how much black magic, dark rituals, sick crap and sacrifices people on the top throws at the wall.
We're humans, after all.
So yeah, it is not against America itself or American companies, but against the ruling class. So naturally , people get excited about China entering into the match!
3...2...1... Fight!
Most everyone wants to come to America for the opportunity. Think an immigrant in China has the same economic opportunity as in America? Any other country come to mind?
Sure, if you spend most your time on the internet in various news forums its doom and gloom. Its what sells.
It needn't be competitive - everybody can succeed more tomorrow than they are today. You win. I win. We win.
Daily reminder that improving your samplers from the garbage default top_p/top_k to min_p or subsequent methods dramatically improves the performance of these models, and makes most quantities like measured "verbosity" and subsequent calculations of "intelligence per token" meaningless
Daily reminder that no one, including within academic AI research, AI engineers, normies, etc takes LLM sampling seriously enough.
But you already know that.
If you poke it just a few times, however, you get to the point where it will eventually say (paraphrasing) that basically only Israel, the US state department, and the ICJ say it's not a genocide.
That is to say that it's framing it as some sort of tricky complex question when it's not. And when interrogated, it basically admits that the only people who dispute it are Israel and it's supporters.
The majority of the world is religious - doesn’t mean the debate on religion isn’t a complex question.
The majority of the world approved of slavery historically.
The majority of countries have ethnically cleansed their Jews, many of them in living memory.
When interrogated you will find that the only ones asserting the war in Gaza is a genocide are people who were anti-Israel anyway.
I'm always suspicious that's the case given how mentions of gaza seem to bring out brand new accounts who only talk about Israel.
[1] https://quincyinst.org/research/the-eighth-front-inside-isra...
We are on a thread discussing Chinese models. Every discussion on here that’s negative about China or its models suddenly gets derailed via whataboutism to Israel/Gaza. A very convenient distraction.
And yes, we were discussing censorship of models which, as I pointed out, doesn't seem like ChatGPT is directly censoring data though it does appear to be manipulating it. Pretty on topic.
It was you, brand new account hiding your past opinions, who came in here to make this solely about Israel.
Yeah, I think you are likely a foreign agent. Prove me wrong and post from an established account.
Under no definition of genocide (as defined either going back to the middle 20th century or as that term is defined legally today) is Israel committing genocide. Simply put, there is no evidence that Israel is committing "specific acts with the intent to destroy, in whole or in part, a national, ethnical, racial, or religious group." Intent is, as with almost everything in the criminal law, everything here. If you want to subdivide Muslimes in a way that results in, say, Hamas and Hezbollah being their own religious groups then you could mount the argument on that basis, but those are political designations and so good luck with that. And of course neither of those groups encompass the entirety of the national polities they claim to represent - just look at how hated Hezbollah is in Lebanon. They are being ostracised and even the Lebanese leaders want nothing to do with them at this point.
People like you degrade actual acts of genocide such as what happened with the Nazis or such as what has happened several times in SE Asia and Africa over the past 50 years. A country acting in self-defense is not genocide no matter how much it might hurt your sensibilities. Genocide is, as is obvious to most people, extremely difficult to prove. See Armenia for how fraught this debate is. But acts of brutality in a conflict or an extreme imbalance of military power between belligerents alone do not mean that genocide is occurring.
Good day.
Any normal user is much more likely to ask questions to which the Anthropic and OpenAI models do not answer, than to ask questions about the modern Chinese history, to which a Chinese LLM will not answer.
This has been debunked here on HN so many times. The Chinese open models do answer the hairy Chinese political questions, and the raw APIs pass-through the response. Now, the answer might be blocked by the agent who's calling the API, specially if you are using a Chinese endpoint instead of the RoW (i.e. Singapore) endpoint.
That's the reason why you should always prefer a open agent/harness as well instead of using the provider's.
I have to admit it rarely comes up in the coding tasks I usually give to LLMs.