Intel D270 GPU AI Accelerator–Up to 480 GB LPDDR5X | Hacker News Reader

Intel D270 GPU AI Accelerator–Up to 480 GB LPDDR5X

intel.com

These are going to be so awesome to buy in 5-8 years!

Right now I’m running 2x 3090 and 128 gb ram on 3.8 flash-next180b (the quant is 97gb) and it hums at120t/s. It’s really a surprisingly good model for general tasks, and I’m soon to see how it does with more agentic kit. For agentic work it does slow down quite a bit when the context fills up… but what do you want for running a 180b model on consumer hardware?

Right now I have it set up as an administrator for our asterisk PBX and radius server, and it does higher level judgment on our cctv footage for notifications to security. It’s been really good so far. Agents are the new UI, everything just works over SSH now.

> These are going to be so awesome to buy in 5-8 years!

Literally my first thought!

> (...) it hums at120t/s. It’s really a surprisingly good model for general tasks, and I’m soon to see how it does with more agentic kit.

This is exactly why I interpret the aggressive price cutting by OpenAI on their Luna model line as a clear sign they are scared shitless of local models, and how they can trigger a "the king has no clothes" moment for AI companies like OpenAI.

At cost of power at $.45kwh here and dual 3090 setup easily approaching 1kw per hour at 120 tok/sec, latest Luna is cheaper per million tokens.

Your electrical costs are likely less than mine though

Yeah electric costs are nil on the microgrid but capacity is limited.

I'm working off a 128gb Strix system on mostly hydro and it's worth it for me. Qwen 3.8 27B and Flash Next, Gemma 4 and MOE models. It Keeps my thinking and methods local while moving at a speed I can actually read and think about.

It can be annoying when you're running something long and realize there is an issue. But hey, I'm not wasting paid tokens. Reset and try again. I've probably thought of a better way by then too.

A dual 3090 system shouldn't go above 600W wall power while running inference, even less with some weights in RAM as you need to do with "only" 48GB VRAM and this model. Unless you are running water cooling, the cards are likely sitting at 200-250W power limit each anyway so they don't run too hot. This is considering a consumer tier CPU and mainboard with about 90W TDP, not a dual CPU or something like a threadripper that can easily push the power draw to 1kw if it runs the calculations on the RAM weights. My dual 3090 setup usually drew about 200W-300W average with LLM workloads according to my HA historical data.

> as a clear sign they are scared shitless of local models,

The number of people with hardware to run a model like that is very small.

It's going to be many years before hardware to run medium size local models is widely available. Running mid-size local models doesn't even register as a factor in pricing decisions right now.

It's such a small part of the market and will be for many years, until RAM and other hardware prices come down.

> The number of people with hardware to run a model like that is very small.

That's not the problem anyone is facing, and analysis anyone is doing. At all.

The problem is the huge AI-related budget that companies who dove head first onto AI are facing.

Therefore the problem everyone is faced with boils down to whether they keep paying the likes of OpenAI a de-facto business tax, or they replace that tax with a one-off CapEx invoice followed by a utilities bill.

And you don't need a frontier model to do most things. In fact, the biggest challenge in using local models in agentic coding is the models supporting these agent-based tools.

> It's going to be many years before hardware to run medium size local models is widely available.

Nonsense. Speaking for myself, my employer already introduced a hard cap on AI credits while rolling out hardware support for local models, and even went to the extents of rolling out workstations with 64GB of RAM as a baseline. Consequently, we are using the likes of gemma4 for a fair share of everyday tasks. This has a transformative effect on your workflows because suddenly you don't ration tokens, and agent orchestration doesn't become a cash guzzler. So far the only constraints are tool support and token throughput, but that's not a concern.

I wonder how this compares to a hard(-ish) cap on Sonnet-and-better models, and unlimited access to Haiku. That’s one of the policies we’re considering.

> The number of people with hardware to run a model like that is very small.

True, but the number of people who pay the non-subsidized API price is also very small, and it's all in big companies who would have no issue buying or renting hardware for that purpose.

Many years is only long in relation to the AI hype cycle. In terms of a human lifespan it is not that much, maybe 10 years at most and that's pessimistic. We technically live in that future already it is just very unevenly distributed.

A human lifespan doesn't have that many decades in it either...

Ram prices suggest there is going to be a mighty glut of the stuff in a couple of years. We may easily see 32->64->128 GB system RAM off the back of that and CPUs seem to be moving to unifying VRAM and main memory.

I don't think running a ~200B parameter model is that out of reach, it might be usual by the mid 2030s for people in the tech world. Factor in the crazy improvements in quality given a number of parameters too.

> The number of people with hardware to run a model like that is very small.

But there are a lot of companies who do and rent it out as a service.

Idk, I’m running 180b on $2500 of hardware at 120 T/S (single user of course) but I bought it before the clown show. Now it’s probably more like $10k lol

I expect I’ll be able to run 1/2 terabyte models in 5 years for a couple $K in hardware. Inference is going to be a lot like Bitcoin mining. Only the peak of the power/performance curve is going to be profitable. Which means a glut of surplus hardware that can’t make money but makes tons of tokens.

but the number of small to medium sized companies which really want to run (some of) the AI usage under their control and are willing to drop some 100ks on it aren't that few (especial for non US companies given the state the US is in)

> as a clear sign they are scared shitless of local models

Partially agree and partially disagree. Qwen 3.8 next (120b + 50b ngrams) is at the upper end of a local model, and serving it at good speeds (esp for pp) is not cheap. So no, oAI are not particularly scared that everyone will suddenly start using local models of this size / quality.

I do agree that having these models in the open is great pressure on the price point of "small" API models (like luna, haiku, etc). And that's good news for us consumers and for the field in general - more competition is always better.

And on the cost estimate, the true power of local models is the guarantee of running them on prem w/o any data going out. Price wise it is still cheaper to go API, for small teams.

The "upper end" of local models as of today would be something like MiMo 2.6 Pro or Kimi K3, either of which is significantly smarter than Qwen Flash-Next. Roughly comparable in smarts to the old GPT Terra series, not quite as capable as Sonnet. But yes, running these models locally at realtime-usable speeds and without heavy quantization would be even harder. They can be quite tolerable for plain Q&A chat and great for running multiple unattended workloads on a 24/7 basis, but that may be a bit of a niche.

Yeah, I should have worded that better. I'd say it's at the upper level of what can be ran on plain consumer hw (like a laptop, mac, etc). For ~k3 you'd realistically need dedicated hw anyways. Or a cluster of consumer hw, and that comes with its own problems and drawbacks (esp in pp which matters a lot for agentic stuff).

Kimi K3 has been demonstrated on a top-of-the-line laptop Mac running at several seconds per token (it has to fetch essentially all experts from SSD, since IIRC the dense part of the model takes around ~105GB on its own!). That's impressive enough if you only care about pure Q&A/chat or pure unattended tasks, but it's not what one would expect for agentic work. You'd need quite a bit of additional RAM to have good caching for the most common experts, and even then that would only help so much. MiMo Pro would be a roughly similar story.

> Or a cluster of consumer hw, and that comes with its own problems and drawbacks (esp in pp which matters a lot for agentic stuff).

Prefill actually ought to work well on a cluster AIUI, since it can leverage both pipeline parallelism and batching quite effectively.

Viewed differently, they make it much cheaper to distill their models.

Are you running club-3090 or something else?

Flash-Next runs on Strata, version 0.1.42, an inference engine built specifically for Qwen3.8-flash-next

How do you do the cctv analysis ?

It’s combined with UniFi I assume, what else has good API, Hooks, or consumer level integration to play around with with.

Frigate is an excellent open alternative.

"These are going to be so awesome to buy in 5-8 years!"

Can't wait. Anyone has a idea of what the price now would be?

The closest I can think of is AMDs MI350P it has 144gbs of hbm3.

The press says amds price is 30-40 thousand dollars. On eBay there is exactly 1 going for close to 90 which is crazy because for that amount you could buy a MI350X and all the related hardware to actually use it and it has double the vram.

So I would peg it at 40-80 thousand dollars given it's Intel, and it's lpddr5 not hbm and not gddr7 so performance is likely to be lackluster.

> Agents are the new UI

Poorly-defined and ever-changing with occasional cases where a button does nothing, does something else, or deletes the whole prod database?

Idk. For a prototype they might be, but the whole reason we have this software thing going on is because we want some level of determinism.

Lol. I’m totally with you on agentic UX development. It’s frequently terrible, and you can tell which companies are telling everyone to 10x with AI. But the problem is testing, not the coding. My first stab at a UX render usually sucks too, it’s the iteration that brings perfection. It’s the same for agentic development.

What I meant though wasn’t agents building UX. I meant that in our case at least, the agent is the UX. “put a new extension 117 for Sam at the port next to Jane’s, and have it ring through to his cell when he’s not in the office. Leave the old 117 as is but only ring where Sam is, if that is known” basically, agents just do everything with SSH, and you can talk to them instead of poking around with a mouse. We don’t build ux for internal systems anymore. Local inference is strong enough to be reliable now with the correct tools and policies, and for 10-20k in hardware you are self contained. (Obviously $more if you are going to have a lot of people using it at the same time)

The “one wierd trick” is to build testing jugs for everything. Before a configuration is deployed, it’s tested against a toy instance, all existing functions are tested in all common cases, new functions verified, inputs fuzzed, adversarily abused, and stress tested against dummy loads. If its code, it gets examined against development policy, self containment, explicit interface walls, race conditions, code standards, complexity, factorability, and often A/B testing of different architectural choices. Except cryptography and a couple of other things, we generally prohibit the use of any libraries, it’s just not needed and the code never passes our tests anyway.

People are just YOLOing it and assuming that if it basically functions and looks right then it must be perfect. You’d have the same luck if you told your UX guy “test only the obvious, and if it looks about right at first glance, merge it. Don’t do extensive testing.”

The real thing with AI is that now you can afford to do a ridiculous amount of automated testing, setting up test fixtures, hardware simulators, mannequins of all the external interfaces, fuzzing, etc. Our testing code is 2-3x our product code, and we are shipping 10% of the bugs we shipped with humans, and it’s still 1/4 the cost and 4-10x the speed.

People shipping all the crap that is everywhere now have bad process engineering. Plain and simple. If you try to build software with AI like you used to with people, you’re going to have a bad time. The pipeline is very different.

This feels like a detached backpedaling pivot to save face tbh.

But anyway.

I'm sure there are people that use the tech wrong. That can be said about any tech in existence.

Most people, actually. Remember, friend - ~half of the population has two digits in their IQ, by definition.

If we aren't in that half, it's by sheer luck, and nothing else.

kudos. Precisely this. People are doing agentic design backwards very often: YOLO then cry. But, if first you write skills, specs, set up orchestration, architecture and flows, including review layer, discrimination layer and trajectory management layer, then you task the agents with creation of verifiable testing and static analysis rig, based on the golden samples of high-quality manually engineered code, and only then you tell then to write conformant code.

Agents (and LLMs) are inherently non-deterministic. In coding practice, this translates to garbage code in the long run. The solution is simple - make the process as deterministic, verifiable and rigid as possible. Suddenly, you get all the bonuses of agentic flows (insane initial speed, trivial cost, express iterations etc.) with none of the usual problems... and suddenly, humans have hard time matching the quality of the final artifact with hand-written code, because we suck at testing, fuzzing and debugging - because we're basically single-threaded and overfocused. Agents, OTOH, excel at it, and context management is also trivial compared to humans.

> My first stab at a UX render usually sucks too, it’s the iteration that brings perfection

With AI, your iteration cycle gets 10x/100x quicker. For the same developer effort, you can effectively try out 10s of ideas the same period of time.

Iteration also has a human component. When a person works hard and writes the code, they are personally invested in its success. So iteration, or, chucking out someone’s hard work is psychologically damaging and causes human conflicts in teams. If AI wrote it, and AI deleted it, no-one gets upset. All that matters are good ideas. Reducing the human cost of iteration is just as important as the time saving.

And, now you have freed up implementation time, you now have way more time for testing, that thing that everyone wished they had more time to do, with a super intelligent 100x developer (ai) as your coding partner, and a 100x lever for automating regression testing. Bad UX has zero excuses.

It works just fine. I've been using it for over three months myself once I tuned my MCP server. I run a 25-30 agent fleet with zero issues, I control it with Codex or Claude Code.

This isn't something you'd be able to manage with a normal UI since every agent has its own at of settings. My agents come in sets, I usually change a set or multiple sets, 10-15 settings per agent. And my harness hasn't made a single mistake once I worked out all the kinks from MCP.

> Agents are the new UI

Funny as I'm using LLMs now to write UIs.

With an impressive 1.5TB/s RAM bandwidth you'd think they'd advertise that more.

Marketing department asleep at the wheel.

Source? Extremely impressive if true

This is a very low number for a server GPU. My V100 from 2017 is 0.9 TB/s.

A single B300 has almost 8 TB/s.

AMD MI300x from 2023 is 5.3 TB/s

Edit: my bad you are saying that this is an impressive number precisely because it is more or less normal RAM, not VRAM

VRAM isn't a type of memory, it's just RAM used for the GPU. So this LPDDR5 is VRAM, just like it can be with my Strix Halo machine.

Very true, though traditionally it did refer to GDDR.

That's only partially true - the GDDRX you'd find on say a 3090 is physically very different than what you'd find on a strix halo or mac or vanilla ram. e.g. runs 4 voltage levels not 2 so that it can transmit twice the info per cycle

Well, what they don't tell you about GDDR6X is that it's 4 voltage levels but it never goes from min to max or vice versa in a single transition (PAM4 MTA). It transfers 1.75 bits per cycle.

And then GDDR7 backed off to PAM3 and sends 1.5 bits per cycle.

There's a good chance [LP]DDR will use the same techniques once it gets into the same speed range.

Yes the cells are pretty much the same as they would be with hbm as well all the magic happens at the bus and the signaling, there is both a reason hbm and gddr have higher bandwidths and tradeoffs.

DDR has better latency hence the preference in CPU workloads where lower latency is more important due to variable workloads.

Its bandwidth is around that of 32 DDR5 DIMMS (50ish GB/s). These boards should have built in NICs but they don't because Intel also wants to sell more servers.

And a 5090 has 1.79 TB/s.

So yeah on its own not very impressive.

The large amount of memory means you can serve larger models, or less quantized, but that also means they're going to be slower due to the low bandwidth.

https://www.techpowerup.com/gpu-specs/geforce-rtx-5090.c4216

Yeah but I think this accelerator is more meant for a small scale deployment. This could probably serve a medium moe llm model at reasonable tk/s plus you could in addition to the llm also run a couple of other models on it in parallel like an embedding, reranker, STT, TTS, decision, image, video, … if you deploy this for a small business they might make use of all of these models but not all at the same time and not with 100s of users but this card could bring a serious broad range of AI capabilities for them.

These things will fly off the shelves with 0 marketing

> Marketing department asleep at the wheel.

The marketing department doesn't have to lift a finger in this market. They could quietly drop it for sale without a single announcement and it would be backordered for a year.

Companies nowadays care more about valuations and stock spikes than sales. Curiously D270 announcement was followed by 3% INTC drop.

everybody is long everything that may touch sand in the next couple years, no buyers left.

> Marketing department asleep at the wheel.

They tasked an AI agent to it, like everyone else.

Is this D for Druid, like the B series was for Battlemage/A for Alchemist? Don't think we had C (Celestial) at any point.

Yes. They announced they were skipping C.

To be honest these ubiquitous AI hardware news look to me more and more like Maybach commercials.

Not targeting me

1.5 TB/s 1.3 PFLOPS with no fast multi-GPU, it's a consumer card that pretends to be a server card.

It would work great on Qwen3.8-Flash-Next or GLM-5.3 as single user decode only hits a few experts, anything with concurrent users and you'd be sitting for a while.

That's the point I believe fast local 2-4 user service for cheap on very large models.

You have to pretend it's for servers to keep investors happy, it would be banks and similar with in house servers that have to run local large models buying these for experiments and cybersecurity. Likely.

Reminds me of the early 2000s era where we'd fight in my household if too many people used the internet at the same time, can see that happening here if it's not 2 to 4 users but 10.

Leather jacket Jensen must be sweating loads now because every big company and their mums are making AI chips

>sweating loads

Interesting choice of words

wdym?

At at the very least, trying to make them

Like cool hardware but in the current situation also quite insulting - totally unaffordable for an individual & one more thing to gobble up useful memory + make the computing hardware affordability crisis worse.

Fun to see the LLMism on the product page: "A, not B"

Productivity!

Slop and placeholder text on the page, including this one. They should have reviewed their webpage.

"What software frameworks are relevant to Intel GPU D270?

Intel GPU D270 messaging should reference familiar AI software and serving frameworks such as PyTorch, vLLM, SGLang, and OpenAI Triton, where approved. This helps developers and platform teams understand how the product fits into existing AI workflows."

Noticed this as well! Was looking for details on the software support and this FAQ entry didn't exactly instill confidence...

I like this. It means a human was involved.

Can I actually buy this? Should I prefer it over Blackwell?

> Can I actually buy this?

No. Certainly not now, maybe not ever. That is the least consumer-focused product page I've seen today.

Intel ai gpu division is so aggressively anti “small” consumer. I think it has something to do with low volume and a small sales/distribution network. The data center cards I have from amd and nvidia are grey market at best. I don’t think intel moved enough units to establish alternate channels.

Serious question, would they bother?

You can work with 10 players that will buy all of them with minimal effort.

What I don't understand is: when you only plan for your product to have 10 customers, why even make a marketing page like this for it? At that point it's essentially the hardware equivalent of consultingware, something you sell to your existing contacts through outbound calls and by shoving ads through the B2B equivalent of a post-consumer sales channel (i.e. proactive outreach by your org's internal champion for that customer, to said customer.)

Because they want the hype to make the stock go up.

In fairness, they spent 5 cents generating it with Claude.

How so? You can walk into Micro Center and buy a B70 right now. Intel is the only one to offer license-free SR-IOV too.

Thats not an AI GPU its a consumer product.

You can tell by the shape of the board that it's meant to go into a server chassis behind a stack of 80mm blower fans. It's not meant for consumers, but you will be able to buy it used for $200 off ebay in 7 years to put in your home lab.

Wonder if today's ram will go up price by then...

It is going up one way or another- the price of money will go down - @5% inflation over 8 years the value of today’s dollar will be 67.7c .

Wonder if there will be consumer video drivers for these GPUs once the AI hardware shortage is over. Hope to buy a gaming rig one day that doesn't cost a fortune.

You can vibe-code the drivers. Someone just did that with the Nvidia drivers for Intel Macs, bless their soul.

I dont think it has a video out

I remember back in the day of AGP cards around 2002, you could use a 3d card on linux to render the grafix but use a different GPU for the actual output. I used XFree86 on linux with a voodoo banshee (or maybe was a riva tnt2) and a matrox mga200 for its sync-on-green support with my monitor to play games.

I know modern card architecture doesn't support this out of the box, but what is the actuall problem of forwarding a once-rendered frame to another card for output? Sureley DMA transfers PCIe->RAM->PCIe nowaday are fast enough to saturate 60fps at 4K?

I had that setup working back in the day with a Voodoo1 which had the pass-trough construction to take over full screen: only on Linux you could read the completed frames and render them in an X window. It was dog slow though.

While transfers have improved a lot the amount data needed for 4K60 with HDR is about 18Gbps. There's no way that doesn't have an impact.

the voodoo1 worked differently as you looped through the video input. The voodoo banshee i mentioned was a separate 2d/3d card (it actually was 3DFx first true, combined 2d/3d card - not an accelerator).

I know. Voodoo 1 and 2 had the ugly VGA passthrough hack. But the principle of reading VRAM from one card and copying it to another would remain the same I think.

We cooked up a setup where we had a PCI video card for X, and a separate Voodoo 1 or 2 and monitor for 3D (Mesa) rendering. I don't remember the details any more, but it was nice to not take out the desktop when someone borked up their OGL homework.

Trivial to do this on Linux. I run my desktop on the on-chip Ryzen 9950X iGPU and I offload games to my nVidia dGPU.

I had a crazier setup in the past with the Intel i7-8809G APU: it had an AMD iGPU for desktop connected to the display, nVidia eGPU for games and the Intel iGPU for video encoding.

Of course there is additional latency due to frame copying, but I can’t notice it in practice.

https://wiki.archlinux.org/title/PRIME

> We want to render applications on the more powerful card and send the result to the card which has display connected.

in latest gamersnexus video https://www.youtube.com/watch?v=e4c2FSSaCsw Wendell from Level1 Techs was showing some older Amazon dedicated server GPUs with no video output that got hacked community driver making it work like laptop dgpu copying rendered video across PCIe.

In 2 years once it fails and Intel shuts it down completely you mean :)

As has been claimed every year for the last 30, at least. Much like predicting the end of the world, one day, if you keep at it, you'll actually be right.

Given the rumored numbers are all true, this could be a huge contender to M5 Ultra(or M7 Ultra if we consider possible delays in the schedule?). Low-batch single-user serving is usually bottlenecked by `E(sum over (memory bandwidth by tier) * (memory size by tier))`.

edit: typo

Looks like Intel is choosing to title its new crescent island cards with the D prefix despite being Crescent Island.

Kind of disappointed that Intel is (probably) not going to release consumer class GPUs. Intel's driver stack was seriously getting mature—both on Linux and Windows. It's always nice to have a third option available.

At least that driver stack will continue to mature?

On Linux mainly, and almost strictly for compute workloads (not much need to optimize for 3D workloads given the customer base).

They're still using Xe based iGPUs, so 3D still matters.

For now. A limited run of Celestials until 2028 when Serpent Lake switches to Nvidia tiles, and a bit longer for the LTS combo (AMD style) IO+GPU die in lower end/embedded products.

Druid is still dead, consumer DGPUs are still, and workstation DGPUs also seems to be dead, and most of the Xe driver team for Windows was laid off, as well as most of the Xe chip design team.

Its wild to me that intel isnt trying significantly harder on gpus

Theres so much money to be made

Why make consumer margins on consumer GPUs when they could be making enterprise margins on enterprise GPUs?

I mean, why not both

"Both" requires more engineering staff on payroll, more wafers to fab/dice/package/test, more boards to manufacture, more inventory to manage, among other things. If their math says it isn't worth it (granted that they don't always get it right), then they don't do it.

"Both" requires allocating limited die capacity to products that have over an order of magnitude less margin.

> "Both" requires more engineering staff on payroll, more wafers to fab/dice/package/test, more boards to manufacture, more inventory to manage, among other things.

Aren't you describing how a business operates?

I mean, doesn't Intel already hire people for those tasks? Unless you are worried about a specific bottleneck, increasing headcount and production output is what businesses do.

One word: Volume.

You can make a nice profit off of the enterprise space, but the big winners are the guys who put the work into servicing the consumer market.

I believe that their fabs are running at full capacity already at this point. If they were to announce new GPUs, where would they make them?

Agreed. I was really rooting for them to deliver that third iteration to consumers, and retain them as a competitive option in the low to mid range. Pretty sad to see.

I have to agree, and it's sad

hmm...wonder where this will land in market segments.

Seems like neither a consumer product nor server. Maybe if they halve the ram and price it reasonably people with desktops will buy it for MoE like prosumer homelab AI enthusiast type situations?

Can intel gift me one?

The cost of a 480gb ram video card seems preposterous to be even marketing at this point.

Intel is going to make a card that costs $1 million dollars.. this is like industrial newsletters bleeding into a public forum

> Commercial Scale Pricing: In the industrial data center sector, battery storage systems are priced per kilowatt-hour (kWh) or megawatt-hour (MWh). Industrial battery storage installations typically cost $300,000 to $450,000+ per MWh of capacity depending on software integration, cooling systems, and structural framing.

Let's all get excited

But can it run Crysis?

Is there any indicator whatsoever of what this will cost per unit?

How much is it expected to cost?

Several hobos worth of organs

How many fps in Cyberpunk 2077?

Can It Run Crysis?

What about Windows Vista with Aero enabled?

That brain dance where you end up at the Skav's place in a bathtub

Seems incredibly little compute for this. Is it meant to be MoE specific?

They originally said it was going to be 120GB but that partners could ship bigger capacities. So much for that!

I do hope this has a price point that makes it somewhat competitive with a Mac! Consumers have had nearly no choices available for mem capacity, and 20 channels of lpddr5 is one way to get that capacity! Go Intel!

Both this GPU and the Mac are constrained by a lack of GPU compute. There's basically no rack-scale GPUs with 480gb on-die that have a sub-400W compute element. It's going to be incredibly prefill-constrained on denser model inference just like the Apple Silicon iGPUs are.

It seems like there’s a market out there that is just happy to run the models locally, even if generation is slow.

This GPU has 1.3 PFLOPS of FP8 compute performance and 1.5 TB/s of memory bandwidth: https://videocardz.com/newz/intel-gpu-d270-is-xe3p-crescent-...

This is almost twice better than RTX 5090. This GPU will not be prefill constrained, I feel.

But I am surprised that they were able to pack so much flops into 300W.

> But I am surprised that they were able to pack so much flops into 300W.

I'm not. When you're doing inference you typically reduce power limits anyway if you're optimizing for efficiency. Even if you don't, it doesn't spike to the max of the card.

My 3090 I power it down to 175W or so for single-digit reductions in performance for like, 60% of the wattage, something like that.

Am I wrong to assume that large MoE models are likely the point anyway? Seems like this card would love DeepSeek V4.1 Flash.

This is a brand new class of GPU that I am excited for, especially because it's rated for air cooling. I hope to see other manufacturers release GPU's with multiple of 100 GB's that can be utilized even in an at-home system's with PCIe 5 support. I probably can't afford this but knowing the engineering is sound at this scale means a similar accelerator maybe attainable.

> Efficient token serving Dense, air-cooled, standard PCIe infrastructure helps maximize tokens per rack and watt, scaling serving capacity and keeping costs in check.

Next the tokens will be asking for tequila and caviar. Bow down to the new token overloads.

Yay more vaporware hardware I get to read about and never see or have a chance to buy.

This stuff you usually buy second hand in 6 years when the data centers get rid of them.

Ai generated no doubt

All these big names in tech will monetize so hard on such products, until this becomes obsolete and available to us all. This should be accessible to everyone at a convenient price, governments should force these companies to mass produce them and sell to people, providing a small subsidy. Everyone deserves to run big models locally.

Reply on news.ycombinator.com