Nvidia's Project Digits is a 'personal AI supercomputer' | Hacker News Reader

Nvidia's Project Digits is a 'personal AI supercomputer'

techcrunch.com

623 pointsmagicalhippo495 comments

I'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years.

Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch etc. became useless.

I'll never buy a SBC from Nvidia unless all the SW support is up-streamed to Linux kernel.

This is a very important point.

In general, Nvidia's relationship with Linux has been... complicated. On the one hand, at least they offer drivers for it. On the other, I have found few more reliable ways to irreparably break a Linux installation than trying to install or upgrade those drivers. They don't seem to prioritize it as a first class citizen, more just tolerate it the bare minimum required to claim it works.

  > Nvidia's relationship with Linux has been... complicated.

For those unfamiliar with Linus Torvalds' two-word opinion of Nvidia:

https://youtube.com/watch?v=OF_5EKNX0Eg

Now that the majority of their revenue is from data centers instead of Windows gaming PCs, you'd think their relationship with Linux should improve or already has.

The Digits device runs the same nVidia DGX OS (nVidia custom Ubuntu distro) that they run on their cloud infra.

I've had a similar experience, my Xavier NX stopped working after the last update and now it's just collecting dust. To be honest, I've found the Nvidia SBC to be more of a hassle than it's worth.

Xavier AGX owner here to report the same.

The Orin series and later use UEFI and you can apparently run upstream, non-GPU enabled kernels on them. There's a user guide page documenting it. So I think it's gotten a lot better, but it's sort of moot because the non-GPU thing is because the JetPack Linux fork has a specific 'nvgpu' driver used for Tegra devices that hasn't been unforked from that tree. So, you can buy better alternatives unless you're explicitly doing the robotics+AI inference edge stuff.

But the impression I get from this device is that it's closer in spirit to the Grace Hopper/datacenter designs than it is the Tegra designs, due to both the naming, design (DGX style) and the software (DGX OS?) which goes on their workstation/server designs. They are also UEFI, and in those scenarios, you can (I believe?) use the upstream Linux kernel with the open source nvidia driver using whatever distro you like. In that case, this would be a much more "familiar" machine with a much more ordinary Linux experience. But who knows. Maybe GH200/GB200 need custom patches, too.

Time will tell, but if this is a good GPU paired with a good ARM Cortex design, and it works more like a traditional Linux box than the Jeton series, it may be a great local AI inference machine.

AGX also has UEFI firmware which allows you to install ESXi. Then you can install any generic EFI arm64 iso in a VM with no problems, including windows.

It runs their dgx os and Jensen specifically said it would be a full part if their hw stack

If this is DGX OS, then yes, this is what you’ll find installed on their 4-cards workstations.

This is more like a micro-DGX then, for $3k.

And unless there is some expanded maintenance going on, 22.04 is EOL in 2 years. In my experience, vendors are not as on top of security patches as upstream. We will see, but given NVIDIA's closed ecosystem, I don't have high hopes that this will be supported long term.

Is there any recent, powerful SBC with fully upstream kernel support?

I can only think of raspberry pi...

rk3588 is pretty close, I believe it's usable today, just missing a few corner cases with HDMI or some such. I believe that last patches are either pending or already applied to an RC.

Radha but that’s n100 aka x64

The odroid H series. But that packs a x86 cpu.

If its stack still works, you might be able to sell or donate it to a student experimenting. They can still learn quite a few things with it. Maybe even use it for something.

Using outdated tensorflow (v1 from 2018) or outdated PyTorch makes learning harder than it need to be, considering most resources online use much newer versions of the frameworks. If you're learning the fundamentals and working from first principle and creating the building blocks yourself, then it adds to the experience. However, most most people just want to build different types of nets, and it's hard to do when the code won't work for you.

If you're expecting this device to stay relevant for 4 years you are not the target demographic.

Compute is evolving way too rapidly to be setting-and-forgetting anything at the moment.

Today I'm using 2x 3090's which are over 4 years old at this point and still very usable. To get 48gb vram I would need 3x 5070ti - still over $2k.

In 4 years, you'll be able to combine 2 of these to get 256gb unified memory. I expect that to have many uses and still be in a favorable form factor and price.

Eh? By all indications compute is now evolving SLOWER than ever. Moore's Law is dead, Dennard scaling is over, the latest fab nodes are evolutionary rather than revolutionary.

This isn't the 80s when compute doubled every 9 months, mostly on clock scaling.

I feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

This is something every company should make sure they have: an onboarding path.

Xeon Phi failed for a number of reasons, but one where it didn't need to fail was availability of software optimised for it. Now we have Xeons and EPYCs, and MI300C's with lots of efficient cores, but we could have been writing software tailored for those for 10 years now. Extracting performance from them would be a solved problem at this point. The same applies for Itanium - the very first thing Intel should have made sure it had was good Linux support. They could have it before the first silicon was released. Itaium was well supported for a while, but it's long dead by now.

Similarly, Sun has failed with SPARC, which also didn't have an easy onboarding path after they gave up on workstations. They did some things right: OpenSolaris ensured the OS remained relevant (still is, even if a bit niche), and looking the other way for x86 Solaris helps people to learn and train on it. Oracle cloud could, at least, offer it on cloud instances. Would be nice.

Now we see IBM doing the same - there is no reasonable entry level POWER machine that can compete in performance with a workstation-class x86. There is a small half-rack machine that can be mounted on a deskside case, and that's it. I don't know of any company that's planning to deploy new systems on AIX (much less IBMi, which is also POWER), or even for Linux on POWER, because it's just too easy to build it on other, competing platforms. You can get AIX, IBMi and even IBMz cloud instances from IBM cloud, but it's not easy (and I never found a "from-zero-to-ssh-or-5250-or-3270" tutorial for them). I wonder if it's even possible. You can get Linux on Z instances, but there doesn't seem to be a way to get Linux on POWER. At least not from them (several HPC research labs still offer those).

1000% all these ai hardware companies will fail if they don't have this. You must have a cheap way to experiment and develop. Even if you want to only sell a $30000 datacenter card you still need a very low cost way to play.

Sad to see big companies like intel and amd don't understand this but they've never come to terms with the fact that software killed the hardware star

It really mystifies me that Intel AMD and other hardware companies obviously Nvidia in this case Don't either have a consortium or each have their own in-house Linux distribution with excellent support.

Windows has always been a barrier to hardware feature adoption to Intel. You had to wait 2 to 3 years, sometimes longer, for Windows to get around us providing hardware support.

Any OS optimizations in Windows you had to go through Microsoft. So say you added some instructions custom silicon or whatever to speed up Enterprise databases, provide high-speed networking that needed some special kernel features, etc, there was always Microsoft being in the way.

Not just in the drag the feet communication. Getting the tech people a line problem.

Microsoft will look at every single change. It did as to whether or not it would challenge their Monopoly whether or not it was in their business interest whether or not it kept you as the hardware and a subservient role.

Raptor Computing provides POWER9 workstations. They're not cheap, still use last-gen hardware (DDR4/PCIe 4 ... and POWER9 itself) but they're out there.

https://www.raptorcs.com/content/base/products.html

There were Phi cards, but they were pricey and power hungry (at the time, now current GPU cards probably meet or exceed the Phi card's power consumption) for plugging into your home PC. A few years back there was a big fire sale on Phi cards - you could pick one up for like $200. But by then nobody cared.

The developers they are referring to aren’t just enthusiasts; they are also developers who were purchasing SuperMicro and Lambda PCs to develop models for their employers. Many enterprises will buy these for local development because it frees up the highly expensive enterprise-level chip for commercial use.

This is a genius move. I am more baffled by the insane form factor that can pack this much power inside a Mac Mini-esque body. For just $6000, two of these can run 400B+ models locally. That is absolutely bonkers. Imagine running ChatGPT on your desktop. You couldn’t dream about this stuff even 1 year ago. What a time to be alive!

The 1 PetaFLOP spec and 200GB model capacity specs are for FP4 (4-bit floating point), which means inference not training/development. It's still be a decent personal development machine, but not for that size of model.

This looks like a bigger brother of Orin AGX, which has 64GB of RAM and runs smaller LLMs. The question will be power and performance vs 5090. We know price is 1.5x

How does it run 400B models across two? I didn’t see that in the article

I think the enthusiast side of things is a negligible part of the market.

That said, enthusiasts do help drive a lot of the improvements to the tech stack so if they start using this, it’ll entrench NVIDIA even more.

I’m not so sure it’s negligible. My anecdotal experience is that since Apple Silicon chips were found to be “ok” enough to run inference with MLX, more non-technical people in my circle have asked me how they can run LLMs on their macs.

Surely a smaller market than gamers or datacenters for sure.

It's annoying I do LLMs for work and have a bit of an interest in them and doing stuff with GANS etc.

I have a bit of an interest in games too.

If I could get one platform for both, I could justify 2k maybe a bit more.

I can't justify that for just one half: running games on Mac, right now via Linux: no thanks.

And on the PC side, nvidia consumer cards only go to 24gb which is a bit limiting for LLMs, while being very expensive - I only play games every few months.

The new $2k card from Nvidia will be 32GB but your point stands. AMD is planning a unified chiplet based GPU architecture (AI/data center/workstation/gaming) called UDNA, which might alleviate some of these issues. It's been delayed and delayed though - hence the lackluster GPU offerings from team Red this cycle - so I haven't been getting my hopes up.

Maybe (LP)CAMM2 memory will make model usage just cheap enough that I can have a hosting server for it and do my usual midrange gaming GPU thing before then.

Grace + Hopper, Grace + blackwell, and discussed GB10 are much like the currently shipping AMD MI300A.

I do hope that a AMD Strix Halo ships with 2 LPCAMM2 slots for a total width of 256 bits.

Unified architecture is still on track for 2026-ish.

32gb as of last night :)

I mean negligible to their bottom line. There may be tons of units bought or not, but the margin on a single datacenter system would buy tens of these.

It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

>It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

100%

The people who prototype on a 3k workstation will also be the people who decide how to architect for a 3k GPU buildout for model training.

> It’s purely an ecosystem play imho. It benefits the kind of people who will go on to make potentially cool things and will stay loyal.

It will be massive for research labs. Most academics have to jump through a lot of hoops to get to play with not just CUDA, but also GPUDirect/RDMA/Infiniband etc. If you get older/donated hardware, you may have a large cluster but not newer features.

Academic minimal-bureaucracy purchasing card limit is about $4k, so pricing is convenient*2.

Devalapers developers developers - balmer monkey dance - the key to be entrenched is the platform ecosystem.

Also why aws is giving trainium credits for free

Yes, but people already had their Macs for others reasons.

No one goes to an Apple store thinking "I'll get a laptop to do AI inference".

They have, because until now Apple Silicon was the only practical way for many to work with larger models at home because they can be configured with 64-192GB of unified memory. Even the laptops can be configured with up to 128GB of unified memory.

Performance is not amazing (roughly 4060 level, I think?) but in many ways it was the only game in town unless you were willing and able to build a multi-3090/4090 rig.

I would bet that people running LLMs on their Macs, today, is <0.1% of their user base.

People buying Macs for LLMs—sure I agree.

Since the current MacOS comes built in with small LLMs, that number might be closer to 50% not 0.1%.

Higher than that buying the top end machines though, which are very high margin

All macs? Yes. But of 192GB mac configs? Probably >50%

I'm currently wondering how likely it is I'll get into deeper LLM usage, and therefore how much Apple Silicon I need (because I'm addicted to macOS). So I'm some way closer to your steel man than you'd expect. But I'm probably a niche within a niche.

Tons of people do, my next machine will likely be a Mac for 60% this reason and 40% Windows being so user hostile now.

my $5k m3 max 128gb disagrees

Doubt it, a year ago useful local LLMs on a Mac (via something like ollama) was barely taking off.

If what you say it's true you were among the first 100 people on the planet who were doing this; which btw, further supports my argument on how extremely rare is that use case for Mac users.

No, I got a MacBook Pro 14”with M2 Max and 64GB for LLMs, and that was two generations back.

People were running llama.cpp on Mac laptops in March 2023 and Llama2 was released in July 2023. People were buying Macs to run LLMs months before M3 machines became available in November 2023.

You could have said the same about gamers buying expensive hardware in the 00's. It's what made Nvidia big.

I keep thinking about stocks that have 100xd, and most seemed like obscure names to me as a layman. But man, Nvidia was a household name to anyone that ever played any game. And still so many of us never bothered buying the stock

Incredible fumble for me personally as an investor

Unless you predicted AI and Crypto then it was just really good, not 100x. It 20x from 2005-2020 but ~500x from 2005-2025

And if you truly did predict that Nvidia would own those markets and those markets would be massive, you could have also bought Amazon, Google or heck even Bitcoin. Anything you touched in tech really would have made you a millionaire really.

Survivors bias though. It's hard to name all the companies that failed in the dot com bust, but even among the ones that made it through, because they're not around any more, they're harder to remember than the winners. But MCI, Palm, RIM, Nortel, Compaq, Pets.com, Webvan all failed and went to zero. There's an uncountable number of ICOs and NFTs that ended up nowhere. SVB isn't exactly an tech stock but they were strongly connected to it and they failed.

It is interesting to think about crypto as a stairstep that Nvidia used to get to its current position in AI. It wasn't games > ai, but games > crypto > ai.

Nvidia joined S&P500 in 2001 so if you've been doing passive index fund investing, you probably got a little bit of it in your funds. So there was some upside to it.

There's a lot more gamers than people wanting to play with LLms at home.

There's a titanic market with people wanting some uncensored local LLM/image/video generation model. This market extremely overlaps with gamers today, but will grow exponentially every year.

How big is that market you claim? Local LLM image generation already exists out off the box on latest Samsung flagship phones and it's mostly a Gimmick that gets old pretty quickly. Hardly comparable to gaming in terms of market size and profitablity.

Plus, YouTube and the Google images is already full of AI generated slop and people are already tired of it. "AI fatigue" amongst majority of general consumers is a documented thing. Gaming fatigues is not.

I'm sure a lot of people see "uncensored" and think "porn" but there's a lot of stuff that e.g. Dall-E won't let you do.

Suppose you're a content creator and you need an image of a real person or something copyrighted like a lot of sports logos for your latest YouTube video's thumbnail. That kind of thing.

I'm not getting into how good or bad that is; I'm just saying I think it's a pretty common use case.

Apart from the uncensored bit, I'm in this small market.

Do I buy a Macbook with silly amount of RAM when I only want to mess with images occasionally.

Do I get a big Nvidia card, topping out at 24gb - still small for some LLMs, but I could occasionally play games using it at least.

>There's a titanic market

Titanic - so about to hit an iceberg and sink?

> There's a titanic market with people wanting some uncensored local LLM/image/video generation model.

No. There's already too much porn on the internet, and AI porn is cringe and will get old very fast.

>There's a titantic market

How so?

Only 40% of gamers use a PC, a portion of those use AI in any meaningful way, and a fraction of those want to set up a local AI instance.

Then someone releases an uncensored, cloud based AI and takes your market?

Sure, but those developers will create functionality that will require advanced GPUs and people will want that functionality. Eventually OS will expect it and it will became default everywhere. So, it is an important step that will push nvidia growing in the following years.

AMD thought the enthusiast side of things was a negligible side of the market.

That’s not what I’m saying. I’m saying that the people buying this aren’t going to shift their bottom line in any kind of noticeable way. They’re already sold out of their money makers. This is just an entrenchment opportunity.

If this is gonna be widely used by ML engineers, in biopharma, etc and they land 1000$ margins at half a million sales that's half a billion in revenue, with potential to grow.

today’s enthusiast, grad student, hacker is tomorrow’s startup founder, CEO, CTO or 10x contributor in large tech company

> tomorrow’s startup founder, CEO, CTO or 10x contributor in large tech company

Do we need more of those? We need plumbers and people that know how to build houses. We are completely full on founders and executives.

If they're already an "enthusiast, grad student, hacker", are they likely to choose the "plumbers and people that know how to build houses" career track?

True passion for one's career is rare, despite the clichéd platitudes ecouraging otherwise. That's something we should encourage and invest in regardless of the field.

We might not, but Nvidia would certainly like it.

If I were NVidia, I would be throwing everything I could at making entertainment experiences that need one of these to run...

I mean, this is awfully close to being "Her" in a box, right?

I feel like a lot of people miss that Her was a dystopian future, not an ideal to hit.

Also, it’s $3000. For that you could buy subscriptions to OpenAI etc and have the dystopian partner everywhere you go.

We already live in dystopian hell and I'd like to have Scarlett Johansen whispering in my ear, thanks.

Also, I don't particularly want my data to be processed by anyone else.

Fun fact: Her was set in the year 2025.

Boring fact: The underlying theme of the movie Her is actually divorce and the destructive impact it has on people, the futuristic AI stuff is just for stuffing!

This is exactly the scenario where you don't want "the cloud" anywhere.

OpenAI doesn’t make any profit. So either it dies or prices go up. Not to mention the privacy aspect of your own machine and the freedom of choice which models to run

> So either it dies or prices go up.

Or efficiency gains in hardware and software catchup making current price point profitable.

OpenAI built a 3 billion dollar business in less than 3 years of a commercial offering.

One man's dystopia is another man's dream. There's no "missing" in the moral of a movie, you make whatever you want out of it.

If Silicon Valley could tell the difference between utopias and dystopias, we wouldn't have companies named Soylent or iRobot, and the recently announced Anduril/Palantir/OpenAI partnership to hasten the creation of either SkyNet or Big Brother wouldn't have happened at all.

I mean, we still act like a "wild goose chase" is a bad thing.

We still schedule "bi-weekly" meetings.

We can't agree on which way charge goes in a wire.

Have you seen the y-axis on an economists chart?

The dystopian overton window has shifted, didn't you know, moral ambiguity is a win now? :) Tesla was right.

they don't miss that part. they just want to be the evil character.

Please name the dystopian elements of Her.

The real interesting stuff will happen when we get multimodal LMs that can do VR output.

Yeah, it's more about preempting competitors from attracting any ecosystem development than the revenue itself.

Jensen did say in recent interview, paraphrasing, “they are trying to kill my company”.

Those Macs with unified memory is a threat he is immediately addressing. Jensen is a wartime ceo from the looks of it, he’s not joking.

No wonder AMD is staying out of the high end space, since NVIDIA is going head on with Apple (and AMD is not in the business of competing with Apple).

From https://www.tomshardware.com/pc-components/cpus/amds-beastly...

The fire-breathing 120W Zen 5-powered flagship Ryzen AI Max+ 395 comes packing 16 CPU cores and 32 threads paired with 40 RDNA 3.5 (Radeon 8060S) integrated graphics cores (CUs), but perhaps more importantly, it supports up to 128GB of memory that is shared among the CPU, GPU, and XDNA 2 NPU AI engines. The memory can also be carved up to a distinct pool dedicated to the GPU only, thus delivering an astounding 256 GB/s of memory throughput that unlocks incredible performance in memory capacity-constrained AI workloads (details below). AMD says this delivers groundbreaking capabilities for thin-and-light laptops and mini workstations, particularly in AI workloads. The company also shared plenty of gaming and content creation benchmarks.

[...]

AMD also shared some rather impressive results showing a Llama 70B Nemotron LLM AI model running on both the Ryzen AI Max+ 395 with 128GB of total system RAM (32GB for the CPU, 96GB allocated to the GPU) and a desktop Nvidia GeForce RTX 4090 with 24GB of VRAM (details of the setups in the slide below). AMD says the AI Max+ 395 delivers up to 2.2X the tokens/second performance of the desktop RTX 4090 card, but the company didn’t share time-to-first-token benchmarks.

Perhaps more importantly, AMD claims to do this at an 87% lower TDP than the 450W RTX 4090, with the AI Max+ running at a mere 55W. That implies that systems built on this platform will have exceptional power efficiency metrics in AI workloads.

> since NVIDIA is going head on with Apple

I think this is a race that Apple doesn't know it's part of. Apple has something that happens to work well for AI, as a side effect of having a nice GPU with lots of fast shared memory. It's not marketed for inference.

Which interview was this?

You missed the Ryzen hx ai pro 395 product announcement

From the people I talk to the enthusiast market is nvidia 4090/3090 saturated because people want to do their fine tunes also porn on their off time. The Venn diagram of users who post about diffusion models and llms running at home is pretty much a circle.

Not your weights, not your waifu

Yeah, I really don't think the overlap is as much as you imagine. At least in /r/localllama and the discord servers I frequent, the vast majority of users are interested in one or the other primarily, and may just dabble with other things. Obviously this is just my observations...I could be totally misreading things.

> I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!

They propelled on unexpected LLM boom. But plan 'A' was robotics in which NVidia invested a lot for decades. I think their time is about to come, with Tesla's humanoids for 20-30k and Chinese already selling for $16k.

This is somewhat similar to what GeForce was to gamers back in the days, but for AI enthusiasts. Sure, the price is much higher, but at least it's a completely integrated solution.

Yep that's what I'm thinking as well. I was going to buy a 5090 mainly to play around with LLM code generation, but this is a worthy option for roughly the same price as building a new PC with a 5090.

>enthusiast AI dev segment

i think it isn't about enthusiast. To me it looks like Huang/NVDA is pushing further a small revolution using the opening provided by the AI wave - up until now the GPU was add-on to the general computing core onto which that computing core offloaded some computing. With AI that offloaded computing becomes de-facto the main computing and Huang/NVDA is turning tables by making the CPU is just a small add-on on the GPU, with some general computing offloaded to that CPU.

The CPU being located that "close" and with unified memory - that would stimulate development of parallelization for a lot of general computing so that it would be executed on GPU, very fast that way, instead of on the CPU. For example classic of enterprise computing - databases, the SQL ones - a lot, if not, with some work, everything, in these databases can be executed on GPU with a significant performance gain vs. CPU. Why it isn't happening today? Load/unload onto GPU eats into performance, complexity of having only some operations offloaded to GPU is very high in dev effort, etc. Streamlined development on a platform with unified memory will change it. That way Huang/NVDA may pull out rug from under the CPU-first platforms like AMD/INTC and would own both - new AI computing as well as significant share of the classic enterprise one.

> these databases can be executed on GPU with a significant performance gain vs. CPU

No, they can’t. GPU databases are niche products with severe limitations.

GPUs are fast at massively parallel math problems, they anren’t useful for all tasks.

> I sure wished I held some Nvidia stocks

I’m so tired of this recent obsession with the stock market. Now that retail is deeply invested it is tainting everything, like here on a technology forum. I don’t remember people mentioning Apple stock every time Steve Jobs made an announcement in the past decades. Nowadays it seems everyone is invested in Nvidia and just want the stock to go up, and every product announcement is a mean to that end. I really hope we get a crash so that we can get back to a more sane relation with companies and their products.

> hope we get a crash

That's the best time to buy. ;)

“Bigger” in what sense? For AI? Sure, because this an AI product. 5x series are gaming cards.

Not expecting this to compete with the 5x series in terms of gaming; But it's interesting to note the increase in gaming performance Jensen was speaking about with Blackwell was larger related to inferenced frames generated by the tensor cores.

I wonder how it would go as a productivity/tinkering/gaming rig? Could a GPU potentially be stacked in the same way an additional Digit can?

Bigger in the sense of the announcements.

Eh. Gaming cards, but also significantly faster. If the model fits in the VRAM the 5090 is a much better buy.

I bet $100k on NVIDIA stocks ~7 years ago, just recently closed out a bunch of them

> they seem to be doing everything right in the last few years

About that... Not like there isn't a lot to be desired from the linux drivers: I'm running a K80 and M40 in a workstation at home and the thought of having to ever touch the drivers, now that the system is operational, terrifies me. It is by far the biggest "don't fix it if it ain't broke" thing in my life.

Use a filesystem that snapshots AND do a complete backup.

Buy a second system which you can touch?

Will there really be a mac mini wirh Max or Ultra CPUs? This feels like somewhat of an overlap with the Mac Studio.

There will undoubtably be a Mac Studio (and Mac Pro?) bump to M4 at some point. Benchmarks [0] reflect how memory bandwidth and core count [1] compare to processor improvements. Granted, ymmv to your workload.

0. https://www.macstadium.com/blog/m4-mac-mini-review

1. https://www.apple.com/mac/compare/?modelList=Mac-mini-M4,Mac...

The nVidia price is closer (USD 3k) to a top Mac mini but I trust Apple more for the end-to-end support from hardware to apps than nVidia. Not an Apple fanboy but an user/dev, and I don't think we realize what Apple really achieved, industrially speaking. The M1 was launched in late 2020.

Did they say anything about power consumption?

Apple M chips are pretty efficient.

Not only that, but it should help free up the gpus for the gamers.

it eats into all NVDA consumer-facing clients no? I can see why openai and etc are looking for alternative hardware solution to train their next model.

I can confirm this is the case (for me).

I would like to have Mac as my personal computer and digits as service to run llm.

Am I the only one disappointed by these? They cost roughly half the price of a macbook pro and offer hmm.. half the capacity in RAM. Sure speed matters in AI, but what do I do with speed when I can't load a 70b model.

On the other hand, with a $5000 macbook pro, I can easily load a 70b model and have a "full" macbook pro as a plus. I am not sure I fully understand the value of these cards for someone that want to run personal AI models.

Are you, perhaps, commenting on the wrong thread? Project Digits is a $3k 128GB computer.. the best your your $5K MBP can have for ram is.. 128GB.

Hm? They have 128GB of RAM. Macbook Pros cap out at 128GB as well. Will be interesting to see how a Project Digits machine performs in terms of inference speed.

Then buy two and stack them!

Also I'm unfamiliar with macs is there really a MacBook pro with 256GB of RAM?

Bro we can connect two ProjectDigits as well. I was only looking at the M4 macbook because 128gb unified memory. Now this beast can cook better LLMs at just 3K with 4TB SSD too. M4 Macbook Max (128 GB unified ram and 4TB Storage) is 5999. So, No more apple for me. I will just get the Digits. And can create a workstation as well.

What slice?

Also, macOS devices are not very good inference solutions. They are just believed to be by diehards.

I don't think Digits will perform well either.

If NVIDIA wanted you to have good performance on a budget, it would ship NVLink on the 5090.

They are perfectly fine for certain people. I can run Qwen-2.5-coder 14B on my M2 Max MacBook Pro with 32gb at ~16 tok/sec. At least in my circle, people are budget conscious and would prefer using existing devices rather than pay for subscriptions where possible.

And we know why they won't ship NVLink anymore on prosumer GPUs: they control almost the entire segment and why give more away for free? Good for the company and investors, bad for us consumers.

> Also, macOS devices are not very good inference solutions

They are good for single batch inference and have very good tok/sec/user. ollama works perfectly in mac.

Nvidia releases a Linux desktop supercomputer that's better price/performance wise than anything Wintel is doing and their whole new software stack will only run on WSL2. They aren't porting to Win32. Wow, it may actually be the year of Linux on the Desktop.

Not sure how to judge better price/perf. I wouldn't expect 20 Neoverse N2 cores to do particularly well vs 16 zen5 cores. The GPU side looks promising, but they aren't mentioning memory bandwidth, configuration, spec, or performance.

Did see vague claims of "starting at $3k", max 4TB nvme, and max 128GB ram.

I'd expect AMD Strix Halo (AI Max plus 395) to be reasonably competitive.

It’s actually “10 Arm Cortex-X925 and 10 Cortex-A725” [0]. These are much newer cores and have a reasonable chance of being competitive.

[0]: https://newsroom.arm.com/blog/arm-nvidia-project-digits-high...

NVIDIA is likely citing 1 PFlops at FP 4 sparse (they did this for GB200), so that is 128 TFlops BF16 dense, or 2/3 of what RTX 4090 is capable of. I would put the memory bandwidth at 546 GBps, using the same 512 bit LPDDR5X 8533 Mbps as Apple M4 max.

Seems more like a workstation. So, that’s just a continuation of the last could Decades of Unix on the Workstation, right?

They should write an AI-centered OS for it, allowing people to write easily AI heavy applications. And you'd have the Amiga of 2025.

Because NVidia naturally doesn't want to pay for Windows licenses.

NVidia works closely with Microsoft to develop their cards, all major features come first in DirectX, before landing on Vulkan and OpenGL as NVidia extensions, and eventually become standard after other vendors follow up with similar extensions.

Where does it say they won't be supporting Win32?

Here he says that in order for the cloud and the PC to be compatible, he's going to only support WSL2, the Windows subsystem for Linux which is a Linux API on top of Windows.

Here's a link to the part of the keynote where he says this:

https://youtu.be/MC7L_EWylb0?t=7259

> their whole new software stack will only run on WSL2. They aren't porting to Win32

Wait, what do you mean exactly? Isn't WSL2 just a VM essentially? Don't you mean it'll run on Linux (which you also can run on WSL2)?

Or will it really only work with WSL2? I was excited as I thought it was just a Linux Workstation, but if WSL2 gets involved/is required somehow, then I need to run the other direction.

No, nobody will run windows on this. It's meant to run NVIDIAs own flavor of Ubuntu with a patched kernel

Yes, WSL2 is essentially a highly integrated VM. I think it's a bit of a joke to call Ubuntu WSL2, because it seems like most Ubuntu installs are either VMs for Windows PCs or on Azure Cloud.

> Wow, it may actually be the year of Linux on the Desktop.

Yeah starting at $3,000. Surely a cheap desktop computer to buy for someone who just wants to surf the web and send email /s.

There is a reason why it is for "enthusiasts" and not for the general wider consumer or typical PC buyer.

I see the most direct competitor in the Mac Studio, though of course we will have to wait for reviews to gauge how fair that comparison is. The Studio does have a fairly large niche as a solid workstation, though, so I could see this being successful.

For general desktop use, as you described, nearly any piece of modern hardware, from a RasPI, to most modern smartphones with a dock, could realistically serve most people well.

The thing is, you need to serve both, low-end use cases like browsing, and high-end dev work via workstations, because even for the "average user", there is often one specific program on which they need to rely and which has limited support outside the OS they have grown up with. Course, there will be some programs like Desktop Microsoft Office which will never be ported, but still, Digitis could open the doors to some devs working natively on Linux.

A solid, compact, high-performance, yet low power workstation with a fully supported Linux desktop out of the box could bridge that gap, similar to how I have seen some developers adopt macOS over Linux and Windows since the release of the Studio and Max MacBooks.

Again, we have yet to see independent testing, but I would be surprised if anything of this size, simplicity, efficiency and performance was possible in any hardware configuration currently on the market.

> Surely a cheap desktop computer to buy for someone who just wants to surf the web and send email /s.

That end of the market is occupied by Chromebooks... AKA a different GNU/Linux.

The typical PC buyer is an enthusiast now.

Never underestimate the open source world's power to create a crappy desktop experience.

You're like 15 years out of date.

I'm a bit surprised by the amount of comments comparing the cost to (often cheap) cloud solutions. Nvidia's value proposition is completely different in my opinion. Say I have a startup in the EU that handles personal data or some company secrets and wants to use an LLM to analyse it (like using RAG). Having that data never leave your basement sure can be worth more than $3000 if performance is not a bottleneck.

Heck, I'm willing to pay $3000 for one of these to get a good model that runs my requests locally. It's probably just my stupid ape brain trying to do finance, but I'm infinitely more likely to run dumb experiments with LLMs on hardware I own than I am while paying per token (to the point where I currently spend way more time with small local llamas than with Claude), and even though I don't do anything sensitive I'm still leery of shipping all my data to one of these companies.

This isn't competing with cloud, it's competing with Mac Minis and beefy GPUs. And $3000 is a very attractive price point in that market.

Have you been to the localLlama subreddit? It’s a great resource for running models locally. It’s what got me started.

https://www.reddit.com/r/LocalLLaMA/

I'm pretty frugal, but my first thought is to get two to run 405B models. Building out 128GB of VRAM isn't easy, and will likely cost twice this.

100% I see many SMEs not willing to send their data to some cloud black box.

Exactly this. I would happily give $3k to NVIDIA to avoid giving 1 cent to OpenAI/Anthropic.

Even for established companies this is great. A tech company can have a few of these locally hosted and users can poll the company LLM with sensitive data.

The price seems relatively competitive even compared to other local alternatives like "build your own PC". I'd definitely buy one of this (or even two if it works really well) for developing/training/using models that currently run on cobbled together hardware I got left after upgrading my desktop.

> Having that data never leave your basement sure can be worth more than $3000 if performance is not a bottleneck

I get what you're saying, but there are also regulations (and your own business interest) that expects data redundancy/protection which keeping everything on-site doesnt seem to cover

Yeah that's cheaper than many prosumer GPUs on the market right now

There's a market not described here: bioinformatics.

The owner of the market, Illumina, already ships their own bespoke hardware chips in servers called DRAGEN for faster analysis of thousands of genomes. Their main market for this product is in personalised medicine, as genome sequencing in humans is becoming common.

Other companies like Oxford Nanopore use on-board GPUs to call bases (i.e., from raw electric signal coming off the sequencer to A, T, G, C) but it's not working as well as it could due to size and power constraints. I feel like this could be a huge game changer for someone like ONT, especially with cooler stuff like adaptive sequencing.

Other avenues of bioinformatics, such as most day-to-day analysis software, is still very CPU and RAM heavy.

This is, at least for now, a relatively small market. Illumina acquired the company manufacturing these chips for $100M. Analysis of a genome in the cloud generally costs below $10 on general purpose hardware.

It is of course possible that these chips enable analyses that are currently not possible/prohibited by cost, but at least for now, this will not be the limiting factor for genomics, but cost of sequencing (which is currently $400-500 per genome)

Doesn't seem like Illumina actually cares much about security: https://arstechnica.com/security/2025/01/widely-used-dna-seq...

The bigger picture is that OpenAI o3/o4.. plus specialized models will blow open the doors to genome tagging and discovery, but that is still 1 to 3 years away for ASI to kick in.

While I kinda agree with you, I don't think we will ever find a meaningful way to throw genome sequencing data at LLMs. It's simple too much data.

I've worked in a project some years ago where we were using data from genome sequencing of a bacteria. Every sequenced sample was around 3GB of data and sample size was pretty small with only about 100 samples to study.

I think the real revolution will happen because code generation through LLMs will allow biologists to write 'good enough' code to transform, process and analyze data. Today to do any meaningful work with genome data you need a pretty competent bioinformatician, and they are a rare breed. Removing this bottleneck is what will allow us to move faster in this field.

Is this for research labs, health clinics, or peoples homes?

ONT sells its smallest MinION to regular people, too. But Illumina's and ONT's main market is universities, followed by large hospitals

Small nitpick: Illumia is the owner of the sequencing market. But not really of the bioinformatics market.

In case you're curious, I googled. It runs this thing called "DGX OS":

"DGX OS 6 Features The following are the key features of DGX OS Release 6:

Based on Ubuntu 22.04 with the latest long-term Linux kernel version 5.15 for the recent hardware and security updates and updates to software packages, such as Python and GCC.

Includes the NVIDIA-optimized Linux kernel, which supports GPU Direct Storage (GDS) without additional patches.

Provides access to all NVIDIA GPU driver branches and CUDA toolkit versions.

Uses the Ubuntu OFED by default with the option to install NVIDIA OFED for additional features.

Supports Secure Boot (requires Ubuntu OFED).

Supports DGX H100/H200."

Nvidia optimize meaning non-public patches, a non-upgradable operating system like what happens if you upgrade with a binary blob Nvidia driver?

You can upgrade to a newer release of DGX OS.

I wonder what kind of spyware is loaded onto DGX OS. Oh, sorry I mean telemetry.

Cybersecurity analysts check and monitor these things daily, and they are pretty easy to catch. Likely nothing malicious, as history shows.

Correct, highly concerning.. this is totally not the case with existing os’s and products

Nvidia just did what Intel/AMD should have done to threaten CUDA ecosystem - release a "cheap" 128GB local inference appliance/GPU. Well done Nvidia, and it looks bleak for any AI Intel/AMD efforts in the future.

I think you nailed it. Any basic SWOT analysis of NVidia’s position would surely have to consider something like this from a competitor - either Apple, who is already nibbling around the edges of this space, or AMD/Intel who could/should? be.

It’s obviously not guaranteed to go this route, but an LLM (or similar) on every desk and in every home is a plausible vision of the future.

Nvidia also brought Mediatek into the spotlight..

Okay, so this is not a peripheral that you connect to your computer to run specialized tasks, this is a full computer running Linux.

It's a garden hermit. Imagine a future where everyone has one of those(not exactly this version but some future version), it lives with you it learns with you and unlike the cloud based SaaS AI you can teach it things immediately and diverge from the average to your advantage.

I'd love to own one, but doubt this will go beyond a very specific niche. Despite there being advantages, very few still operate their own Plex server over subscriptions to streaming services, and on the local front, I feel that the progress of hardware, alongside findings that smaller models can handle a variety of tasks quite well, will mean a high performance, local workstation of this type will have niche appeal at most.

I have this feeling that at some point it will be very advantageous to have personal AI because when you use something that everyone can use the output of this something becomes very low value.

Maybe it will still make sense to have your personal AI in some data center, but on the other hand, there is the trend of governments and mega corps regulating what you can do with your computer. Try going out of the basics, try to do something fun and edge case - it is very likely that your general availability AI will refuse to help you.

when it is your own property, you get the chance to overcome restrictions and develop the thing beyond the average.

As a result, having something that can do things that no other else can do and not having restrictions on what you can do with this thing can become the ultimate superpower.

"garden hermit" is a very interesting and evocative phrase. Where is that from?

It's a real thing: https://en.wikipedia.org/wiki/Garden_hermit

In the past, in Europe, some wealthy people used to look after of a scholar living on their premises so they can ask them questions etc.

>> The IBM Roadrunner was the first supercomputer to reach one petaflop (1 quadrillion floating point operations per second, or FLOPS) on May 25, 2008.

$100M, 2.35MW, 6000 ft^2

>>Designed for AI researchers, data scientists, and students, Project Digits packs Nvidia’s new GB10 Grace Blackwell Superchip, which delivers up to a petaflop of computing performance for prototyping, fine-tuning, and running AI models.

$3000, 1kW, 0.5 ft^2

Digits is petaflops of FP4, roadrunner is petaflops of FP32. So at least a factor of 8 difference, but in practice much more. (IE I strongly doubt digits can do 1/8th petaflop of FP32)

Beyond that, the factors seem reasonable for 2 decades?

Why even use a floating point if you have only 4 bits? Models with INT8 features are not unheard of.

> roadrunner is petaflops of FP32

Isn't it actually FP64?

This looks like a successor to the Nvidia Jetson AGX Orin 64GB Developer Kit:

https://www.okdo.com/wp-content/uploads/2023/03/jetson-agx-o...

I wonder what the specifications are in terms of memory bandwidth and computational capability.

Hopefully, the OS support isn't as awful as the Jetson platforms usually are. Unless they change, you'll get 1 or 2 major kernel updates ever and have to do bizarre stuff like install a 6 year old Ubuntu on your x86 PC to run the utility to flash the OS.

The community likely will make instructions for installing mainstream Linux distributions on it.

The Jetson Orin Dev Kit is squarely aimed at being a dev kit for those using the Jetson module in production edge compute (robotic vision and the like). The only reason it's so well known in tech circles is "SBC syndrome" where people get excited about what they think they could do with it and then 95% end up in a drawer a year later because it what it's actually good at is unrelated to why they bought it.

This is more accurately a descendant of the HPC variants like the article talks about - intentionally meant to actually be a useful entry level for those wanting to do or run general AI work better than a random PC would have anyways.

The AGX Orin was only 64GB of LPDDR5 and priced at $5k so this does seem like a bargain in comparison with 128GB of presumably HBM. But Nvidia never lowers their prices, so there's a caveat somewhere.

The memory is LPDDR accordning to the specs graphic on the NV product page: https://www.nvidia.com/en-us/project-digits/

Anyone willing to guess how wide?

I've followed progress since Moravec's "When will computer hardware match the human brain?" since that came out in 1997. It starts:

>This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain are estimated. Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s.

and this is about the first personal unit that seems well ahead of his proposed specs. (He estimated 0.1 petaflops. The nvidia thing is "1 petaflop of AI performance at FP4 precision").

(paper https://jetpress.org/volume1/moravec.pdf)

Finally a real ARM workstation from Nvidia! This will be much faster than Apple's offerings for AI work. And at $3000 it is much cheaper than any Mac with 128 GB RAM.

On the CPU size the Neoverse N2 doesn't compete particularly well with apple's M4, or the Zen5 for the matter.

Bit bit hard to tell what's on offer on the GPU side, I wouldn't be surprised if it was RTX 4070 to 5070 in that range.

If the price/perf is high enough $3k wouldn't be a bad deal, I suspect a Strix Halo (better CPU cores, 256GB/sec memory interface, likely slower GPU cores) will be better price/perf, same max ram for unified memory, and cheaper.

It’s actually “10 Arm Cortex-X925 and 10 Cortex-A725” [0]. These are much newer cores and have a reasonable chance of being competitive.

[0]: https://newsroom.arm.com/blog/arm-nvidia-project-digits-high...

AI work happens predominantly on the GPU, not the CPU. This GPU with CUDA will run rings around M4 with MLX. And with much more RAM than you can get in a Mac for $3k.

A lot of people have been justifying their Mac Studio or Mac Pro purchases by the potential for running large AI models locally. Project Digits will be much better at that for cheaper. Maybe it won't run compile Chromium as fast, but that's not what it's for.

Not much was unveiled but it showed a Blackwell GPU with 1PFLOP of FP4 compute, 128GB unified DDR5X memory, 20 ARM cores, and ConnectX powering two QSFP slots so one can stack multiple of them.

edit: While the title says "personal", Jensen did say this was aimed at startups and similar, so not your living room necessarily.

From the size and pricing ($3000) alone, it's safe to conclude it has less raw FLOPs than a 5090. Since it uses LPDDR5X, almost certainly less memory bandwidth too (5090 @ 1.8 TB/s, M4 Max w/ 128GB LPDDR5X @ 546 GB/s). Basically the only advantage is how much VRAM it packs in a small form factor, and presumably greater power efficiency at its smaller scale.

The only thing it really competes with is the Mac Studio for LocalLlama-type enthusiasts and devs. It isn't cheap enough to dent the used market, nor powerful enough to stand in for bigger cards.

The product isn't even finalized. It might never come to fruition, and I cannot fathom how they will make the power profile fit. I am skeptical that a $3000 device with 128GB of RAM and a 4TB SSD with the specs provided will even see reality any time within the next year, but let's pretend it will.

However we do know that it offers 1/4 the TOPS of the new 5090. It will be less powerful than the $600 5070. Which, of course it will given power limitations.

The only real compelling value is that nvidia memory starves their desktop cards so severely. It's the small opening that Apple found, even though Apple's FP4/FP8 performance is a world below what nvidia is offering. So purely from that perspective this is a winning product, as 128GB opens up a lot of possibilities. But from a raw performance perspective, it's actually going to pale compared to other nvidia products.

I believe $3,000 is for the unmentioned minimum config, no idea on the mentioned 4TB storage and 128GB ram version.

Running a 96GB ram model isn't cheap (often with unified memory 25% is reserved for CPUs), so maybe it will win there.

Of course. It has much less FLOPs than the 5090, after all this will have a TDP of ~50W and run off a regular USB-PD power supply.

It's basically the successor to the AGX Orin and in line with its pricing (considering it comes with a fast NIC). The AGX Orin had RTX 3050 levels of performance.

Making comparisons to the 5090 is silly. That thing draws 500W+ and will require a boat anchor of metal to keep it cool. The device they showed is something more along the lines of a mobile dev kit.

this image seems to be AI Generated - :-)

https://s3.amazonaws.com/cms.ipressroom.com/219/files/20250/...

Source: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...

The text on the screen is an obvious giveaway.

Agree, unless I've missed some recent invention where keyboards now have two of either Enter/Backspace/Shift keys on the right side.

Not sure if that isn't expected though? Likely most people wouldn't even notice, and the company can say they're dogfooding some product I guess.

Prompt: something with some splashy graph on screen.

$3k for a 128GB standalone is quite favorable pricing considering the next best option at home is going to be a 32GB 5090 at $2k for the card alone, so probably $3k when you’re done building a rig around it.

The press-release says "up to 128GB" while the price is a single figure of $3,000. So it won't be out of the real of possibility that the 128GB version would cost quite a bit more.

The memory bandwidth has not been announced for this device. It's probably going to be more appropriate to compare vs a 128GB M4 Max (410-546GB/s MBW) or an AMD Ryzen AI Max+ 395 (yes, that's its real name) at 256GB/s of MBW.

The 5090 has 1.8TB/s of MBW and is in a whole different class performance-wise.

The real question is how big of a model will you actually want to run based on how slowly tokens generate.

Agreed. I care more about LLM size than tokens/sec so the GB10 or Strix Halo with 128GB are my leading choices. Both look to be cheaper than a similar mac studio with 128GB (minimum $4,800 currently). Will have to wait on final config, pricing, and performance.

With more and more personal AI, i think having a truly private device that can run large LLMs (remember: larger is better) is fantastic!

Ideally we can configure things like Apple Intelligence to use this instead of OpenAI and Apple's cloud.

How about “We sell a computer called the tinybox. It comes in two colors + pro.

tinybox red and green are for people looking for a quiet home/office machine. tinybox pro is for people looking for a loud compact rack machine.” [0]

[0] https://tinygrad.org/#tinybox

This was my second thought; while we don’t have full performance data, it’s probably a bad day for tinybox.

These look terrible. For 5 times the price you get worse performance.

Going by the specs, this pretty much blows Tinybox out of the water.

For $40,000, a Tinybox pro is advertised as offering 1.36 petaflops processing and 192 GB VRAM.

For about $6,000 a pair of Nvidia Project Digits offer about a combined 2 petaflops processing and 256 GB VRAM.

The market segment for Tinybox always seemed to be people that were somewhat price-insensitive, but unless Nvidia completely fumbles on execution, I struggle to think of any benefits of a Tinygrad Tinybox over an Nvidia Digits. Maybe if you absolutely, positively, need to run your OS on x86.

I'd love to see if AMD or Intel has a response to these. I'm not holding my breath.

>tinybox

>the size of several ATX desktops

Is there any effort in local cloud computing? I can't justify $3000 for a fun device. But if all devices (6 phone, 2 iPads, a desktop and 2 laptops) in my home can leverage that for fast LLM, gaming, and photo/video editing, now it makes so much more sense.

Open WebUI, SillyTavern, and other frontends can access any OpenAI-compatible server, and on Nvidia cards you have a wealth of options that will run one of those servers for you: llama.cpp (or the Ollama wrapper), of course, but also the faster vLLM and SGLang inference engines. Buy one of these, slap SGLang or vLLM on it, and point your devices at your machine's local IP address.

I'm mildly skeptical about performance here: they aren't saying what the memory bandwidth is, and that'll have a major impact on tokens-per-second. If it's anywhere close to the 4090, or even the M2 Ultra, 128GB of Nvidia is a steal at $3k. Getting that amount of VRAM on anything non-Apple used to be tens of thousands of dollars.

(They're also mentioning running the large models at Q4, which will definitely hurt the model's intelligence vs FP8 or BF16. But most people running models on Macs runs them at Q4, so I guess it's a valid comparison. You can at least run a 70B at FP8 on one of these even with fairly large context size, which I think will be the sweet spot.)

That is literally how it was announced as. AI cloud in a box. That can also be used as Linux desktop.

You can just setup your local openAI like API endpoints for LLMs. Most devices and apps won't be able to use them, because consumers don't run self-hosted apps, but for a simple chatGPT style app this is totally viable. Today.

For companies interested in integrating machine learning into their products, both soft and hard - essentially training models specific to a particular use-case - this could be quite a useful tool to add to the kit.

Most tools expose openai-like apis that you can easily integrate with.

Amortized cost is 10 cents per petaflop hour if you run it 5-6 years 24/7. I'm including the cost of electricity.

This is really game changer.

They should make a deal with Valve to turn this into 'superconsole' that can run Half Life 3 (to be announced) :)

I bought a Lamba Labs workstation with a 4090 last year. I guess I’m buying one of these things now because the Lambda workstation just became a relic…

One thing I didn't see mentioned: this would be a good motivation for Nvidia to release "open weights" models.

Just like Mac OS is free when you buy a Mac, having the latest high-quality LLM for free that just happens to run well on this box is a very interesting value-prop. And Nvidia definitely has the compute to make it happen.

They already do release open weight models, in this very keynote he released some of the bigget open weight models yet: https://huggingface.co/collections/nvidia/cosmos-6751e884dc1...

They did exactly this, announcing at the same event new Nemo models upcoming (a fine-tune of llama).

What we need is more diversity in the space. Something between the Jetson and this thing, at under $1000 that can run in a LAN to do LLM, STT, TTS, etc. would be an awesome (if niche) device to enable truly local / private AI scenarios for privacy-sensitive folks.

So, a company that doesn't feel like sharing all their secret sauce with Anthropic can run DeepSeek Coder on three of these for $9K, and it should be be more or less the same experience.

Do I understand that right? It seems way to cheap.

Such a setup would likely work - assuming DS3 support on software stack but wouldn’t be able to serve as many requests in parallel as a classic gpu setup.

Main issue is the ram they’re using here isn’t the same as is in GPUs

If they end up actually shipping this, lots of people will buy these machines to get an AArch64 Linux workstation—even if they are not interested in AI or Nvidia GPUs.

At $3,000, it will be considerably cheaper than alternatives available today (except for SoC boards with extremely poor performance, obviously). I also expect that Nvidia will use its existing distribution channels for this, giving consumers a shot at buying the hardware (without first creating a company and losing consumer protections along the way).

> At $3,000, it will be considerably cheaper than alternatives available today

$3000 gets me a 64-core Altra Q64-22 from a major-enough SI today: https://system76.com/desktops/thelio-astra-a1-n1/configure

And of course if you don't care about the SI part, then you can just buy that motherboard & CPU directly for $1400 https://www.newegg.com/asrock-rack-altrad8ud-1l2t-q64-22-amp... with the 128-core variant being $2400 https://www.newegg.com/asrock-rack-altrad8ud-1l2t-q64-22-amp...

Is this going to make up for the lack of VRAM in the new consumer GPUs?

This is it guys, the most valuable company in the world is selling Linux desktop, after decades of speculating finally the year of Linux desktop is 2025!

Joking aside, personally will buy this workstation in a heartbeat if I have the budget to spare, one in the home and another in the office.

Currently I have an desktop/workstation for AI workloads with similar 128GB RAM that I bought few years back that cost around USD5K without the NVIDIA GPU that I bought earlier for about USD1.5K, with a total of about USD6.5K without a display monitor. This the same price of NeXT workstation (with a monitor) when it's sold back in 1988 without adjusting for inflations (now around USD18K) but it is more than 200 times faster in CPU speed and more than 1000 times RAM capacity than the original 25 MHz CPU and 4 MB RAM, respectively. The later updated version of NeXT has graphic accelerator with 8 MB VRAM, since the workstation has RTX 2080 it is about 1000 times more. I believe the updated NeXT with graphic accelerator is the one that used to develop original Doom software [1].

If NVIDIA can sell the Project Digits Linux desktop at USD3K with similar or more powerful setup configurations, it's going to be a winner and probably can sell by truckloads. It seems to has NeXT workstation vibe to it that used to develop the original WWW and Doom software. Hopefully it will be used to develop many innovative software but now using open source Linux software eco-system not proprietary one.

The latest Linux kernel now has real-time capability for more responsive desktop experience and as saying goes, good things come to those who wait.

[1] NeXT Computer:

https://en.m.wikipedia.org/wiki/NeXT_Computer

I think this is version 1 of what's going to become the new 'PC'.

Future versions will get more capable and smaller, portable.

Can be used to train new types models (not just LLMs).

I assume the GPU can do 3D graphics.

Several of these in a cluster could run multiple powerful models in real time (vision, llm, OCR, 3D navigation, etc).

If successful, millions of such units will be distributed around the world within 1-2 years.

A p2p network of millions of such devices would be a very powerful thing indeed.

> A p2p network of millions of such devices would be a very powerful thing indeed.

If you think RAM speeds are slow for the transformer or inference, imagine what 100Mbs would be like.

Will there be a healthy "personal AI supercomputer" economy to generate demand for this? (NB: spam generators are only a parasite on any digital economy, viable to the extend they don't kill the host).

One can only wish for this, but Nvidia would be going against the decades-long trend to emaciate local computing in favor of concentrating all compute on somebody else's linux (aka: cloud).

NVIDIA has a robotics department where I could see this tech fitting pretty nicely.

Also I consider this a dev board. Soon this tech will be everywhere, in our phones, computers...

You could already plug that to your home assistant and have your own Star Trek computer you can ask questions from. And NVIDIA seems to know this is the future and they were the first in the market.

Can one game on it?

If one can skip buying gaming rig with a 5090 with its likely absurd price then this 3k becomes a lot easier for dual use hobbyists to swallow

Edit 5090 is 2k

I'm not sure it has a video out. There is a AI produced image of the digit and a monitor though.

It is not made for gaming and the form factor says a lot. With this being an option for AI there is less of a need to buy tons of nvidia RTX GPUs.

The 5090 surprised me with the two slot height design while having a 575W power budget.

Smart move for Nvidia to subsidise their ARM CPU and platform business by selling a big GPU packet with a CPU that most users don't really care about. Even if the margin is less than selling the raw GPU power would be (which I doubt), it'll look good on the shareholders conference if other business segments go up steep, too.

Little by little, we're getting an answer to the question: "What kind of investment does an outrageous influx of capitalization spur?" One might think it would be an AI silicon-moat, and it might yet be some of that.

But it's clear that everyone's favorite goal is keretsuification. If you're looking for abnormal profits, you can't do better than to add a letter to FAANG. Nvidia already got into the cloud business, and now it's making workstations.

The era of specialists doing specialist things is not really behind us. They're just not making automatic money, nor most of it. Nvidia excelled in that pool, but it too can't wait to leave it. It knows it can always fail as a specialist, but not as a kereitsu.

This seems surprisingly cheap for what you get! Excited to see what people cook with this

It's based on Mediatek CPU cores, so I am really pessimistic about their open source support...

I'm bracing for a whole new era of unsufferable binary blobs for Linux users, and my condolences if you have a non-ultramainstream distro.

The press release[1] says it's using NVIDIA's Grace CPU[1], as well as this:

MediaTek, a market leader in Arm-based SoC designs, collaborated on the design of GB10, contributing to its best-in-class power efficiency, performance and connectivity.

I assume that means USB and such peripherals is MediaTek IP, while the Blackwell GPU and Grace CPU is entirely NVIDIA IP.

That said, NVIDIA hasn't been super-great with the Jetson series, so yeah, will be interesting to see what kind of upstream support this gets.

[1]: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...

I'm buying one. It's cheaper then my 4090RTX+192GB of ram for more performance and model traning headroom. It's also probably a beast for data science workloads.

I am pretty sure memory bandwidth will be low it doesn't eat up their enterprise lineup. If we are luck we might get 512GB/S this is still half of 4090

With a bit of luck it'll mean some of the Jetson series will get cheaper.

While I'm quite the "AI" sceptic I think it might be interesting to have a node in my home network capable of a bit of this and that in this area, some text-to-speech, speech-to-text, object identification, which to be decent needs a bit more than the usual IoT- and ESP-chips can manage.

The monitor is AI generated in the product photo.... Nvidia please

https://s3.amazonaws.com/cms.ipressroom.com/219/files/20250/...

Running large language models locally could be the next major shift in personal computing, similar to how GPUs transformed gaming. The privacy implications of not sending data to cloud services are significant.

I highly doubt it's half or ever quarter of GB200, unless they have hidden water cooling or something outside. GB200 is 1200 Watts. Digits doesn't look like it would be above 200W, and cooling 200W would be impressive.

I think it's 1/40th of the GB200, but I think that's the one with two blackwells, so 1/20th of a full blackwell.

GB200 is ~USD 60,000/. So, it should be like 20th of that.

> It’s a cloud computing platform that sits on your desk …

This goes against every definition of cloud that I know off. Again proving that 'cloud' means whatever you want it to mean.

What are the CPU specs? Idk about the GPU but a really fast ARM cpu and a ton of ram and it already runs Linux?!! If it’s competitive with the M chips from Apple this might be my next box.

10 cortex-x925 and 10 cortex-a725 I believe.

Finally!

First product that directly competes on price with Macs for local inferencing of large LLMs (higher RAM). And likely outperforms them substantially.

Definitely will upgrade my home LLM server if specs bear out.

Does this also answer the question "What am I supposed to do with my old 4090 and my old 3090 once I get a 5090?" ie can we attach them as PCIe hardware to Digits?

Nvidia could potentially bring us all the year of Linux Desktop.

$3000? The GB10 inside it seems to be a half of GB200 which is like $60K. One can wonder about availability at those $3K.

No way is the GPU half a GB200. I'd expect something much lower end and power conscious.

They mention 1 PFLOP for FP4, GB200 is 40 PFLOP.

TheRegister mentions:

   Specs we’ve seen suggest the GB10 features a 20-core Grace CPU and a GPU that packs manages a 40th the performance of the twin Blackwell GPUs used in Nvidia’s GB200 AI server.

Would love to see something like this with an ATX form factor, socketed GPU, socketed GPU and non-soldered RAM.

Isn't that just a regular PC with one or more 5090 or equivalent workstation GPU?

$3000 seems incredibly good value

Newbie question: can you connect a RTX 5090 to the GB10 Digits computer?

Of course no chance of this with x86 because of market segmentation.

This Nvidia is quite a bit like the AMD Strix Halo, which has the same 128GB max memory, x86, unified memory, and a 256 bit wide bus. I've seen much hopeful speculation that the nvidia is 512 bits wide, but I'm dubious.

No chance because they don't have x86 license.

Feels like data center AI HW demand peak is over now that these things are trickling down to consumers and they are diversifying customers. Also going lower than expected on gaming HW, seems like they have enough fab capacity.

Their DC sales likely aren’t growing at the rates prior seen. Law of big numbers. They gotta diversify to satisfy growth & profit expectations

Do I need a "personal AI supercomputer"?

No but you want one regardless

Anyone know if it can run training/fine-tuning and not just 4-bit inference? Does it support mixed precision training with either BF16 or FP16?

Honestly surprised at how affordable this is, I was expecting $5-6k as I scanned the opening paragraphs.

Might well be, from what I can tell the price "starts at $3k", which might well be config'd like the minimum mac studio for RAM and storage. Mac studios easily hit $5k-$6k or more.

Pricing seems off!

Welcome to tomorrow's "personal" computer, a single unmodifiable SoC with closed source software stack.

That is the PC of today and yesterday and yesteryears.

This would be closer to a personal cloud. It's meant for other devices to connect with.

So exactly like the PC of yesterday then?

Like a mac?

...but will it run DOOM?

It costs the equivalent of 2 years of cloud GPU H100s at current prices.

Edit: Sorry fucked up my math. I wanted to do 40x52x4, $4/hr being the cloud compute price but that us actually $8300, so it is actually equivalent to about 4.5 months of cloud compute. 40 hours because I presume that this will only be used for prototyping and debugging, i.e during office hours.

what are current h100 price ? lowest i have seen is only 0.99 per hour

Would consider at a lower price of $500 USD, way too expensive for what it brings.

I think it's cheap at $3000. 128gb RAM, top of the line GPU capabilities, 4tb storage... it's much better than what a top shelf mbp can do and much cheaper

$500? that won't even buy you a decent new graphics card, anybody claiming this is overpriced doesn't have a clue.

Reply on news.ycombinator.com