DeepSeek-VL2: MoE Vision-Language Models for Advanced Multimodal Understanding | Hacker News Reader

DeepSeek-VL2: MoE Vision-Language Models for Advanced Multimodal Understanding

github.com

36 pointsselvan7 comments

> Note: You may need 80GB GPU memory to run this script with deepseek-vl2-small and even larger for deepseek-vl2.

Is this a typo? 80GB VRAM? I make great money, but there's no way I'm talking myself (or my wife) into spending 15k+ on an AI rig. I feel like I'm 16 again, pining after video cards I cannot afford or justify.

How did you arrive at 15k? For personal use, it's common to get aftermarket 3090's, link them up four-way (96 GB is not half-bad) and put it in some server motherboard with as many DDR5 channels as possible... imho in 2025 this is a total waste of time, but it's possible, and at a fraction of your estimated cost... Seriously though, H100's go for $2/hr these days. I personally spot g6e.48xlarge (384 GB) which goes for under $1. Why bother hoarding many years out-of-date hardware which is like running three washing machines all day...

> I personally spot g6e.48xlarge (384 GB) which goes for under $1.

How? I see this around $30?!

My mistake! I used to get 48xlarge in eu-central-1 for $2/hr but it could be luck for all I know, seems Spot pricing for these instances are fluctuating considerably: it's at $8/hr now.

https://aws.amazon.com/ec2/spot/pricing/

Absolutely not $15k, you can go and buy a 2 year old MacBook with 96GB of unified memory (around 85GB usable for the GPU) for a few thousand USD, or a brand new 128GB version for $4500 USD~, or grab 4x 24GB GPUs second hand etc... I personally have a 96GB MacBook Pro from 2023, and my home server has 80GB of vRAM (2x 3090, 2x A4000). It's not as uncommon as you might think if you work in tech.

That’s your H100 and A100’s GPU memory, not too sure but I imagine many AI practitioners would not find that a weird number

Definitely not a typo.

Also:

> # vl2-tiny, 3.37B-MoE in total, activated 1B, can be run on a single GPU < 40GB

Reply on news.ycombinator.com