DwarvenEngineer

wdym? Honda's hyrid technology is very different from what Toyota has architecturally

does that mean they're measuring bandwidth differently than how others (like nvidia) does it? memory bandwidth is the gating factor of running models locally, so if it's actually 8x 150GB/s, it may help something like prefill, but would it actually speed up decode comparatively?

Ironically for home-lab users (ie those who primarily have access to consumer hardware), apple silicon is probably the best value (and also the most obtainable for running some of the smaller sized models. You can get unified memory of up to 192 GB in the mac studios. The equivalent for the "open market" is probably AMD's halo strix which offers up to 128GB unified memory, and the speed of the memory is much slower.

I guess the Nvidia Jetson AGX Thor also exists, but good luck getting your hands on that... and regardless that's also 128GB.

Put it this way, with 192 GB VRAM you can run deepseekv4... with 128gb only with lots of quantization

Edit: WTH! I was curious and i went to apple's website to price out a high end unit just to price comparison, and it looks like the high ram units are gone now, and they max out at 96GB?? Guess i missed that...

Honestly, I just hate the term "physical AI". They're robots. It's unfortunate that we had to adopt a term with the words AI in it, just to get investor's attention.

yeah, i'd say so. what if lyft's lidar manufacturers said that they should be included in this release because it's their lidars? I agree, it doesnt really need mentioning