Hacker News Reader

Comment by andy_ppp | Hacker News Reader

I was under the impression things start getting doable at 2 dedicated Nvidia GPUs (preferably with 1tb of ram each)?

1 decent GPU is usually fine for reasonable things. But the memory is way off: the gpus I use have 40GB of memory (a cluster of the higher end Nvidia A100s) and they're basically state of the art. Most gpus near sota are in the 16GB range and up.

Yeah, I personally like using an Nvidia Jetson AGX Xavier.

There are some cool packages out there, that detect your emotions and attentiveness while driving.

Aside from the Jetson, I have an 2080 Super on my laptop.

1TB? Isn't it more like 12-64GB?

It was clearly an exaggerating joke.

That's not clear at all, if it was a joke. It definitely doesn't come across as a joke, but perhaps just someone who isn't up to date on gpu memory, which is a fairly niche subject.

One would have to be pretty out of the loop to think that GPU memory was in the terabyte range, though.

Enterprise grade workstations and servers can be configured with 1TB of ram though, so it's not completely inconceivable that you could have a dedicated GPU workstation with 1TB of graphics ram. I mean the Tesla A100 cards have 40GB of VRam each so you'd only need 26 of them to have 1TB of VRAM in a single machine. While I have no idea if such a machine exists, it certainly could exist, 26 GPUs wouldn't fill even half of a server cabinet.

Edit: actually Quattro A6000 cards ship with 48GB vram each, so you only need 22 of those to have 1TB total.

Here you go friends

> Radeon Pro SSG (2016) https://www.amd.com/en/products/professional-graphics/radeon...

That's a GPU with a terabyte SSD on it, not a terabyte of VRAM.

It would be really interesting if micron or intel collabed to make a gpu with a ton of 3d xpoint.

I'm not super knowledgeable in machine learning but given that system RAM (not VRAM) in the TB range is really not that rare these days for high end workstations I assumed that the poster meant that you'd have 1TB of working RAM per GPU for the learning job (which would be paged in and out of VRAM as necessary).

I guess the lesson to be learned here is that if you want to use an implausibly huge amount of RAM to make a point, a TB is not safe anymore. Go for Exabytes instead, that should be unambiguous for a couple of years.

Haha, that might just be the right takeaway!

I thought it was but because 1TB is not too far out of the park, it would he been more obvious if something like '1 exabyte gpu' was used. Then, at least for a few years to come, it is indeed crazy talk.

Reply on news.ycombinator.com