Hacker News Reader

Comment by cbozeman | Hacker News Reader

I dunno... $8000 builds a 64c/128t 256 GB RAM workstation with the same GPU these researchers used (https://pcpartpicker.com/list/P6WTL2). That's arguably in the realm of home computer for just about anyone making $90,000 and above, I would think; I would also think anyone working in those fields could command at least that salary or greater, unless they're truly entry level positions. Seems it would be a reasonable investment for someone actively working in the area of machine learning / artificial intelligence.

View on HN

They used an Nvidia 2080 TI, which is expensive today, but rumours are that the entry level 3XXX series will be similar to the 2080 TI in performance. That should be dropping in about a month!

Anyone that can afford a car can afford this.

The interesting places you could take it!

If you don't shoot for top end, you can get get a rocking machine for under $1600: https://pcpartpicker.com/list/DGhwBZ

12c/24t, 64G with a high end video card.

In a ML setup your GPU is going to be the workhorse. I'd spend less on the CPU and memory and spend more on the GPU. Throwing something together REALLY quick (I'm being lazy and not checking) I'd go something closer to this https://pcpartpicker.com/list/T9Ds27 because of the graphics card. But getting a Ti would improve a lot just for GPU memory. You could get a lot done with a machine like this though (I assume it would be good for gaming too. Disclosure: not a gamer).

tldr: upgrade GPU, downgrade CPU and ram to keep similar pricing.

This depends quite a lot on the domain. In some image processing tasks you can actually be cpu-bound during dataloading. So either you get tons of RAM and preload the dataset, or you use more cores to queue up batches. You still need a good GPU and generally I'd agree to prioritise that first.

You can get CPU bound, I am $100 under and you could put money towards that or RAM. I did also leave a path to upgrade RAM. But that said, I've been working with image generation a lot lately and CPU really isn't my bottleneck.

If doing image processing (or NLP with rec nets), I wouldn't save on VRAM. 11GB minimum (2080/1080TI). Otherwise you can't even run the bigger nets with good image resolution.

I think the more important cuts that went unmentioned are the CPU cooler and mobo. You could arguably cut the CPU cooler completely, since these Ryzen CPUs include one.

The mobo cut is also a pretty useful savings, though it will be an obstacle to multiple-GPU setups.

Also, the parent comment misleadingly suggests that a 3900X costs less than $300. That seems like an error in pcpartpicker, since clicking through reveals a true price of $400+.

what's the per-hour cost spot price of this machine on AWS ?

well an m5ad.24xlarge is 96 threads and 384G with your own 2xSSD (saves on EBS bandwidth costs). So fewer threads but a bit more memory. (we'll guess that is a 48 core EPYC 7642 equivalent with 96 threads since there is no 96 core version)

That bad boy costs $0.96/hr on the current spot prices (https://aws.amazon.com/ec2/spot/pricing/)

How does having your own SSD save on bandwidth? You have to send the data to and from the cores where the computation is taking place.

There are certainly some workloads where it makes sense to own your own storage and rent computation, but you can't assume that by default for a "powerful AI" workload.

It saves on bandwidth $ costs to EBS (AWS block storage)

That instance type is missing the RTX 2080 Ti-equivalent GPU, though...

The closest in performance to that GPU would be the V100 found in P3-instances.

Yea which are $3 / hour. Or, you could, shameless plug, use Lambda's GPU cloud which has V100s for $1.5 / hour!

https://lambdalabs.com/service/gpu-cloud

I'm embarrassed that I feel so compelled to post this--on a Friday night at that--I apologize.

> I'm embarrassed that I feel so compelled to post this--on a Friday night at that--I apologize.

Don't be. Finding gems like this is why some of us read the comments.

Frowning on self-promotion got us in those silos.

There is no exact cloud equivalent - the researchers used commodity hardware for their GPU, something that NVIDIA doesn't allow for use in data centres.

The closest you can get on AWS (more like System #3 in the paper with 4x GPUs) would be something like a p3.8xlarge instance [1] that'll cost you $12.24 (on demand) or $3.65 to $5 (spot price, region dependent) [2].

A single GPU instance (p3.2xlarge) only 16 vCores, though) will cost you $3.06 on-demand or $0.90 to $1.20 (spot).

[1] https://aws.amazon.com/ec2/instance-types/p3/ [2] https://www.aws4less.com/AWS/AWSCurrentSpotPrice.php

around 3-4 months on AWS would equal buying hardware on the spot.

The cost of a similarly spec’d machine on AWS is 0.73 kidney/hr

Reply on news.ycombinator.com