Comment by dsacco | Hacker News Reader

Comment by dsacco | Hacker News Reader

I think there should be a disclaimer here that while, yes, this will work, it will quickly become suboptimal (perhaps even dysfunctional) for a lot of work as you scale your hobby into something resembling more professional work. I personally would not start with a thousand dollar machine if you have any intention of doing serious computation in the future. While you can trade out parts pretty easy on a custom built machine, you don't want to be in a position where you have to trade out most of the core parts 6 months in.

For reference, I built a home PC that I successfully do deep learning and data analysis on (mostly tensorflow and scipy stack) for about ~$10k. It's liquid cooled, has 15 fans, four radiators, an i7-6900K CPU, 128GB RAM, four GTX 1080 GPUs (controversial), four TBs of HDD space and 1TB of SSD space. I don't recommend you start with this at all, but my point is that porting your hardware from point A to point B will be a pain if it comes to it.

I used the guide here as a reference about 8 months ago when I built it: http://graphific.github.io/posts/building-a-deep-learning-dr.... My purpose in doing this was, essentially, to pay for electricity rather than AWS/GCP/Azure compute resources (and in that regard it's been very successful!).

I know I'm hijacking a thread here to talk about building home machines for professional deep learning work when this story is clearly not intended for that, but I wanted to throw in this perspective so that it's understood this is very different from just "build this machine to start out and upgrade it later." There's a law of diminishing returns here, but in general my point is that I do not think this is a minimum for "start doing deep learning effectively at home." If you want to learn hands on deep learning cheaply, my opinion is that it would be more efficient to use compute resources from a cloud provider before diving into this with a home-based custom machine.

tl;dr: The demographic of folks who probably want/should/need to build a home deep learning machine probably has little overlap with the demographic of folks who want to do it non-professionally, or at least with only $1k in resources.

On the other hand, by the time anyone starting just now learn enough ML to be dangerous, the price of the $10,000 hardware would drop to $5,000 or less, so starting with the cheapest yet decent option seems a better approach...

Fair point. I'm just quibbling about where "decent" begins, and adding in the point about ease of upgrades. Replacing a GPU is pretty easy. Replacing a motherboard because you don't have enough PCIe slots will require tearing down the majority of the machine and rebuilding it.

The latter is more likely to happen if you start down this path at rock bottom prices.

I began building one a year and a half ago, starting with a 5930k, 1x980ti, 32GB. This put me in a position to incrementally upgrade it starting at ~$2.5k. That being said, at this point the smallest AWS p2 instances are equivalent to my current setup, and it would take 150 days worth of running my hardware to make it cost-effective vs. AWS.

When you build your first DL box you also get a moral boost. If you think every second is counted and penalized against you, you might use it less (such as when you're using cloud servers). But if it is already yours bought and paid for, then you can afford to run any crazy idea. A good compromise would be a local DL box and extra virtual machines on demand, for when you need to run extensive hyperparameter searches. If you make the leap and invest, at least you commit your money into the direction of ML, strengthening your resolve.

On the flip side, dropping $2.5k for a hobby you haven't investigated yet and then looking back 6 months later on the computer collecting dust will likely dissuade you from investing any time into potential future hobbies. I play piano, and I'd never encourage anyone to go drop $2.5k on an electric keyboard just to provide a motivation. A decision to pick up a new hobby/skill should be based on desire, not a handcuff.

I wouldn't recommend 4 GPUs unless you are very confident the algorithms you are going to run will parallelize well, or that you need to run extensive parallel trials for various hyperparameters. It takes special effort to make use of multiple GPUs.

Serious question, why build a 10K$ machine like this rather than just spinning up some AWS instances? -edit- I know you mention wanting to pay for electricity rather than AWS. But that doesn't necessarily make financial sense.

Many prioritize protecting their data, eliminating the possibility of using the cloud. This is usually a business use case though rather than personal.

This is correct. I handle my own (daily) backups as a result, which are end-to-end encrypted from the machine.

If it's a business use case severe enough that you can't trust AWS, you aren't going to wast time scraping together a cheap $1K or even $10K build while your $1K-$10K/day team is idle. You'll go looking for accessible powerful systems, and be willing to replace hardware as needed.

This rationale is exactly what the AWS team has seeded in the community. I think it's time we quit parroting an inevitable conclusion that all things will and must go to the cloud.

So what you're saying is that if you start with the thing in this article, and then want to upgrade to the thing you described, don't expect it to cost $9k, it'll cost $10k :-)

(and as a bonus leftover: a nice machine to donate your local volunteer hackerspace, youth tech center, school etc etc)

I don't see the problem, as long as it's roughly 9-10x faster for an embarrassingly-parallel task of larger size, that 10% isn't going to make a big dent, is it? :-)

so do you earn money out of that 10K investment on machines? I didn't read the article but what's the benchmark on deep learning machines? Still TFLOPS?

Yes, and yes.

How do you make money off a deep learning rig? As a consulting data scientist?

Are multi-GPU systems worth it until you have a TON of experience building parallel models? Almost all of the reference architectures I've seen are essentially serial, and TF doesn't have good data-parallelism built in to make use of N GPUs, N > 1.

Maybe this is a stupid idea, but if you had two GPUs, doesn't that basically mean, in terms of good parallelism, that at least you can train two models/networks at the same time, roughly?

Because as long as you're in the research and development phase, that'll help cut your coding/training/testing/adjusting cycle. I assume you'll be tuning your hyperparameters, perhaps on somewhat smaller test models, but they will still take half a day or so to train? That means you can try (almost) twice as many hyperparameter configurations in the same time. It still helps to spin up a second test with a different selection of parameters, even if you haven't gotten the results back from the first test, right?

Can you share your PCPartPicker list? I'm curious how the costs breakdown.

Reply on news.ycombinator.com