Hacker News Reader

Comment by rememberlenny | Hacker News Reader

I got a eGPU too for machine learning work. I have a 1080ti which I put in the Akitio Node Pro. Its nice because it has a handle on the top, is portable, and sits on my desk at home.

I also have a desktop machine that I built with another 1080ti in it. I use the desktop machine as a remote machine for longer running jobs, but like the convenience of running files directly on my machine.

If anyone has questions, I'm happy to answer.

View on HN

I heard the eGPU gets throttled memory wise because there are less pcie lanes. does this have a big impact on ML performance?

I would hope not; any reasonable implementation would try to keep as much data on-board or on-chip as possible.

Edit: This thread is helpful. https://www.reddit.com/r/MachineLearning/comments/6tyyyt/d_d...

Yes, but the bulk of bandwidth should be eaten up by intermediate results rather than the input data e.g., for imagenet, 224x224x3 tensors are tiny compared to the intermediate activations that have to be moved around (and saved during training).

Hopefully you would also only copy your model over once.

I see, but I think for areas like Reinforcement Learning or sequence models though eGPUs would be significantly slower since you have to constantly shuttle new data into the GPU from your computer.

It gets 70-80% performance. I’m not sure about throttling. It’s definitely not 1-to-1 based on my desktop, but it’s so much faster than the CPU.

When connecting on Windows the eGPU seems to be using a PCIe x8 slot which may limit the GPU.

Do you have a blog post or could recommend one that covers your setup? Particularly any software that needs to be installed to get ML setup properly. I’m putting together an eGPU together in the near future and have read quite a bit, however any pitfalls that could be avoided would be helpful to know up front.

I don’t have one, but based on your request - I’ll write one.

The biggest issue is OS X version limits. I’m on a version two points back due to the NVIDIA hacks needed. There’s a whole community forum on egpu.io that goes into details.

Reply on news.ycombinator.com