Hacker News Reader

Comment by mk_stjames | Hacker News Reader

I recently saw a rack server of 8x SXM2 P100's (so, the 2016 equivalent of this) for sale for $3600 on ebay. And you can buy individual P100 pci-e versions for about $300 now. They retailed for about $7600 each back in 2016.

I wonder how long it'll be until A100's depreciate that amount.

I see articles from "Serve The Home" pop up now often and I've yet to see one discussing hardware that I've ever come across in... someone's home.

View on HN

STH has been around for 14 years and so we have tested these big servers from the P100 days (see our DeepLearning12 build) and newer. Home was the /home/ directory in Linux.

We do have some content for homes. Check out our TinyMiniMicro series as an example.

>> Home was the /home/ directory in Linux.

I have never (until reading this) thought of it in that context. Was that the original meaning for the name, or did it shift over time?

It is in the "About" page :-)

I always get confused by why folks ask why we have server content. The Wall Street Journal covers more than just one road in NYC.

Understood. But what about this? https://web.archive.org/web/20110201124803im_/http://www.ser...

Now that is a throwback! That old logo was created with GIMP at a table outside NetApp. STH started at a time when the SMB market (and VERY high-end homes) were using Dell, HP, and IBM rackmount gear off-lease. The original STH idea came from me learning Linux as an alternative to Windows for that segment. STH was what I used to learn the rest of the market outside of work. The first product we were sent for a review was a rackmount case and that started us on the path of reviewing new gear ~2010-2011 when it was a part time blog instead of having a team of folks working on the site. I still try to keep 15-20% of our content in the SMB/ high-end home arena.

Direct link: https://www.servethehome.com/introducing-project-tinyminimic...

Those P100s are so cheap because they're more trouble than they're worth. They don't mount easily in most cases, will consume a lot of power at idle, generate lots of heat without active cooling and require a Linux machine to use properly.

It's probably possible to engineer a home-scale inferencing network like that, but I've had plenty of success just running GPT-Neo on off-the-shelf ARM chips. It's not worth the extra $800, exploding power bill and engineering effort to get diminishing returns.

Oh I know for sure, it's still just amusing to me to look up these just slightly older cards and see them going for so little compared to what they were new;

I do also look at various benchmarks and comparisons over and over because I would like to build a home gpu server for some machine learning fun and I find it interesting that the TFLOPS/$ numbers always come out very much linear... A card that gives 2x performance costs 2x more.... if cards depreciate 50%, it's because new cards have been launched that have double performance for the same price... etc... Even simpler, the CUDA cores-per-dollar is pretty much a constant.

What isn't taken into account there is various optimizations new cards have that speed up training or inference, like better use of the tensor cores. At some point incompatibility with newer CUDA compute levels makes the older cards much less useable.

Not directly applicable for large language models, I was doing some searching for cards with lots of double-precision calc performance. Nvidia has been leaving that out of all of the consumer versions of hardware; cheap P100's seem to be by far the highest DP FLOPS per dollar solution available right now.

I have an interest in doing lattice Boltzmann fluid dynamics simulation on GPUs, and this is something I've noticed with the P100's as well. That and memory bandwidth is super important for such calcs, along with total memory. Commercial codes I've seen aren't using any ML ops compute optimizations or tensor cores and anything like that, it's mostly just OpenCL based and thus the P100s throughput per dollar is pretty high.

MI25

Faster, cheaper, crippled by AMDs poor software support.

Agree, I have 14 V100's in my home but its pretty rare. I use it with fiber backhaul to supply services for marketplaces like vast.ai

With residential power costs, is it economically viable to do this? What's your utilization look like?

In Washington power can be 2.95¢ per kWh so yes, very!

From the utility? That's crazy.

Yep! Actually 2.7c: https://www.chelanpud.org/my-pud-services/rates-and-policies

Where? Is there also 1G or better 5G+ fiber?

You can get 1Gb, 2GB and 5Gb fiber to the home from ATT, TDS, Google etc. for backhaul.

I mean in the area with 3c/kWh electricity that the above poster mentioned.

Chelan county: https://www.chelanpud.org/my-pud-services/rates-and-policies

Dang, you are making me deeply regret not buying a house on the lake there a few years ago.

It’s a beautiful place too. It’s never too late ;)

They even have community fiber! https://www.chelanpud.org/my-pud-services/residential-servic...

Depends on how many marketplaces and the performance of the entire rig, power is $0.139kwh where I am at.

"I see articles from "Serve The Home" pop up now often and I've yet to see one discussing hardware that I've ever come across in... someone's home."

You're just not visiting the right homes ...

it might land in someone's home...after 7 years of launch like the p100s

Reply on news.ycombinator.com