Hacker News Reader

Comment by habitue | Hacker News Reader

But not, apparently, ahead of what you can do with 8 consumer GPUs

A $3k Tesla K80 GPU is not 'consumer', nor do most consumers (or most researchers or small businesses) have $24k to drop on a set of GPUs alone.

You're right - a $3k k80 is generally inferior for most DNN applications than a $1k Titan X. The primary reasons that big companies use K80s have to do with achieving high computational density, and licensing issues, more than the performance of a single GPU. Sticking 8 Titan X boards in a machine is a bummer job if you want to pack them closely together. But for academic researchers, a quad Titan X box is pretty solid and quite affordable.

A quad Titan X is still $4k for the GPUs alone, and was only possible in the past few months - people might have wanted to get stuff done in the years in between the last generation and the current generation...

Ah, sorry I missed that they were tesla gpus

>On WMT En→Fr, it takes around 6 days to train a basic model using 96 NVIDIA K80 GPUs.

Then they use custom ASICs to actually run the thing in production.

> The training setup follows the classic data parallelism paradigm. There are 12 replicas running concurrently on separate machines. Every replica updates the shared parameters asynchronously.

So it's actually 96 GPUs for a model. And they have many languages to train, and probably don't use cheap GPUs, either.

If it's just expensive to train the model, I wouldn't expect that to stop Google. They don't have to do the actual training that often, after all. Right?

They would have to do that for each language (or language pair), but they can afford it. I think the training the network is cheap compared to running it for user queries, at Google scale.

"On WMT En → Fr, it takes around 6 days to train a basic model using 96 NVIDIA K80 GPUs"

Notice that the actual Google Translate uses a much bigger dataset than that.

That would make the cost about $15..30K for a model.

Reply on news.ycombinator.com