Automate deep learning training with Kubernetes GPU-cluster | Hacker News Reader

Automate deep learning training with Kubernetes GPU-cluster

github.com

57 pointsLanghalsdino14 comments

If the author is going to use GCP why not use their `container engine` [1]?

[1] https://cloud.google.com/container-engine/

You can't load your own kernel modules (or generally bring your own base image) with Container Engine, and that includes NVIDIA's driver (nvidia.ko). We can't distribute nvidia.ko because the linked artifact is ultimately GPLv2 (like any Linux kernel module).

Given the above, today your only option is DIY.

Disclaimer: I work on Google Cloud, but IANAL.

thanks, that's good to know.

I know you said you're not a lawyer, but.... Linux is GPLv2, right? So why is it ok to distribute Linux, but not the NVIDIA driver?

You can distribute GPL things if you make the source available. The source for Linux is freely available. The source for NVIDIA's entire driver is not (the shim layer that actually interacts with Linux does have the source code in the .run file, that's how you build it yourself).

God this makes so little sense.

Do y'all have any way of tracking feature for custom images and machine types for GCP?

Cloud ML feels not nearly ready, and but being able to do something like this on GKE would be magical.

This submission got a bunch of promo-votes and booster comments. This (especially the comments) is the worst thing you guys can do to help your friends. HN readers can smell it a mile away and then they get mad and use unkind words like 'spam'.

Since it looks like a good submission, we've turned off the penalties that otherwise would apply. But submitters, voters, and commenters: please don't do this!

How do you distinguish between Kubernetes + Deep Learning hype and promo-votes?

Fwiw, it seems like the person is new here (and should have done Show HN given that the author submitted it!), so I can imagine them passing it around with "please upvote". Is that what you're referring to with promo-votes?

Yup. And it's true that new users often do this without realizing how much the community dislikes it. That's why we go easier on them, especially if the work is good.

I am really sorry for what i have done. I will learn from my mistakes.

Thank you for being so kind to me!

Looks awesome!

Very interesting read, will check this out soon.

Thorough anaylsis of scaling up via Kubernetes!

Reply on news.ycombinator.com