Show HN: Dstack – an open-source engine for running GPU workloads | Hacker News Reader

Show HN: Dstack – an open-source engine for running GPU workloads

dstack.ai

101 pointsAsDivyansh18 comments

Thanks for sharing dstack on HN. I'm the creator of dstack and would love to hear your feedback. We're also working on creating a new standard for GPU orchestration, similar to Kubernetes.

Any chance you'll support "traditional" batch HPC environments like Slurm?

Yup, that’s what we are building now. Tasks will allow to specify how many nodes you need, and dstack will automatically provision a cluster or will run it using the one that is already provisioned. Please feel free to ping me on our Diacord if you’d like to test an early version!

> Thanks for sharing dstack on HN

Seems like this is wrongly tagged as Show HN. @dang, please fix the title after confirmation.

From the screencast on the homepage and in the docs [1], I see Google and Azure clouds are supported as targets.

Is there a list of other targets? Is Runpod and vast.ai supported for example (since they're more focused on GPUs)?

1: https://dstack.ai/docs/concepts/dev-environments/#define-a-c...

Vast.ai is supported. RunPod not yet but on our list! The full list of all supported targets can be found at https://dstack.ai/docs/installation/

> We're also working on creating a new standard for GPU orchestration, similar to Kubernetes.

What about improving GPU orchestration on Kubernetes instead?

To be frank, I’m not a fan of Kubernetes as long as GPU is concerned. Kubernetes has a very large legacy. Why we think dstack can do better: 1. Lightweight-ness - that it very easy to integrate dstack with modern cloud GPU providers. 2. AI-friendly interface built-in - no need to things like KubeFlow and alike.

FTR, we integrate dstack with Kubernetes too - already [1] But our native cloud integrations can be a lot more efficient. For example, when it comes to auto-scaling - this part is in work

1. https://dstack.ai/changelog/0.15.1/

This looks really well done. Feels like a Hashicorp product -- does one thing well, abstracts across cloud providers and provides a tight DX.

Kudos!

Thank you! Yes, that's precisely our intent. Most AI platforms offer nice yet proprietary solutions for training and deployment. We wish there were a simple and open standard that any provider could support.

I remember Andrey (the founder) leaving JetBrains to start this. I'm glad he stuck to it, and that it is slowly finding a space in developer discourse. I'm still not sure I totally understand MLOps yet, but then again I don't do a lot of ML workloads.

Super useful for spinning LLM fine-tuning jobs!

Looks really amazing! Thanks dstack team :)

great to see this project coming up so well. I had a chat with Adrey years ago. Congratulations!!

Congrats! Looks great

Looking great, thanks for building this

"Embrace Dev Environments, Leave Manual SSH Behind" ?

This is from https://dstack.ai/blog/2023/06/29/say-goodbye-to-managed-not... Yup, that's why we support dev environments as first-class citizens. Now, also support tasks (for fine-tuning, other batch jobs) and services (deployment, incl. OpenAI compatibility for LLMs).

Reply on news.ycombinator.com