Hacker News Reader

Have you taken a look at the "Zero to JupyterHub with Kubernetes" guide? https://zero-to-jupyterhub.readthedocs.io/en/latest/index.ht...

Alternatively, you could check out the QHub project from Quansight. https://qhub.dev/

I think these are less relevant to you since you already have in-house k8s resources, but Coiled and Saturn Cloud offer services that might be interesting to other people with similar goals but who don't have a kubernetes cluster already.

https://coiled.io https://www.saturncloud.io/

How about using Kubeflow? I am not a user of Kubeflow, so might be misinterpreting it. But here is my understanding.

Requirement: Use the notebooks used on laptops in production. Be able to scale as needed. Multi-cloud.

Kubeflow: Kubeflow basically breaks down the notebook into chunks of images, which run as separate pods. These pods are scheduled by Argo workflow - called pipeline. Kubeflow serving enables you to expose the note book as an API. You can schedule different pods w/ different specs, including GPU.

If that meets your need, then you need a simple way to run on laptops - look for miniflow.

There is a tool that is translate from notebook to images directly, so your data scientists do not need to write YAML. Look for miniKF + Kale.

Hope this helps.

So I am not sure that I understand the question. I.e. I would like to separate the online notebook development (which is done against jupyterhub) - I.e. an interactive session against a container, from the batch job execution of the notebook. Usually you would like to convert the notebook to python code in the batch case.

Are you looking for EXECUTION of the notebook on a cloud VM? I.e. as a k8s job?

thanks for replying, just updated my question. What I would like to do is create Notebooks for my Data Scientists using JupyterHub and have each JupyterLab notebook in K8S, when needed I can allocate more resources per user, instead of having dedicated VMs in Cloud or in their workstations where managing environments is painful.

Ok I understand. So you can create a JypterHub container , however, I assume that the notebook is stored locally to the container. So you would want to attach PVC and PV to each container and make sure that the notebook are saved to the PVC each time you resize the container (Since K8s will restart the node after resize).

Also note that (I think) you cannot share GPU between Pods. So you might want to have different JH containers inside the same Pod (with different ports).

We were in almost exactly the same situation. Workstations at the office. GCP account. We started pushing for more remote work about two years ago, and we built our internal machine learning platform (https://iko.ai) around our workflow solving one bit after the other, precisely because we didn't want to be tied to one library/framework, and without having to pollute our notebooks.

Here's what we're doing:

- No-setup, fresh, notebook environments with the most popular libraries pre-installed: this saves a lot of time, avoids having environments that break, etc. We start afresh with large Docker images, and then people can install libraries.

- Real-time collaboration on notebooks: so we can troubleshoot and pair-program on the same notebook at the same time, we can see cursors, selections, changes, etc. We also use that for our weekly calls where we have the agenda, code snippets, etc. all in the same place. We can talk through agenda items, edit in real-time, add snippets of code, and brainstorm.

- Multiple versions of your notebooks: there's a regression in JupyterLab that we fixed. We're working to make it available in upstream JupyterLab[0].

- Long-running notebook scheduling with output that survives closed tabs and network disruptions: we select the Docker image and the output file. This solves the common problem of launching a long-running notebook, and having to keep the browser tab open and hope the network connection doesn't break. Some people circumvent that porblem by scheduling the notebook and saving the artifacts right from the notebook, but we wanted to be able to have the output streamed whether the tab was closed or not. Bonus: we can watch the same notebook running on multiple laptops.

- Automatic experiment tracking: automatically detects your models, parameters, and metrics and saves them without you remembering to do so or polluting your notebook with tracking code.

- Easily deploy your model and get a "REST endpoint" so data scientists don't tap on anyone's shoulder to deploy their model, and developers don't need to worry about ML dependencies to use the models, and be dragged into the "ML realm". The models also have a page where you can invoke the model by entering JSON and get predictions, or uploading a CSV file and get predictions.

- Build Docker images for your model and push it to a registry to use it wherever you want: currently we push to DockerHub and GitLab

- Monitor your models' performance on a live dashboard: requests, latency, performance, etc.

- Publish notebooks as AppBooks: automatically parametrize a notebook to enable clients to interact with it without exporting PDFs or having to build an application or mutate the notebook. This is very useful when you want to expose some parameters that are very domain specific to a domain expert. For example, some features in a nuclear engineering problem are very domain specific, and a nuclear engineer can bring in a lot of value tweaking the parameters themselves. It also spares our people from spinning up yet another VM on GCP, loading the model, creating a Flask application, setting authentication, etc. The AppBook runs are also tracked, of course.

Much more on our roadmap. We're only focusing on actual problems we have faced serving our clients, and problems we are facing now.

The notebook servers are run on an arbitrary cluster that happens to be ours for now, but we're pretty much approaching it with the following mindset: we are a company that uses this platform, and the platform just happens to be ours.

- [0]: https://github.com/jupyterlab/jupyterlab/issues/5923

So what happen when a notebook on a given container needs more resources, how do you handle that ?

We specify resources as parameters to limit usage both for notebook servers, and for long-running notebooks we schedule[0][1].

In other words: for notebook servers, we can select the resources we want when starting a notebook, say 5GB of RAM for exploring something and dry-running some code on a sub-sample. Then we schedule the notebook with more resources (GPU, RAM, etc). We can set the timeout to kill the job.

We also shut down the notebook servers automatically, and use resources only for the duration of the long-running notebook when they're scheduled.

OP might want to look into JupyterHub spawners[2]. They can use the API to make custom spawners that fit their specific needs (handling storage, using specific images, etc).

One limitation in ZTJPH in one comment above if I recall correctly is the locality of the cluster. OP also might want to take a look at JupyterHub Enterprise Gateway[3] which may solve their need.

It depends on the experience OP wants to provide for their users. We wanted to let users roam around, have files, use the console, collaborate in real-time, load data, and fresh environments... Similar to what they'd have on their own laptops, except with a better experience. Not just "one notebook view" or a rigid pipeline.

That's what we disliked in most of the products we've seen so far. That and the fact many of them focused way too much on pretty stylesheets and animations instead of actual problems ML people struggle with.

- [0]: https://kubernetes.io/docs/concepts/configuration/manage-res...

- [1]: https://docs.docker.com/config/containers/resource_constrain...

- [2]: https://jupyterhub.readthedocs.io/en/stable/reference/spawne...

- [3]: https://jupyter-enterprise-gateway.readthedocs.io/en/latest/

JupyterHub in Kubernetes