404 karma · joined December 11, 2015
Now I can ask it to do some frontend while I focus on backend in the meantime.
We just need the sales agent now.
It would be good if someone made a fork with fixed setup and docker images for self-hosting :)
- GKE kubernetes - Managed Postgres (CloudSQL) - GCS buckets for file storage - Cloudflare for DNS - Countour for ingress - Keel for automating deployment updates - Mailgun - Sentry (errors) - Node-RED various little automations such as healthchecks
Planning to introduce Elasticsearch too, so far I have been testing the operator and it seems to be pretty good quality.
From maintenance perspective GKE is great, as long as the bills are paid, no need to login there pretty much ever. As a one man company this means a lot, I try to never spend any time on ops.
And https://www.goodreads.com/book/show/40376072-children-of-rui...
Lots of fun reading them, great author :)
- CLI/UI.
- Main backend for storing projects, user accounts, managing pull requests, forks, runners, deployers, loadbalancers.
- Data backend (dotmesh).
- Auto provisioning of VMs with jupyterlabs running and data synced to GCP, AWS
- Runners that configure environment, install dependencies open up tunnels so users can access them and start working.
- Optimized machine imagine builds so the startup takes ~1min (some of the docker images like jupyter lab are very big)
- Model packaging into docker images.
- Model metrics capturing (a proxy that runs as a sidecar and intercepts requests) and then attaching relevant classes for your models.
- Kubernetes operator to deploy the actual models. User didn't have to worry about creating deployment manifests, services or ingresses (they wouldn't even care about docker images). They would just say which model to deploy and they would get a URL. Models could be deployed in a k8s cluster built from nodes with spot instances so would run pretty cheap :)
- Last component that I worked on was probably one of the most fun - an inference router that could allow canary deployments for models and also shadow deployments where traffic is sent to many models at once but responses are taken only from primary. We got a really nice UI for this as well where you could drag sliders around to configure % of traffic and so on. Unfortunately never managed to write docs for this.
- Terraform to wrap everything and deploy to GCP/AWS.
Our team was always quite small so we were stretched thin. In the end sales were going well as well, probably 6 more months and we would have broken even and then profitable :)
I guess "Git for data" is not very useful if you don't have the whole platform built around it to actually use the features. We mainly use it for data synchronization between the nodes and provenance tracking so people can see what data was used to build specific models and to track how the project evolves itself without forcing people to "commit" their changes manually (as we have seen that often data scientists don't even use git, just files on their Jupyter notebooks).
This is mostly fuelled now by feature requests where people are integrating forms and other services to report covid cases. Trying to help them for free and ensure they don’t have to pay for service as they are non profits. Partner not happy about these late nights:)