OneDiffusion
github.com
github.com
So this library abstracts away BentoML with a custom API. BentoML itself abstracts away its own internal framework. That inner framework abstracts away HuggingFace diffusers with a different API. And HF diffusers itself is a rather huge library that abstracts away using PyTorch to load the model and run it. HF diffusers contains lots of useful things for designing and training models, but none of that makes it through the 3 layers of API changes introduced here.
I predict that debugging any kind of production app using this is going to be pure hell. You'll have 30 function deep call stacks across millions of lines of code. And that's especially sad since the alternative is probably a single 300 LOC python script calling PyTorch directly.
I don't think calling it abstraction is correct. OneDiffusion is designed to be opinionated and help users to run diffusion models easily. It includes best practice, default options, optimisation baked in such that it helps developers to move faster.
You can also make that argument about debugging any AI application in production. If you were just writing a simple web server to run this model behind a APIs, there are a plethora to think about, scaling, resource utilisation, load balancing, batching, etc. Hence the argument for creating an standarized interface for these type of serving problems.
We haven't even touched k8s, and throughput, latency, serving optimisation, but you get the idea of building this from scratch versus using library such as OneDiffusion.
Of course we are also working to improve the library and include more features, so looking forward to hearing more constructive feedback!
Docker does not change the API. I can take any simple program and run it with Docker, no modifications needed. That also implies that Docker is fully optional. If I ever need to, I can just run the program outside of Docker. Or I could run gdb inside the Docker container, so it adds no complexity to debugging.
OneDiffusion is exactly the opposite. To use it, I need to rewrite my app for it's API. Once done, I'm 100% locked in and it won't work without the framework anymore. And if there are any issues, I always have to check all OneDiffusion source code, too, because it is impossible to know a priory if the issue was caused by my code or by the framework.
Just imagine if Docker would require you to recompile your OS from scratch for every update: only hardcore Gentoo fans would use it. But that's the level of commitment that OneDiffusion asks of me.
You can just use huggingface and Automatic111 for this, not sure why this is needed at all.
OneDiffusion project aims to solve a very different problem, which is bringing a SD pipeline to serve production traffic and scale in the cloud. For example, configuring a large diffusion model to run on multi-GPU: https://huggingface.co/blog/deploy-deepfloydif-using-bentoml