- Pay openai less than $50mo
- Manage cloud gpus, hire ml engineers > $1000/mo
- Buy a local 4090 and put it under someone's desk, $no reliability +$1500 fixed
Any larger business will need scalability and you still can't compete with openai pricing.
Maybe one of you startup inclined people can make an openllama startup that charges by request and allows for finetuning, vector storage
I've had great fun with the " Easiest 1-click way to install and use Stable Diffusion on your computer."
https://github.com/easydiffusion/easydiffusion
And while Whisper is OpenAI, it is trivial to use locally and extremely usefull
The use case is based on public information on the internet. News articles, PRs, social media posts, etc.
LLMs are used to extract info from text in a structured format. It used to have several classification and NLP models to do the job, but now a single LLM can do it faster and with better accuracy.
I'm currently building www.lalamon.us specifically to provide a fully hosted open source model experience. One slight difference is that I'm providing a private chat instance for each user, so charging based on hours of active chat usage seemed to make more sense. Per-request charging seems more unpredictable for users, but I'd be interested in hearing the case either way.
Feel free to reach out with more questions if interested; my email is in my profile.
And more models coming soon.
Vector storage isn’t on the roadmap (what stops using a separate vector store from working well? Could add to roadmap but want to add understand more first), and we could add fine tuning if it’s a common request.
If people can drag and drop some files from their nas, you parse them with apache tika or similar https://tika.apache.org/ , they can start using personalized branded bots. It also lets you do things like refusing to answer, if the vector database returns nothing and the use case requires a specific answer from the docs only (not the llm to make stuff up).
Issuer: McAfee OV SSL CA 2
Expires on: Aug 3, 2023
Current date: Jul 8, 2023
PEM encoded chain: -----BEGIN CERTIFICATE----- MIIGfzCCBWegAwIBAgIQKt9VNrFtaozA1bILX1OcfzANBgkqhkiG9w0BAQsFADBk MQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0
Will your pricing be competitive with Replicate?
The instant a company has sensitive data, this becomes very viable.