Yes, the Tea app inspired me to write this actually
14 karma · joined February 3, 2020
X: https://x.com/FarouqAldori
LI: https://www.linkedin.com/in/farouq-aldori/
Once a model is fine-tuned, you can run inference on Llama 3.2 3B for as low as $0.12 per million tokens. This includes access to logging, evaluation, and continuous dataset improvement through collaboration, all without needing to set up GPUs or manage the surrounding infrastructure yourself.
Our primary goal is to provide the best dataset for your specific use case. If you decide to deploy elsewhere to reduce costs, you always have the option to download the model weights.
About Gemini Flash, we add new model providers entirely based on feedback. Gemini is next on the roadmap!
Happy to discuss this in detail, how do you measure performance?
We've built the platform primarily for companies that serve LLMs in production, so even if we allowed you to fine-tune on device, sooner or later you will find yourself in a position where you want to deploy the model.
We want to streamline this whole process, end-to-end.
With that being said, I do agree that we shouldn't store everything on the cloud, this is what we're doing about it:
1. Any data in FinetuneDB like evals, logs, datasets etc. can be exported or deleted.
2. Fine-tuned model weights for OS models can be downloaded.
3. Using our inference stack is not a requirement. Many users are happy with only the dataset manager (which is 100% free).
4. We are exploring options to integrate external databases and storage providers with FinetuneDB, allowing datasets to be stored off our servers
1. Where exactly did you see this? There are internal FinetuneDB API keys, and external API keys like OpenAI. Though it's confusing, I agree!
2. Work in progress.
3. I agree, thanks for the feedback.
There are multiple components working together, so it's hard to define a single tech stack. When it comes to the web app, Remix is my framework of choice and can highly recommend it.
For a defined specific use-case it's certainly possible to beat their performance, but things get harder when you try to create a general solution.
To answer your question, the format of the data depends entirely on the use-case and how many examples you have. The more examples you have, the more flexible you can be.
We utilize LoRA for smaller models, and qLoRA (quantized) for 70b+ models to improve training speeds, so when downloading model weights, what you get is the weights & adapter_config.json. Should work with llama.cpp!
With that being said, feel free to email us with your use-case, I could build the SDK within a few days!
No matter if you build or buy, what you want to achieve is the following:
1. Visibility over what's going on in production. 2. Ability to identify both good and bad outputs. 3. Performance measuring (custom evaluations). 4. Feedback loop with fine-tuning.
With those pieces in place, you can start making data-driven decisions.
What we've learned so far is that each application is unique, so if you could tell me more about your app I could share some insights.
Also, in general, Chinese products tend to be slow which is understandable if the servers are located in China.
How many hours does Samantha work for you on a monthly basis, and how often do you publish?
Cool component nonetheless, makes sense that it's fake since the majority of users wouldn't bat an eye.
I loved the parts where they showed some PHP echoing some HTML to give the video that hacker vibe.