Hacker News Reader

Comment by farouqaldori | Hacker News Reader

Hey all, co-founder here happy to answer any questions!

Nice product overall. I had some feedback and questions

feedback :

1. Since you guys have support for multiple models, it would be cleaner and more correct to give the API some name which doesn't start with openAI.

2. sdk using other languages like Python in `show code` would be nice.

3. It was a bit confusing to figure out how to fine tune the model, would be nice if it was explicitly available as a side pane.

Questions:

1. Can you speak a bit about your tech stack if that's alright

2. How do you currently scale inference if there is more incoming requests coming in?

Thank you so much!

1. Where exactly did you see this? There are internal FinetuneDB API keys, and external API keys like OpenAI. Though it's confusing, I agree!

2. Work in progress.

3. I agree, thanks for the feedback.

There are multiple components working together, so it's hard to define a single tech stack. When it comes to the web app, Remix is my framework of choice and can highly recommend it.

Congrats on the launch, UI looks sleek! Is tracking logs available in the free plan?

Thanks, and yes, tracking logs is included in the free plan!

What benefits does this bring me vs just using OpenAI's official tools?

Other co-founder here, so we offer more specific features around iterating on your datasets and include domain experts in this workflow. And I'd argue that you also want your datasets not necessarily with your foundation model provider like OpenAI, so you have the option to test with and potentially switch to open-source models.

Is it possible to fine-tune language models using plain text completions, or is it necessary to use datasets consisting of structured conversations?

Yes, you can fine-tune using plain text completions. You don't need structured conversations unless you want conversational abilities. Plain text works great if you want the model to generate text in a specific style or domain. It all depends on what you're trying to achieve.

Nice.

And about the cost of finetuning: is there a difference in price when only training the model on completions?

The cost depends on the number of tokens processed, so fine-tuning on completions costs the same per token as any other data.

What's the cost of fine tuning and then serving a model, say Llama 3 8B or 70B? I couldn't find anything on the website...

Hi, current pricing for Llama 3.1 8B for example is: Training Tokens: $2 / 1M, Input and Output Tokens: $0.30 / 1M. We'll update pricing on the website shortly to reflect this.

Reply on news.ycombinator.com