Hacker News Reader

Comment by falaki | Hacker News Reader

We don't expect this to be as good as the latest OpenAI GPT release. This is just to demonstrate that developing a conversation agent using an existing foundation model is not as hard as some may assume. Take a foundation model that is not capable of Q&A and tune it with a fairly small Q&A data and you get your in-house ChatGPT.

Disclaimer: I work at Databricks.

View on HN

Thanks for the feedback. The potential edge with Dolly is huge. Building a firewalled model with custom corpus is a big deal. I have been experimenting with openai and even with public data (but really limiting to the domain), yields great improvements (openai may be stale because of cut off data). I am excited to see where Dolly goes.

Dolly appears to fundamentally be a tech demo advertising how you can use Databricks for compute. I honestly wouldn't expect them to take it that much further, particularly in the context of larger models that would be significantly more expensive to fine-tune. But I'm happy to be proven wrong.

We plan to continue working on it and invest more.

I imagine they will sell fine tuning as a service to Databricks customers. If I put all my data into their lake I too can get my own custom ChatGPT. That's compelling.

I also see that as the use case and would find it useful. However I feel this is somewhat low-budget so far coming from such a large company.

you are referring to the dolly model? I think the training set could achieve similar performance if we would fine tune similarly sized model

Reply on news.ycombinator.com