Hacker News Reader

Why bother use a GPT-3 model when you can use the free BLOOM model? Large "foundation" models will become a commodity -- a race to the bottom.

View on HN

I have used BLOOM a little, and I struggle to get it to produce the same kind of output as GPT-3. Anyone have tips there? It often seems to not keep the right context of the previously generated text.

GPT-3 has been fine-tuned after release to better interpret prompts (see InstructGPT). Perhaps Bloom is more like the original GPT-3; a little more raw and requiring better prompt engineering?

In my small amount of testing of Bloom so far it seems capable of advanced behaviour but it can indeed be trickier to coax that out. Playing with temperature and sampling matters for sure.

I've been testing BLOOM for a while and it seems it is working very well with good few-shot learning. See this post about prompt examples: https://nlpcloud.com/effectively-using-gpt-j-gpt-neo-gpt-3-a... All of these examples work great with BLOOM.

But for the moment you can't just use pure natural language instructions with BLOOM indeed. Maybe it will come!

Reply on news.ycombinator.com