Hacker News Reader

Comment by simonw | Hacker News Reader

I'm really curious to understand more about this.

Right now there are top tier LLMs being produced by a bunch of different organizations: OpenAI and Anthropic and Google and Meta and DeepSeek and Qwen and Mistral and xAI and several others as well.

Are they all employing separate armies of labelers? Are they ripping off each other's output to avoid that expense? Or is there some other, less labor intensive mechanisms that they've started to use?

View on HN

There are middle-men companies like Scale that recruit thousands of remote contractors, probably through other companies they hire. There are of course other less known such companies that also sit between the model companies and the contracted labelers and RLHF’ers. There’s probably several tiers of these middle companies that agglomerate larger pools of workers. But how intermixed the work is and its scale I couldn’t tell you, nor if it’s shifting to something else.

I mean on LinkenIn you can find many AI trainer companies and see they hire for every subject, language, and programming language across several expertise levels. They provide the laborers for the model companies.

I'm also very interested in this. I wasn't aware of the extent of the effort of labelers. If someone could point me to an article or something where I could learn more that would be greatly appreciated.

Just look for any company that offers data annotation as a service, they seem happy to explain their process in detail[0]. There's even a link to a paper from OpenAI[1] and some news about the contractor count[2].

[0]: https://snorkel.ai/data-labeling/#Data-labeling-in-the-age-o...

[1]: https://cdn.openai.com/papers/Training_language_models_to_fo...

[2]: https://www.businessinsider.com/chatgpt-openai-contractor-la...

I added a reply to the parent of your comment with a link to an article I found fascinating about the strange world of labeling and RLHF -- this really interesting article from The Verge 2 years ago:

https://www.theverge.com/features/23764584/ai-artificial-int...

Reply on news.ycombinator.com