yogoism

Greetings from Japan.

Thank you for sharing such an insightful point. This really resonates, speaking from my experience as an annotator on crowdsourcing platforms. I also found that a genuine commitment to quality from fellow annotators can be quite rare.

This makes me curious about a few things:

1. What are some concrete examples of the "unintended consequences" you ran into?

2. When you initially considered outsourcing, what was the main benefit you were hoping for (e.g., speed, cost)?

3. On the flip side, what have been the biggest frustrations or challenges with the in-house approach?

Would love to hear your thoughts on any of these. Thanks!

Thanks so much for that clear explanation—it really made me realize that while companies like Scale AI can thrive during the hype of the foundational-model race, it’ll likely get tougher down the road.

If you don’t mind me asking, as someone on the front lines of AI product development, what challenges have you found to be even more difficult than data annotation?

I’d really appreciate any insights you can share.

That's a great insight, Paul. As someone who has been researching the data annotation space, your perspective really resonates.

I completely agree that the first-hand, contextual information you get from actual users is something an external firm can never replicate. It seems like the most effective and efficient way to spin the data flywheel at high velocity.

This leads me to a question I've been struggling to understand: If this approach is so powerful, why do you think even companies with the vast resources of Big Tech still rely on what seems to be a riskier path—using external human evaluators—instead of fully building this feedback loop in-house?

I feel like I'm missing a key piece of the puzzle. I would be very interested to hear if you have any thoughts on this.