Quality over quantity.
Apparently its better to pay $100 million for 10 people than $1 million for 1000 people.
Apparently its better to pay $100 million for 10 people than $1 million for 1000 people.
So it depends on the type of problem you're trying to solve.
If you're trying to build a bunch of Wendy's locations, it's clearly better to have more construction workers.
It's less clear that if you're trying to build SGI that you're better off with 1000 people than 10.
It might be! But it might not be, too. Who knows for certain til post-ex?
I always get slightly miffed about business comparisons to gestation: getting 9 women pregnant won't get you a child in 1 month.
Sure, if you want one child. But that's not what business is often doing, now is it?
The target is never "one child". The target is "10 children", or "100 children" or "1000 children".
You are definitely going to overrun your ETA if your target is 100 children in 9 months using only 100 women.
IOW, this is a facile comparison not worthy of consideration.[1]
> So it depends on the type of problem you're trying to solve.
This[1] is not the type of problem where the analogy applies.
=====================================
[1] It's even more facile in this context: you're looking to strike gold (AGI), so the analogy is trying to get one genius (160+ IQ) child. Good luck getting there by getting 1 woman pregnant at a time!
Your designing one thing. You're building one plant. Yes, you'll make and sell millions of widgets in the end but the system that produces them? Just one.
Engineering teams do become less efficient above some size.
You might well be making 100 AI babies, and seeing which one turns out to be the genius.
We shouldn’t assume that the best way to do research is just through careful, linear planning and design. Sometimes you need to run a hundred experiments before figuring out which one will work. Smart and well-designed experiments, yes, but brute force + decent theory can often solve problems faster than just good theory alone.
In this case, you want one foundation model, not 100 or 1000. You can’t afford to build 1000. That’s the one baby the company wants.
I am going to repeat the footnote in my comment:
>> [1] It's even more facile in this context: you're looking to strike gold (AGI), so the analogy is trying to get one genius (160+ IQ) child. Good luck getting there by getting 1 woman pregnant at a time!
IOW, if you're looking for specifically for quality, you can't bet everything on one horse.
At some point, even companies like Meta need to make a limited number of bets, and in cases like that it's better to have smarter than more people.