Comment by j-g-faustus | Hacker News Reader

Comment by j-g-faustus | Hacker News Reader

Parent Full thread

I may be out of tune with many others here, but I think certain constructive processes (as in creating a many-faceted proof, outlining overarching software architecture, making the storyline of a good novel/movie) are most efficient with small groups of people. Otherwise you run into issues of "design by committee" or "too many cooks".

It is easier to crowdsource once the main outline is in place, it structures a vague problem into a set of more specific subtasks that can (more or less) easily be distributed across many people.

Like in this case: Deconstructing an existing proof can work very well with crowdsourcing, since a proof contains a relatively small set of specific claims, and each "crowd" contributor can focus on one particular issue.

But this works because the crowd now has a common focus point and task list created by the paper being published. It's not clear to me how (or whether) the process could be reversed, that the same crowd and effort could be coordinated to cowrite an original paper with an alternative proof.

I'd agree that crowd sourcing might not always be the best way to approach things. But the general situation that the person who gets the final result wins very likely does inhibit effective work on the solution of very difficult problems.

I mean, I think there's a consensus that if Wiles had been publishing his results as he went along, Fermat would have been solved earlier but probably not by Wiles himself. Whether the current N/NP proof is right or not, it also is clearly the product of long work in a vacuum. That's not the best way to do things if nothing else as a matter of sanity. If you're publishing as you go along, you've got a lot more of a sanity check. Further, if research is open, you can do single person, small-committee and crowd-sourced versions.

Still, I don't think this primarily a matter of the Millennium Prizes in particular but of the tremendous competition of academia in general - Wiles' effort happened before the Millennium Prizes - and he couldn't get Fields Medal either - too old.

Oddly enough, the Netflix prize actually seems to have produced a fusion of efforts in the end. Perhaps future prize creators could think about that. Both the Millennium Prize and the Fields Medal seem deeply problematic in their effect on mathematics in general.

Would they pursue for the proof in the first place, if not for the prizes (recognition, among others)? I think it's just analogous to how businesses work, with competition, secrecy, etc. And the debate on whether this approach to proof is good or not overall is similar to the debate of capitalism vs. communism.

Good points, and I agree.

The Netflix prize is interesting. According to a summary post on the Netflix forum

    the early results were mainly by individuals... 
    the team members began to coalesce and combine, and 
    in the end, entire teams coalesced and recombined.

http://www.netflixprize.com//community/viewtopic.php?pid=961...

I can only speculate that the combination of a deadline, a problem that was too hard for any individual and a common forum for exchanging ideas helped encourage team formation. (I.e when you realize that you can't do this alone, teaming up is a rational thing to do.)

I assume that the problems we (as in society, humanity) want to tackle will grow ever more complex in the future, and may easily outgrow the capacity of individuals.

So yes, perhaps we could/should learn from how the Netflix prize played out...

I think some risk-averseness drove the combination as well: at the very end, two teams with very similar performance were looking at it being more or less random chance who'd finish a hair ahead, with one getting $0 and the other getting $1m. So they merged and split a much safer $500k each.

Reply on news.ycombinator.com