Hacker News Reader

Comment by mwcampbell | Hacker News Reader

How far does the similarity extend? Specifically, the big question for me is whether AlphaFold will be freely available like ImageNet, or proprietary.

View on HN

The competition requires enough revealing about the methodology for other teams to replicate it so open implementations are going to be available for sure.

It also looks like they came up with a brand new jiggling algorithm which is probably just V1 now, this really changes things in a significant way!

I expect this to be quickly replicated once published. Training data is public and training compute is not enormous and AlphaFold of 2018 did get replicated.

CASP typically works this way: one person "wins" by getting a slightly higher score than everybody else. Two years later, the top teams have all duplicated the previous winner's tech, and two years after that, there's a github you can download and run on your GPU to reproduce everything.

How do you define enormous? "It uses approximately 128 TPUv3 cores (roughly equivalent to ~100-200 GPUs) run over a few weeks". Also last time it took about a year for good replications to pop up.

A year is a fast time to replication in many scientific fields.

While substantial, the resources here are well within reach of many labs, research institutes, and organizations. For this result this big, I'd guess we'll have 2-6 additional implementations in the next 18 months. The problem has been 'open' for 40+ years, so that's lightening fast!

A couple of hundred GPU's is well within the reach of many even moderately well heeled research institutes. It'd seem that about 3 weeks of compute time with 128 TPU v3's would be about $170,311.68.

But of course that cost would only be for the final model. Anyway, I think I am just living in a different world... :-) We could never compete with that

Yah, big grant money. Now the grad students programming the open source clones will only make approximately $0.56, or 4.2 Ramen packs, for their effort. ;)

Also with keeping in mind that once a good open source model is available, researchers with less resources can still use it to fine tune and get new results for far cheaper than training a new model from scratch.

or cryptominers

A lot of labs have access to the various strategic supercomputers of the USA.

Ex: Summit has 27,648 V100 GPUs (and those V100s have Tensor units). If you're saying that only 200 GPUs are needed to replicate the experiment, that doesn't even use up 1% of Summit's available utilization.

ImageNet is a competition and a dataset, AlphaFold is a neural network.

Reply on news.ycombinator.com