Hacker News Reader

What size was the test set?

From what I gather, training was done on 170,000 Amino Acids (features) and the resultant protein structure (labels). This is out of 200 million possible proteins.

How many examples were in the test set?

EDIT: Looks like N=100 for test set: “Entrants get amino acid sequences for about 100 proteins whose structures are not known” https://www.sciencemag.org/news/2020/11/game-has-changed-ai-...

View on HN

As usual with ML, I now wonder how “similar” the test set is to the training set, compared to the examples that are neither in the training set, nor the test set:

TODO: 200 million - 170,000 training - 100 test ~= 199.8 million proteins

They trained on 170k sequences/ structures/ proteins, each sequence has 10s to 100s or even 1000s amino acids. Structure is much more conserved than sequence. Out of the 100 targets, roughly 1/4th have no similarity to known structures, so there shouldn't be an overlap for those with the training set. They did very well on those targets.

Reply on news.ycombinator.com