Hacker News Reader

Comment by RocketSyntax | Hacker News Reader

"The resulting algorithm outperformed all entrants at the most recent blind assessment of methods used to predict protein structures, generating the best structure for 25 out of 43 proteins, compared with 3 out of 43 for the next-best method."

View on HN

This is remarkable. Teams of researchers all over the world have taken part in the CASP competitions for decades. Many attempts using machine learning and ANNs have been made. What is it about DeepMind that allowed them to make such a breakthrough? Do they have expertise in deep learning that does not exist in academia? Incredible amounts of compute that academia cannot afford?

Alphabet owns way, way more computers than anyone else. You could lose any of the “top supercomputers” in the cracks of their datacenters.

The techniques DM used are popular in academia right now, too. Using evolutionary data to shortcut hard problems has been key to advancement in protein research for decades. DM just executed better, a combination of smart people, some good ideas, and lots of experimentation. NEver underestimate the ability of company that exists to win games, to win competitions.

And never underestimate the amount of money that a big tech company can throw at a random problem. DeepMind probably blew through the equivalent of multiple R01 grants writing that paper.

Big biotech can throw big amounts too.

And I read that the size of the team was 10 people - that's not a big number.

The compute power applied was not why they had this outcome.

If their salaries are anything like what Bay Area companies are shelling out for top AI engineers, each one of those 10 people is probably costing as much as 10 grad students in any of the other labs working on this problem. Big Biotech does not usually have the money to get into a bidding war for engineering talent with companies like Google.

"There are dozens of academic groups, with researchers likely numbering in the (low) hundreds, working on protein structure prediction. We have been working on this problem for decades, with vast expertise built up on both sides of the Atlantic and Pacific, and not insignificant computational resources when measured collectively. For DeepMind’s group of ~10 researchers, with primarily (but certainly not exclusively) ML expertise, to so thoroughly route everyone surely demonstrates the structural inefficiency of academic science."

"What is worse than academic groups getting scooped by DeepMind? The fact that the collective powers of Novartis, Pfizer, etc, with their hundreds of thousands (~million?) of employees, let an industrial lab that is a complete outsider to the field, with virtually no prior molecular sciences experience, come in and thoroughly beat them on a problem that is, quite frankly, of far greater importance to pharmaceuticals than it is to Alphabet. It is an indictment of the laughable “basic research” groups of these companies, which pay lip service to fundamental science but focus myopically on target-driven research that they managed to so badly embarrass themselves in this episode."

From: https://moalquraishi.wordpress.com/2018/12/09/alphafold-casp...

I completely disagree with his interpretation. It would be surprising if group that concentrates some of the top expertise in AI weren't able to make a big impact on a well-defined optimization problem that has been studied for decades.

I think a lot of the commentary is missing two essential points:

1. Protein structure prediction is to a large extent a solved problem for small-ish, soluble targets. AlphaFold is a significant improvement on the current state of the art, but the state of the art was already far enough along that the best computational models in 2007 were good enough to bootstrap experimental structure determination (https://www.ncbi.nlm.nih.gov/pubmed/17934447). In other words, it's not like the entire academic community was stumbling around helplessly in the dark.

2. The value of these predictions to pharmaceutical companies is extremely marginal. Having a high-accuracy model is very helpful but it's rare that the researchers have so little information available that a completely de-novo prediction is necessary. And when they really don't have much information at all, it's usually because the target is sufficiently messy to defy traditional structure determination methods - which means it's almost certainly more than AlphaFold can handle too.

It's critical to understand that having a measured benchmark is what makes this result so important and tells us that we're making progress.

Without measurable benchmarks we have no idea if we're making real progress towards human level AI.

Reply on news.ycombinator.com