AlphaFold’s new rival? Meta AI predicts shape of 600M proteins
nature.com
nature.com
> Meta’s network, called ESMFold, isn’t quite as accurate as AlphaFold, Rives’ team reported earlier this summer2, but it is about 60 times faster at predicting structures, he says. “What this means is that we can scale structure prediction to much larger databases.”
> Burkhard Rost, a computational biologist at the Technical University of Munich in Germany, is impressed with the combination of speed and accuracy of Meta’s model. But he questions whether it really offers an advantage over AlphaFold’s precision, when it comes to predicting proteins from metagenomic databases. Language model-based prediction methods — including one developed by his team3 — are better suited to quickly determine how mutations alter protein structure, which is not possible with AlphaFold. “We will see structure prediction become leaner, simpler cheaper and that will open the door for new things,” he says.
DeepMind has no trouble getting the inference time at Google to compute predictions using their model for any protein of interest or the largest database.
Thus, a faster but more inaccurate system is not really desirable unless it really does provide another feature, such as (as Rost says) predictions on mutations alter structure. But if alphafold works, then it would indeed be capable of predicting how mutations alter structure.
AlphaFold (I haven't looked at Meta AI yet, beyond a cursory glance), is solving the state of the folded protein, based primarily off the state of other similar folded proteins. There's some, but extremely limited, modeling of unknown states. The major breakthrough for AlphaFold appears to be that it is substantially better at detecting meaningful signal in homology-based sequence alignments. That means, it can figure out the parts of a protein that are similar to other known folded proteins, with much greater success than previous models (2x-3x better iIrc). As far as modeling proteins or portions of proteins for which there are no known homologs, the model becomes significantly weaker (although not nonexistant).
In my opinion, and also the opinion of a number of my colleagues, advertising AlphaFold as having solved the "protein folding problem", and other similar language (used by both the media and the AlphaFold press releases) is completely disingenuous. We are effectively no closer to understanding how proteins fold, nor are we closer to being able to predict how a protein folds without any a priori information. Furthermore, homology modeling, while quite successful so far for a lot of proteins, breaks down on edge cases, higher-order structures, and unknown folds. It seems unlikely that it will ever completely solve the problem, and therefore, an entirely different approach will be needed to modeling protein folding that encompasses the entire field, edge cases and all (a quite lofty goal that may also never be reached).
All this is to say: AlphaFold is a great tool that I use frequently and am grateful it exists, but it hasn't solved protein folding, and the thing is has solved is intrinsically limited, and we should probably use different language to describe what it's done. Either way, I'm glad to see progress being made here, and eagerly await finding out how proteins actually fold.
It's a genuine question. A lot of experts have a lot of blind spots especially around prediction, rate of growth, potential, vision. History is littered with people who are really good at a field but fail to see 2/3/5/10 years ahead.
>It seems unlikely that it will ever completely solve the problem, and therefore, an entirely different approach will be needed to modeling protein folding that encompasses the entire field, edge cases and all (a quite lofty goal that may also never be reached).
Of course I'm fallible, but it doesn't change the fact that right now, this is the case - AlphaFold teaches us essentially nothing about how proteins are actually folding, and instead is solving for the folded state. If someone can backtrack from there, I'll be nothing but ecstatic.
meaning a protein could fold in multiple ways (similar to a swiss army knife "folding" into different shapes), and alphafold only predicts a subset of these for now (which is still amazing).
The list of reasons for this- both functionally and inadvertently- is extremely long. But the long and short of it is that the ability of proteins to reproducibly fold to a single structure was figured out using this technique- putting an already folded protein into a very strong solution (of Urea or guanidinum chloride), detecting that it "unfolded", then putting it back into salty water and watching it reform to the same original structure.
lots more detail here: https://en.wikipedia.org/wiki/Anfinsen%27s_dogma (this dogma is literally the core of protein folding/structure prediction/function prediction).
Even more detail here: https://en.wikipedia.org/wiki/Hofmeister_series which is basically a series of progressively stronger solutions that interfere with water/protein interactions and disrupt or enhance folding. This data was key to establishing that hydrophobic collapse (one of the dominant models for how proteins spontaneously form structure) is a significant force in driving folding free energy.
Yes, look up “metamorphic proteins”.
There's no way a human will ever understand this. It's likely a problem that is beyond what the human mind is capable of understanding.
Isn't that a bit like saying about image classifiers, that we still don't _understand_ how to recognize cat pictures, and that we couldn't do it without a large set of training data?
(To an extent this is true, that's also why a lot of work goes into interpretability and attribution in models.)
To describe the folding of a protein chain, a “picture” of its lowest energy conformation (which is the goal of AlphaFold—and any structural modelling tool) is not enough. We rather need a “movie” showing the steps said chain undergoes, from an unfolded polypeptide to its final, folded, native state (the “picture”).
What you are discussing is of course being studied but the problem is that it is a lot more computationally expensive. We do have simulations of simple (read very small) proteins folding and unfolding but for larger ones the computational time to watch them fold can be gigantic, if not impossible due to the fact that proteins often fold as they are being made. Which means including a much larger process into a folding one which just further stresses computational resources.
This computational problem is so enormous that a company at the cutting edge of research D.E. Shaw built a specialized computer solely for simulating proteins. Also most of the software used for this until recently had abandoned multi-GPU paralellism because it didn't scale well. The pandemic caused the need to simulate the virus on the entirety of Summit and introduced some work back into that route but it is still specialized (and wouldn't help for systems below a certain size anyways).
Also my previous points have been for atomic models (e.g. we treat everything more like newtonian particles and ignore quantum effects) some things definitely need more resolution and at that level you are lucky to see protein fluctuations let alone folding.
Do you want to answer these questions for the satisifcation of understanding the underlying physical rules that drive folding? Why? It's unclear that knowing those things would actually make a large impact in any industrially/medically useful contexts. It uses huge amounts of CPU to sample these functions accurately enough to replace actual physical experiments on protein motion. Just doesn't seem like an effective investment of brain or computer time.
(I say this as somebody whose entire career was predicated on using MD to answer these questions; see https://www.nature.com/articles/nchem.1821 for our attempt in that space)
Also a number of protein folding diseases.
And am I right that the goal is to create a best-effort short-cut to the final stage of brute-force final folding to get a real result?
There are other simplifications where a bunch of forces and masses can be cancelled out, or aggregated - but we can't see them. Can a computer see them with enough processing? It seems like it.
"As a test case, they decided to wield their model on a database of bulk-sequenced ‘metagenomic’ DNA from environmental sources including soil, seawater, the human gut, skin and other microbial habitats."
Meta has a lot of data, but I'm unaware of them having a presence in the environmental/medical diagnostics industry which is where I assume this would be applied
Perhaps they are going for a Bell Labs kind of structure?
Why? Recruiting most likely.
Unclear what products this would apply to
Whether or not we like, or agree with, or believe in their goals (I don't), I think it's hard to argue that competence in AI is not useful for them.
AI can predict the behavior of the users, feeding them as many relevant ads as possible, while keeping them engaged.
They have a stated goal of eradication all diseases by 2100
Genuinely sorry.
> "Sergey Ovchinnikov, an evolutionary biologist at Harvard University in Cambridge, Massachusetts, wonders about the hundreds of millions of predictions that ESMFold made with low-confidence. Some might lack a defined structure, at least in isolation, whereas others might be non-coding DNA mistaken as a protein-coding material."
Understanding of how these proteins function requires high-resolution information about bond angles and atom-atom distances, particularly for non-structural proteins (i.e. interesting catalytic enzymes). Hence, wet-lab work and protein structure characterization via X-ray and NMR methods aren't going anywhere.
For me, as someone solving structures on a monthly basis, AlphaFold is great because after I get back my electron density map, I dump my sequence into AlphaFold, get a model (of any confidence) and most of it fits well enough into my density map that I don't have to start trying to model from nothing, and it saves me honestly days of work.
If your question is why does an architecture designed for NLP work well for aminoacid sequences, it turns out that the Transformer architecture is surprisingly versatile. It works amazingly well for other sequence-like data (like audio), and even for images (which, to me, is surprising, since images are not a one dimensional sequence like text or audio).
The Transformer architecture (Particularly, the Multihead self-attention[0] mechanism within the Transformer) is, in my estimation, once of the key innovations in deep learning in the past 5 years. It's used in pretty much everything in deep learning (GPT-3, AlphaFold, DALLE-2/Stable Diffusion, OpenAI's Whisper, come to mind).
[0] https://lilianweng.github.io/posts/2018-06-24-attention/#mul...
Smart. If you might get cut, having all your work available to pick up from is a great idea.
This would be like condemning research for AT&T/Bell Labs because the company was (even at the time) a terrible monopolistic corporation.
Meta is easily among the top few AI research institutions in the world.