Comment by cowsandmilk | Hacker News Reader

Comment by cowsandmilk | Hacker News Reader

Parent Full thread

For the general question of pharma investing in structure prediction, I think participants in CASP overestimate the importance of structure. It is nice to have and there certainly are structure-driven projects, but docking is so poor that often computational models of how a molecule binds, even when you have a structure of a protein, are unreliable and there are plenty of case studies of them sending teams in the wrong direction. This would only be worse in the case of AlphaFold since, as the post shows, GDT_HA is still quite poor.

From my experience in research, pharma has found that cellular models and phenotypic assays are far more meaningful for pushing projects forward. So, there is far more interest in applying machine learning to that data than for building protein structures. And those same methods can be applied to target-based projects regardless of whether you have structure. And regardless of how flexible your protein is. Huge portions of structure-based modeling has no ability to deal with protein flexibility, even if you know there are open and closed conformations of the protein or a loop that adopts half a dozen configurations.

Basically, academics working on folding often believe far too much in the importance of structure in drug discovery. The author appears to fall into that category.

The author appears to fall into that category.

They seem to acknowledge this:

The above graph is misleading in one way though because it is dependent on a specific metric, GDT_TS, which only measures gross topology. If we care about high resolution topology, which we certainly do for most practical applications, then a more appropriate metric is GDT_HA, and using it the picture looks a bit different:

[graph]

Still a good trendline, but much further down from a “solution”.

Another caveat is that both of these metrics measure global goodness of fit, which is important in terms of the basic scientific problem, but is often not indicative of functional utility. Local accuracy, for example the coordination of atoms in an active site or the localized change of conformation due to a mutation, is what is often sought when answering broader biological questions. Global metrics hide local discrepancy by diluting it in the sea of generally good agreement between experimental and predicted structures.

and

Even for MSA / family-level predictions, there is the question of desired accuracy, which hinges on the biological application. If one is predicting protein structures to ascertain their general fold for function classification, then high accuracy is unnecessary. If on the other hand the objective is to design small molecule drugs that bind proteins, which require ~1Å accuracy in the local pocket, it is unclear if we have made any detectable progress.

The author does make a point of discussing the question of what business does a team like DeepMind have researching the folding problem? The solution is of no apparent value to the parent company Alphabet, and yet they were still funded. Perhaps this has to do with the attitudes or values of "modern" tech companies? Historically, there seems to have been a cyclically nature to the volume of basic research in industry, peaking with Bell Labs, sinking with the rise of Welch, and now coming back with the Googs and Facebooks.

You seem to be implying that the world would be better if they didn't do this research, that doing it as a tech company to just apply the tech is immoral. Is that what you really mean?

Well, one could put it cynically as Deep has developed a giant "cannon" that can be fired at different very hard but reasonably well defined problems. They've hit go, chess and similar things. Hitting proteins keeps their activity in the limelight. Essentially, DeepMind is demonstrating the value of the money Google paid to purchase it.

Cowsandmilks points out the limit of AlphaProtein above: https://news.ycombinator.com/item?id=18647187

The problem is one runs out of reasonably isolated problems with a reasonable metric.

So, have you refuted the argument in anyway or just downplayed it?

I would have preferred that text in the comment you responded to had less of a trash talking feel to it.

Also it’s a bit overplayed because although these are special moments, there’s no shortage of examples in history where a new approach in one field is applied to get results previously unattainable in another.

However there’s no need for rationalizations. The result doesn’t generalize to anyone is smarter than anyone else, or that it’s not important because it’s not the super most important drug discovery tool.

It’s just a special moment where a technique is impacting another area of research. Which is part of a special period in history where AI/ML is evolving from being literally a joke that could defund your research, to finally having a broad set of useful applications.

We should be glad to see as many of these moments as possible from any discipline to any other.

> academics working on folding often believe far too much in the importance of structure in drug discovery.

It reminds me of the 90's and early 2000's when many were convinced that sequencing the human genome would suddenly make drug discovery a simple problem.

Reply on news.ycombinator.com