Hacker News Reader

Hacker News

Comment by choeger | Hacker News Reader

Full thread

choeger

I am not an ML engineer (except when I program in ML, of course ;)). But this sounds a lot like the following:

1. We had a model that worked in principle, but the search space was practically infeasible.

2. We made an observation that a different model might exist that makes the search space irrelevant.

3. We threw ML at it.

4. Now we might have a model that fulfills (2) but we cannot be sure because we used a black-box approach.

5. Somehow the results are exciting. Better results would be really exciting.

6. We hope that more data yields these better results.

Is that correct? Am I the only one to lament these black-box approaches? Should there not be a bunch of people now studying the learned models to figure out if much better results can actually exist?

View on HN

Think of it as an engineering problem rather than a science problem.

In much of drug discovery/development (and disease research), being able to predict protein structure would be very valuable. Being able to quickly find candidate structures (that can then be searched for in the lab) speeds things up immensely. Reducing false positives (or just coming up with possibilities at all) is a huge win.

But you’re right that this probably doesn’t help the protein theoretician much if at all. We already “know” how it works (it’s just thermodynamics and quantum mechanics) and of course have no idea how it works (“well they wiggle around until they find a low energy state” doesn’t really tell you anything). But that doesn’t keep this from being exciting.

Protein structure prediction, at the current levels of precision (and I include AlphaFold here), is not useful for drug discovery.

It’s the sort of thing researchers say to get grants, but as a distant goal, not a practical reality.

For structure-based drug discovery (which isn’t even the majority of drug discovery), the details are what matter (e.g. “does this water molecule mediate a binding interaction, or do the sidechains shuffle a bit, and kick the water out?”), and these methods don’t even come close to predicting detailed interactions.

Metrics in this space are focused on “general correctness” of protein backbone conformation. Success is to achieve a kind of blurry view of the overall shape of the molecule, and drug design is trying to predict specific atomic interactions. They’re two wildly different problems.

About the best you can say is that if we had a generalizable model of physics that could predict protein structure, it might also be able to do a good job of evaluating how a small molecule binds to a protein target. But even that is a huge leap, and when you start using black-box methods like AlphaFold to specifically solve the problem of structure prediction, it’s not really clear that generalization is even possible.

There are potential practical uses in drug discovery for a method that can design a protein which takes a particular shape, but even that is pretty different from what AlphaFold actually does.

Not everyone is focused on solving folding. It's just one step of many. https://www.nature.com/articles/s41592-019-0666-6

https://github.com/yangkky/Machine-learning-for-proteins

I work in the field.

There are indeed many different applications of ML to drug discovery, but AlphaFold is a niche technique, and protein structure prediction is not practically useful, in general.

I see. My point is that all these efforts are really validating deep learning as an approach for solving previously intractable biophysics problems. Science is a series of stepping stones.

Yes, again, I work in the field.

There are many areas where ML is being used to advance drug discovery and other kinds of practical, difficult biophysics problems. Casual readers would do well not to get too fixated on this particular area.

Your expertise is acknowledged. We are in agreement

Ha...I wasn’t repeating it to make some kind of assertion of dominance, just to explain that I’m aware of the point you’re making, and that it’s related to, but different from my own.

I see, no worries then :)

>>> Am I the only one to lament these black-box approaches?

Far from it. Prediction looks like a tool in the arsenal for better understanding. One still has to correlate the structure with the complex interactions in vivo. Even using AI in classification mode, where we can segment a large atlas of tumor cells and identify a dozen or so classes of cell anomalies may lead to faster breakthroughs in immunotherapy.

What I am trying to wrap my head around is the synthesis problem. Say AlphaFold generates a promising candidate. One that does not exist naturally. You still need the DNA or mRNA transcription sequence to synthesize the protein, right? Won't some candidates simply be too complex and unstable to reliably produce using existing mammalian or baculovirus platforms?

You can use permuted feature importance to tell you what inputs are most correlated with the condition. WAYYYY better than the binary correlation methods most statistics use for each feature.

>Won't some candidates simply be too complex and unstable to reliably produce using existing mammalian or baculovirus platforms?

You can add that to the objecting function that your model training function is optimizing, ensuring model output is not "too complex or unstable to reliably produce".

This is far from the only time this has happened, though...Many moons ago I was required to take a class in fluid dynamics in College. It was all observational best-fit statistical estimation that kinda modeled observed behavior.

ML is that with significantly more horsepower.

Either it was a poorly taught class, it was aimed at civil engineering and similar where you want simple models with broad applicability and don't care about the wide error bars, or maybe you remember best the observational stuff.

But fluid mechanics has a very deep theoretical underpinning, and generally has interpretable models. I'd suggest skimming through a copy of either Lamb or Batchelor's book, to see how far you can get just with pen and pencil and no statistical input at all.

It WAS for Civil Engineering and it was 30 years ago. FEA was JUST starting to be a thing and it sure didn't reach down into anything I encountered.

looking back through the google, I may be dimly remembering the calculations dealing with turbulent flow. It was a whole different experience to a College Junior that was used to the relatively simple equations from Physics, Statics and Dynamics.

Sounds like the standard way to teach fluids for civ.eng. All the flows you'll ever encounter will be turbulent, no point in learning about all the beautiful results that are mainly for laminar flows.

The books I mentioned are classics in the field and were first published in 1895 and 1967, respectively. Both are still in print. No computers, just advanced math (vector calculus etc).

Unlike in other cases, with protein structure, we really only care about the structure, most of the time no one cares how the chain folded to get there. ML based methods seem perfect for that.

Not an expert but I have read a lot on the subject recently.

Methods from the domain where already not truly model-based (compared to things like physical equations). It is mostly observations on existing proteins coupled with a form of gradient descent so, for this particular application, things have not degraded. (I am aware that there is more to it, this is just a fast summary)

But to be honest, it is far from a solved problem and I expect more breakthroughs from deep understanding and modeling than ML.

> I expect more breakthroughs from deep understanding and modeling than ML

Huh, I would've said exactly the opposite. The feature space of the variations of amino acid sequences is so big that I wouldn't bet on breakthroughs derived from understanding. Most recent advances seem to focus on how specific sequence motifs interact with each other, which are generally only applicable to certain kinds of proteins, but not protein folding as a whole.

My prediction would be that next breakthroughs that push the field over the usability limit are derived from more general machine learning advances.

I think deep learning can only get us so far with the existing, limited, data. I doubt it will be enough for precise structure reconstruction (but I would love to be wrong).

What it can be is a force multiplier for every progress in our understanding of why those sequences are transcribed into those structures.

But I have to admit that, with years of head-on, the research has not gotten there so I might be wrong.

> Now we might have a model that fulfills (2) but we cannot be sure because we used a black-box approach.

It's not like the alternative is any different. Being sure you have the right answer is not viable, the other methods of getting there are just more transparent heuristics.

3a. We took the best results from ML and did full quantum mechanical simulations on the much smaller set of remaining candidates.

I have seen the word “watershed” and immediately guessed some it’s some biased unverifiable claptrap from google.

They provide code and data for full verification. https://github.com/deepmind/deepmind-research/blob/master/al...

Will they provide thousands of tensor cpus as well

Except that they won a competition for protein folding by a huge margin as explained. Such claptrap indeed.

Review the casps13 results pages and then tell about “huge margin”

http://predictioncenter.org/casp13/zscores_final.cgi?formula...

First place z score sum 54 vs second place 42 and in a single category. The only thing watershed here is the headline dishonesty.

You think 54 vs 42 is a small difference in this area?

This is 90% of science these days, just keep tweaking numbers in the model until it fits with what you want to see. The why and the how is not really something we have the capability or tooling to tackle.

I got bored with physics really quickly when I realized this is what everyone does.

>This is 90% of science these days, just keep tweaking numbers in the model until it fits with what you want to see.

If the result is immediately applicable to real world problems, like in this case, that's a perfectly valid approach.

In this case we just need the model to produce answers we can use -- how we made the model, what's in it, etc, we don't need to care.

Did you read the end of the paper.

What do you mean by “black box” model? How do you know when a model “explains” something or not? How do you know if you are missing a key feature or degree of freedom in a model? Suppose you have two models, A and B, where A gives much poorer predictions than B. Is it possible for A to “explain” things better when it is not capable of producing adequate predictions? Since nature itself is not boiled down to logical elements like math proofs, what about the inherent ambiguity of language or mental models when assessing explainability? Did we really explain anything, or just decompose a problem into units of thought that artificially feel compelling for human minds at this moment of history? What would distinguish a “real explanation” from a convenient fiction, if not purely predictive capability?

Black box model is a perfectly well understood term of art.

It means a model which has a somewhat opaque internal working. A lot of modern ML approaches treat the model as a black box in that it is not particularly clear how the features actually interact to reach the prediction.

I work professionally in machine learning for over ten years now and I deeply dispute what you say, and many other practitioners do as well.

> “not particularly clear how the features actually interact to reach the prediction.“

This is quite true of many models, such as linear regression in the wild. You may also have a clear but wrong picture of how features interact, eg looking at coefficients of interaction terms in a misspecified linear model.

See for example, “The Mythos of Model Interpretability”

- https://arxiv.org/abs/1606.03490

Without a coherent definition of what “not black box” or “explainable” means scientifically, the buzzword of “black box” is also meaningless and is more of a political game to be a gatekeeper over what models are allowed to be used than any kind of honest intellectual inquiry.

I’ve worked professionally on systems where misspecified linear models and text models using simple ngram boosting were vastly more inscrutable than comparable neural networks or dimensionality reduction models for the same applications.

Nobody has any scientifically cogent idea of what makes a model “explainable” other than arguing semantics.

>Nobody has any scientifically cogent idea of what makes a model “explainable” other than arguing semantics.

Bullshit, any client you show a regression to can easily fully understand how the inputs change the output.

That's only true in some limited cases.

Section 4.1 of the paper he linked discusses extensively the limitations of linear model interpretability. Here's an example:

"With respect to algorithmic transparency, [the claim that linear models are more interpretable] seems uncontroversial, but given high dimensional or heavily engineered features, linear models lose simulatability or decomposability, respectively."

Given how many interesting problems are high-dimensional or solved through heavy feature engineering, you must be lucky to live in the happy space where problems are either low-dimensional enough to not need heavy feature engineering or where the client is already familiar with the high-dimensional features you're going to limit the model to using.

I call bullshit on your comment, not mine.

- http://www.saramitchell.org/achen04.pdf

The functional mechanism of a regression has nearly nothing to do with whether the regression explains the phenomenon it’s being used to model or not.

The fact that the software in question is not publicly available or runnable certainly makes it a "black box", regardless of whether the underlying models are human-interpretable. EDIT: my apologies, apparently it is available: https://github.com/deepmind/deepmind-research/tree/master/al... I must have gotten mixed this up with a different black-box-advertised-in-scientific-journal story.

It may be because I'm not an expert that I think I understand, but the gist of making a model explainable sounds straightforward from what I've read. You train a model that does who knows what, and then you define a very limited language and use ML to match the first model as well as possible. If the language is simple enough, humans can understand it, and according to what I read, it likely generalizes better than the original model as a bonus. I assume it's harder than it sounds though.

Quote:

"1. There’s a tiny functional language based on a small number of side-effect free combinators

2. For a given task, a program template (which the authors call a sketch), further constrains the set of programs that can be learned for the problem in hand. This also very handily constrains the search space of course, helping to make learning a suitable policy program tractable.

3. To help guide the search within the set of programs conforming to the sketch, a standard reinforcement learning algorithm is used to learn a (black box) policy.

4. The black box policy is used as an oracle (the Neural Policy Oracle), and a neurally directed program search (NDPS) tries to find the sketch-conforming program that behaves as closely to the oracle as possible."

The paper described:

http://proceedings.mlr.press/v80/verma18a/verma18a.pdf

...this seems kind of similar to the recent success in solving mathematical equations using deep learning. The language translation paradigm seems to have a lot more potential than some realized.

Yes, you are oversimplifying what is required to make a model explainable. The logical mechanism of the functional form of the model has very little to do with explainability. Explainability involves various kinds of model checking, overfitting analysis, tests for confounding variables, multicollinearity, interaction effects, model misspecification, etc.

Merely decomposing a prediction into say a linear combination of predictors does not, itself, provide any type of explanation unless a variety of more complicated assumptions about statistical model checking turn out to be true.

Even worse, because the naive approach is to just treat those regression coefficients as if they do automatically give explanatory power or feature-wise attribution, people misrepresent things unwittingly and don’t carry out enough robust model checking, leading to wrong explanations that appear to have deceptive degrees of confidence associated with them.

"The logical mechanism of the functional form of the model has very little to do with explainability"

I'm not sure we are talking about the same thing, or which of the three sources in my comment you are reacting to.

My understanding of "explainability" in this context is that it means "humans can understand why the model gives a certain prediction".

It seems like you are using it to mean "humans can understand why the thing modeled does something (presumably by referring to the model)". That isn't what I thought people meant by the term, nor does it seem like a reasonable goal to me.

Do we agree these are distinct ideas? Feel free to elaborate on where I've gone wrong.

Reply on news.ycombinator.com