Hacker News Reader

That quote is from 2018.

Here is what Mohammed AlQuraishi said in 2020:

CASP14 #s just came out and they’re astounding—DeepMind looks to have solved protein structure prediction. Median GDT_TS went from 68.5 (CASP13) to 92.4!!!! Cf. their 2nd best CASP13 struct scored 92.8 (out of 100). Median RMSD is 2.1Å. I think it's over [0]

and

“I think it’s fair to say this will be very disruptive to the protein-structure-prediction field. I suspect many will leave the field as the core problem has arguably been solved,” he says. “It’s a breakthrough of the first order, certainly one of the most significant scientific results of my lifetime.” [1]

[0] https://twitter.com/MoAlQuraishi/status/1333383634649313280

[1] https://www.nature.com/articles/d41586-020-03348-4

I can't square this with the actual competition results. There are tons of targets where AlphaFold2 didn't score over 70%. It was typically only 10% ahead of the next best program. The metric everyone is looking at, GDT_TS, gives marks for atoms that are within eight angstroms of where they should be - by experimental standards, eight angstroms is a huge miss! There are targets where on some important metrics, like C-alpha RMSD, AlphaFold2 did worse than another program!

You're making extremely strong claims, counter to scientists with no incentive to praise Google being quoted with reactions such as expecting a mass exodus from computational biology. I, and I assume we, are open to hearing more but I'm not sure cherry-picking a couple examples is enough to credit your claims contra theirs.

For instance, the gentlemen who was presented as a skeptic is instead shown to also say it's solved, and in reply to this you say "everyone" is looking at the "wrong" metric, and it allows errors of 8+ atom widths - he notes the median error is 2.1 angstroms, or 2.1 atom widths.

I feel like there's a certain je ne sais quoi to the comments Google employees write where I can just immediately sense them without even needing to check.

You are not helping your case by calling out people based on where they work instead of what they say. Refulgentis is raising factual points.

Pharma companies don't feel any embarrassment because it's really not much of a priority for them. Protein structure determination simply isn’t a rate-limiting step in drug discovery in general. [1]

[1] https://blogs.sciencemag.org/pipeline/archives/2019/09/25/wh...

That is true but every pharma has a structural biology department that solves product-related crystal structures with the intent of using that data to speed development. It's an established thing, $100+M/year budget per company. Generally well-respected and considered some of the most important IP the company can generate.

They also have theory groups which try to do structure prediction and drug docking but I don't think those groups get any respect inside the companies any more.

Counterpoint: no-one needs to feel any embarrassment. The advancement of human understanding is not a boxing match.

The author acknowledges as much in the preamble, "my mood lifted during the meeting due to the general excitement and quality of discussions, [...] my tribal reflexes gave way to a cooler and more rational assessment".

Tell that to said company's stockholders and get their response :) . In the big picture though is new, correct knowledge is always welcome to the pool.

I think it's arguably the exact opposite. It would have been surprising if medical companies had proven machine learning can model protein folding so much better before the world's leading AI research group could. The demonstration was one of machine learning research being applied to medical problems not medical research being applied to machine learning problems so DeepMind was the one with the head start on expertise.

And let's not forget that deep learning started really exploding and becoming mainstream around 2015-2016. I'm not sure how you could expect someone like Pfizer to hire top experts and build a competitive lab in such a short time, particularly since, Pfizer would want these people to focus exclusively on biotech, which would not be that attractive to most deep learning researchers.

OpenAI was founded in Dec 2015. They have a smaller budget than Pfizer and work on less practical problems.

It's a lack of vision and leadership from pharma labs.

> It's a lack of vision and leadership from pharma labs.

That's probably true, but I don't think Pfizer could have beat DeepMind, even if they had really tried. DeepMind is in a unique position to recruit lots of young deep learning researchers.

> The fact that the collective powers of Novartis, Pfizer, etc, with their hundreds of thousands (~million?) of employees

Deepmind have at least a few thousand employees, I'm willing to go out on a limb and say that the industry doesn't have "hundreds of thousands" of people working on protein folding alone.

well, not everyone at DeepMind is working on protein folding either presumably (disclosure: I work at Google, but I know nothing about DeepMind other than whatever's public). I think the point they're making is that DeepMind has an order of magnitude or two fewer people working on this for a much shorter period of time than pharma companies.

In the video on the announcement page, there was a short scene where they showed a Zoom call where the results were announced. I counted 20 people. The paper they published in Nature has 1 authors, so it matches up. Probably this is not the precise team size, but gives you a rough estimate.

What's important is I think which people they hired, as well as Deepmind-internal infrastructures. You can just walk over to one of the world experts in deep learning and ask them how to address some issue.

In pharma companies they have people who are really smart about drugs and biology, but ML experts would all have to be hired externally, and likely don't represent the top of the field as well as Deepmind does.

Has Deepmind ever released how many employees it has? Several thousand seems much larger than I would have thought. Please cite your source on that.

Here's a June 2020 article that says they have ~1,000 people: https://www.cnbc.com/2020/06/05/google-deepmind-alphago-buzz...

At first it sounded like a lot but when I remembered by FAMANG org of similar size it's actually not that big. A 1000 people org probably nets you around 15-30 teams which have to include frontend, backend, infra, theory, experimental, internal tooling etc. I guess the biggest question is how many people in deepmind are sales/consulting etc. Do they have any external for profit contracts?

There’s the rub. Depending on who you ask DeepMind appears to, as of a year or two ago, be running deep in the red.

How would DeepMind get any revenue? I thought it was solely for research.

DeepMind will always be a loss center. That's what research groups are.

I don't know how UK R&D tax works, but most countries allow companies to write off some profits against R&D. Apart from the fact Google wants to do research, having a large center based in the UK probably helps their UK (and formerly EU?) tax position.

This post is from 2018, when AlphaFold won the CASP competition, but nobody was claiming that it "solved" the problem, as you are now hearing in response to AlphaFold's performance at the 2020 competition.

The same person who wrote said post in 2018 wrote about the 2020 AlphaFold results:

> CASP14 #s just came out and they’re astounding—DeepMind looks to have solved protein structure prediction. Median GDT_TS went from 68.5 (CASP13) to 92.4!!!! Cf. their 2nd best CASP13 struct scored 92.8 (out of 100). Median RMSD is 2.1Å. I think it's over

https://twitter.com/MoAlQuraishi/status/1333383634649313280

This blog post is two years old, whereas the linked blog post for this overall discussion is current.

DeepMind's first entry into the competition was two years ago.

Right, so it's not exactly pertinent to whether the most recent work "solved" (for whatever definition of solved) protein folding.

My impression is that DeepMind's performance have improved significantly since then.

And they have improved significantly in the two years since then, so many if not all of these criticisms that apply to their efforts from two years ago may now be out of date.

You're right, I should have pointed that out. I edited my original post to clarify. Thank you.

I think it is also very embarrassing to IBM, which has been trying to get into healthcare with Watson.

I bet they are hiring BERT-ologists now.