Hacker News Reader

Comment by qumpis | Hacker News Reader

I'm yet to see an ML PhD be required to learn chemistry to a similar extent that chemists would need to doing ML (especially at research level)

View on HN

That's because application and research are quite different. If one does a PhD in ML they learn how to research ML. Someone with a PhD in chemistry learns how to research chemistry, they only need to apply ML to that research

Back when O'Reilly was still hosting events (sigh), at one of their AI conferences, someone from Google gave a talk about differences between research/academic AI and applied AI. I think she had a PhD in the field herself but basically she made the argument that someone who is just looking to more or less apply existing tools to business or other problems mostly doesn't need a lot of the math-heavy theory you'll get in a PhD program. You do need to understand limitations etc. of tools and techniques. But that's different from the kind of novel investigation that's needed to get a PhD.

>>math-heavy theory you'll get in a PhD program

Lol. With the exception of niche groups in compressed sensing, math doesn't get too hard. Furthermore, ML isn't math driven in the sense people are trying things and somebody tries to come up with the explanation after the fact.

Well I think the issue is more of if you’re Genentech and you need ML people and can’t afford to pay them you’re probably better off retraining chemistry PhDs.

What if they don’t need “ML people”? Computational biology has been a thing for a while.

Well they had a whole suite of presentations at NeurIPS that suggests they hired a bunch.

https://www.gene.com/scientists/our-scientists/prescient-des...

They could afford them then…

I think you missed my point. Genentech, AFAIK, was not doing research on machine learning as in the principles of how machine learning works and how to make it better. They do biotech research which uses applied machine learning. You don't need a PhD in ML to apply things that are already known

As a PhD student working on core ML methods with applications in chemistry, I second this. During my PhD, I read very few papers by chemists that were exciting from a ML perspective. Some work very well, but the chemists don't even seem to always understand why they made the right choice for a specific problem.

I don't claim that the opposite is easy either. Chemistry is really difficult, and I understand very little.

Genentech has several ML groups that do mostly applied work, but some do fairly deep research into the model design itself, rather than just applying off-the-shelf systems. For example, they acquired Prescient Design which builds fairly sophisticated protein models (https://nips.cc/Conferences/2022/ScheduleMultitrack?event=59...) and one of the coauthors is the head of Genentech Research (which itself is very similar to Google Research/Brain/DeepMind), and came from the Broad Institute having done ML for decades ('before it was cool').

They have a few other groups as well (https://nips.cc/Conferences/2022/ScheduleMultitrack?event=60... and https://neurips.cc/Conferences/2022/ScheduleMultitrack?event... and https://neurips.cc/Conferences/2022/ScheduleMultitrack?event...).

I can't say I know anybody there who is doing what I would describe as truly pure research into ML; it's not in the DNA of the company (so to speak) to do that.

Sadly a lot of foundational ML research works for single-label image classification and not much else. ImageNet is a niche problem and way too much ML research is over-indexed on it. If you can make your problem look like ImageNet, you're going to do OK, but if not you effectively need to re-invent the wheel...

What you wrote was true until 4-5 years ago. These days almost all foundational ML research is about generative models.

It's still problematic.

The HEAR benchmark is a great eye-opener. They have basically three classes of audio tasks, and find very different models excel in each, with the best overall models being kinda-mindless ensembles of the ones that do well on particular problems.

So if you've got something that works well for text... it'll take a couple years and maybe an entire new branch of research (diffusion!) to work well for image generation. I have no idea what generative models for chemistry will look like, but will happily bet that it takes some significant specialized effort.

I see two things currently happening in ML:

1. The era of ML benchmarks is ending. New models have to be and will be evaluated the same way human experts are evaluated.

2. Foundational models are becoming multi modal. There will be no separation of text and image generation. Sure, different methods will be used for each, but the learned representations of visual and textual objects in models like Stable Diffusion already live in the same conceptual space.

I don’t think there will be specialized generative models for chemistry two years from now. There will be GPT-5 (and similar competitors) which will be used to perform all kinds of research, including chemistry.

I think this is a bit over confident, though...

For example, AlphaFold just fundamentally isn't a language model, but is fundamentally useful. We'll still need these models that Do Stuff in many areas, and that will still involve benchmarks... Even if we're able to ask GPT-N+1 to design the next version of the model for us.

You can get an ML PhD doing applied ML.

I’m pretty sure the author is implying that the new crop of ML PhDs are just not a smart group of people - at least the level of intelligence required to do truly transformative things with ML in any field.

I think what you’re saying is a commonly found attitude that relates to this topic: it’s pretty limiting to think a cursory knowledge of a field is sufficient to go change it. That’s likely why most “use ML to solve x” projects fail when some like AlphaFold succeed because the ML engineers truly understood the fundamental tenets of the topic and exploited it.

    AlphaFold succeed because the ML engineers truly understood the fundamental tenets of the topic and exploited it

What makes you think it wasn't chemists or biologists who learned enough ML to solve the problem?

Because the chemists and biologists have been doing that for decades without success.

maybe he just say do not expect people resolve problems outside of their discipline.

I don’t understand what you mean. Here’s how many applied ML papers work: create a new dataset for a novel problem, download a PyTorch model, point model at dataset directory. Is it novel? By construction. Is the ML technique novel? No.

I actually meant exactly this. People apply ML to their domains and say thay that this is similar to having a chemist with a PhD.

Reply on news.ycombinator.com