Hacker News Reader

Comment by AlbertCory | Hacker News Reader

In the early 2000s, I took a bunch of UCSC Extension courses on mol bio, bioinformatics, and drug discovery. Back then, abundant DNA information was the thing revolutionizing the field.

What the scientists (all from Roche) said was, more or less, "yeah, that helps a lot. It doesn't solve the whole problem, though."

20 years later they've gotten yet more help with Alphafold. Once again, they can do things faster, but it isn't a Moore's Law-type of change. It's still a really hard problem demanding culture, animal, and human tests, and those take time and money.

View on HN

AlphaFold is in part impressive because a team with no protein design experience solved a challenge which had eluded experts for decades.

It's absolutely a step change in demonstrating what's possible. The author can sit and criticize alpha fold from his armchair, but the dude is missing the forest for the trees.

Do you remember how people on HN used to take comfort that AI couldn't write code? Now they're taking comfort in the fact that AI can't write code well.

We're headed towards a word where data gets put in a blender to decide what to try next. That entire pipeline can and will be automated. And in that world, human ingenuity and expertise may become far, far less relevant.

>We're headed towards a word where data gets put in a blender to decide what to try next. That entire pipeline can and will be automated. And in that world, human ingenuity and expertise may become far, far less relevant.

Let's hope we'll be dead by then...

We can’t beat a Ferrari in a foot race either. I’m not sure this future is necessarily bad.

It’s bad if you’re ruled over by a dictator who uses the AI to oppress you and you don’t have any rights. People worry about AI killing everyone but I think the AI panopticon angle is just as likely.

Ferraris don't compete with the essense of being human (human ingenuity and expertise) - just with walking, if that.

The world you are describing has been depicted in the movie Idiocracy.

I think you missed the point. No one is dissing AI or this achievement, per se. At least I'm not.

The point is a great deal of drug development time did not go away when DNA sequencing and bioinformatics became available, and a great deal will not go away with Alphafold.

> We're headed towards a word where data gets put in a blender to decide what to try next. That entire pipeline can and will be automated.

No, we're not, and no, it will not. Alphabet has an entire large organization trying to bring the benefits of Big Data to traditional sciences. It's not a waste of time by any means. It's also not a steamroller ready to crush every problem in its path.

> It's also not a steamroller ready to crush every problem in its path.

That is not particularly convincing, given that they've crushed multiple problems that were viewed as being decades away in just a few years.

> not particularly convincing

It wasn't meant to "convince." It's an assertion.

So, it's going to land humans on Mars in the 2020s, too? How about cheap nuclear fusion?

Some problems are amenable to AI approaches, including many that were thought not to be.

Some are not.

Machine learning is already contributing to figuring out how to get energy from a fusion reactor. https://spectrum.ieee.org/can-ai-make-a-better-fusion-reacto... and https://www.deepmind.com/blog/accelerating-fusion-science-th...

You're changing the question. OP said "they've crushed multiple problems that were viewed as being decades away in just a few years."

"contributing to figuring out how" is not the same as "crushing the problem." It's helpful, of course.

Containment is the hardest problem with fusion reactors, and ML directly addresses it. (Obviously, ML doesn't directly solve fusion.) Now it's likely that there will be a commercially viable fusion reactor in the 2030s, whereas it used to be the case that a proof of concept was perpetually 30 years away.

We could have landed on Mars already if we had funding.

There’s no incentive for the state to fund it so Musk has to do it while still relying on government funding for other SpaceX projects. If we get the National Security elites on board we’ll be there so fast it’ll make your head spin. We could have AGI 2-5 years sooner than we otherwise will (2030) if the government wasn’t run by geriatric humanities majors. These problems are mosyly downsyream from money, lots of it, and you need Manhattan project scale budgets for them.

AGI and fusion also require some theoretical breakthroughs but Mars? Mars doesn’t. Making it livable is hard but just landing there just means scaling up the moon landing.

So... AI can solve every problem, but not that one, because reasons.

Each major advancement does not imply that more major advancements are necessarily coming.

Deep learning did turn out to be a very major advancement! And it's still bearing new fruit. But it does not at all imply that future advancement is inevitable or that advancement must continue at its present rate.

It's also worth considering that the domains where deep learning now excels are the same domains where it excelled when it first became popular (AlphaGo and AlphaFold are perhaps exceptions) and that all this advancement comes from hammering away at the same problem spaces for so many years, with huge budgets. Maybe the next area for advancement is something like combining fast on-device deep learning inference with robotics, and maybe that opens up a whole new world of possibilities. But that's a maybe.

> they've crushed multiple problems that were viewed as being decades away in just a few years.

What are these multiple problems you speak of?

> Do you remember how people on HN used to take comfort that AI couldn't write code? Now they're taking comfort in the fact that AI can't write code well.

No, actually: I am 40, and I don't remember that ever being the case. We could have computers write code back 20 years ago, at least... we've had automated scaffolding, pattern extrapolation, contract generation from test cases, random forests designed to approximate unknown algorithms; the code just wasn't good or useful or some other non-boolean axis, and that is exactly the debate we are having today, despite some clear expansions in the scope of practical applicability.

Yes, exactly. The fundamental problem with computer-generated code is that it’s almost always harder to read code than to write code [0], so making programming involve less writing code and more reading code actually makes the job more difficult.

[0] https://www.joelonsoftware.com/2000/04/06/things-you-should-...

For now

Program synthesis has been around for decades. To make a program synthesizer that can economically compete with humans is the hard part.

The first author of AlphaFold paper is John Jumper. John Jumper wrote a PhD thesis on protein folding.

It is not true that AlphaFold team did not have domain expertise. I would accept that AlphaFold team had less domain expertise than expected. Could it be done with no domain expertise? Maybe, maybe not, but AlphaFold is not an evidence either way.

I exaggerated. Jumper was a few years out of grad school competing against groups like the Baker Lab which are household names in the protein design community.

A lot of faculty shocked when AlphaFold 1 was released. AlphaFold 2 /really/ hit hard because the model used was even more "dumb" in that in leveraged even less domain knowledge in it's construction.

> Do you remember how people on HN used to take comfort that AI couldn't write code?

This was posted four years ago about a precursor of Copilot, there is no comment.

https://news.ycombinator.com/item?id=18147697

> We're headed towards a word where data gets put in a blender to decide what to try next. That entire pipeline can and will be automated.

I personally consider this to be lazy and dangerous thinking. I agree that something like this is going to become increasingly possible for more and more scenarios, but in no way should "putting data in a blender" without understanding a good long term strategy for research and engineering.

The danger of losing fundamental understanding and alternative approaches because there is a low-effort sometimes-good-enough way is real and should not be treated lightly, much less cheered on.

The lesson of AlphaFold is if you take some experienced senior scientists and pay them well to work on a big challenge, they make progress. This is so very different to what actually happens in academic research.

The lead on the project was a few years out of grad school.

The technique is what mattered. Not the experience. There have been many labs dedicated to this problem for years. They never even came close.

Nonsense.

The first and corresponding author on the original Nature paper, Andrew Senior, was not a few years out of grad school[1], he's worked at Google for the last 14 years, and started working at IBM in 1994 [2].

[1] http://www.andrewsenior.com/technical/ [2] https://research.google/people/author37792/

>human ingenuity and expertise may become far, far less relevant.

Human ingenuity produces the data and builds the blender.

>> Do you remember how people on HN used to take comfort that AI couldn't write code? Now they're taking comfort in the fact that AI can't write code well.

If you're referring to neural program synthesis with large language models (LLMs), the performance of current systems is pitifully low, with best results in the range of 5-7% correct programs when evaluating on a test set similar to the one used for training (and allowing a single "guess"; it goes up to 30% with 100 guesses; happy to point to references if required).

So no, "AI" (meaning deep learning I presume) cannot write code very "well".

Of course there exists an entire field of research in program synthesis that predates LLM code generators by a long time, and that can do much better than LLMs, but you haven't heard of it because DeepMind and OpenAI don't choose to champion it. But that's a story for another time.

[Correction: dammit, I misquoted the metrics. OpenAI claims ~30% correct results for their Codex model when allowed one guess on their test set; ~70% when allowed 100 guesses. See link to Codex paper in child comment. The 5-7% is the rate of correct programs for SalesForce's CodeRL on the "Introductory" level problems of the third-party APPS dataset and when allowed one guess; see Table 1b in the CodeRL paper summarising results on APPS for various LLM code generators: https://arxiv.org/abs/2207.01780]

Which benchmarks? OpenAI claims 37% accuracy on common prog synthesis benchmarks

Where is this claim from? The Codex paper claims 28.8% on OpenAI's novel dataset:

On HumanEval, a new evaluation set we release to measure functional correctness for synthesizing programs from docstrings, our model solves 28.8% of the problems, while GPT-3 solves 0% and GPT-J solves 11.4%.

https://arxiv.org/abs/2107.03374

The numbers I quote (5-7%) are on the APPS dataset, reported by SalesForce:

https://arxiv.org/abs/2207.01780

(Recently shared on HN).

Note that there is no standard dataset like ImageNet or MNIST in program synthesis so it's not correct to speak of a "benchmark". LLM code generators in particular tend to be evaluated on whatever dataset their creators thought best demonstrates their system's capabilities.

https://gpt3demo.com/apps/openai-codex They claim the current version has 37% accuracy, they may have continued to improve it, I assume thats on the original dataset they used but I can't find a source for that

Thanks. I guess we'll have to wait for a publication to know what was done exactly.

one of the people (David Jones) who worked on alphafold has been doing neural-network based protein folding for decades.

> It's still a really hard problem demanding culture, animal, and human tests, and those take time and money.

But is there a distinction between drug discovery and drug development? Because in my totally uninformed brain AlphaFold could contribute to the former, but not the latter, which seems to be the focus of this sentence.

Reply on news.ycombinator.com