> That simply is not what AlphaFold (or any homology modeling) is doing
Ha! The reality is we have very little idea of about what any neural net AI is doing. Not no idea as provided the raw material these things build themselves out of, but then we throw billions of training points at it with the desired results, and a vast stew of polynomials begins to self organise into something that produces the results.
How those polynomials do it? We certainly don't know from first principles. If we alter compression in a car engine, or moderators in a nuclear reactor, or tension in prestressed concrete beam and an engineer will be able to make a fair stab at the result. Ask the people who built these vats of self organising what happens if they change the polynomials coefficients - and they will have no idea, whatsoever.
Any claim you know what AlphaFold "is doing" is simply fanciful. We've crossed Aruthor C Clark's threshold of "Any sufficiently advanced technology is indistinguishable from magic". It seemed so reasonable when he said it. What he, and we didn't foresee is how it would look if you were the ones who built the technology. "I've build a machine so complex I don't know how it works" crosses a line even SciFi authors had trouble imagining. And now we are confronting it as as our new reality.
I wonder if the all the comments on HN about how AI's don't really understand the subject, or don't fit some definition of "conscious", or can't really "think" are just a form a self denial. It really doesn't matter people. The reality is the only way people could tell it was GPT-3 posting to reddit wasn't because of any of those things. It was because no human could churn out so much error free high quality text indistinguishable from what a human would produce in such a short time.