Hacker News Reader

Comment by api | Hacker News Reader

Has anyone trained a transformer model yet on genotype phenotype mapping? Like feed in loads of human, animal, etc sequences and get it to predict phenotype as accurately as possible.

This seems like it could be the key to genetic engineering. You could probably build something analogous to a diffusion model for organisms that renders genomes or fine tunes them from phenotypic traits.

View on HN

No. In general there are many orders of magnitude more variations throughout the genome than there are observable phenotypes. We don’t know what genes most cis regulatory elements regulate; and we don’t know what most genes do in which contexts. Hell, we don’t even know how many different types of cells there are in the human body.

The problem is data scarcity. You're stuck with a few million rows of data, which just isn't enough given how noisy and subtle the genotype->phenotype relationships are. Neural nets work well on language, Chess, Go, etc, because we have gigantic datasets where the subtle patterns can be teased out without overfitting.

I tried to do this ages ago in 2018 by adapting OpenAI's flow architecture and it sort of seemed to work (was at least promising). With today's models with a significantly more disentangled latent space it should be much easier to do! I saw a transformer trained on the UK Biobank recently, excited for this space!

If such a thing were possible, it might involve some of the techniques from this paper:

HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution

https://arxiv.org/abs/2306.15794

Reply on news.ycombinator.com