lucidrains

if one prefers video, Yannic Kilcher does an excellent explanation of the seminal paper https://www.youtube.com/watch?v=iDulhoQ2pro

yea definitely, the ideas are orthogonal

not even. like those MasterCard commercials, you could say it is, priceless...

use CLIP to generate a title for each piece of art, followed by a description with GPT, and it is ready for exhibition

Accompanying blog post https://arankomatsuzaki.wordpress.com/2021/06/04/gpt-j/

Lol, thanks for mentioning Ice Cream

Thanks for the kind words :) Hope you train something amazing with the code

Thanks :D

Thanks for the kind words and glad you found them useful :)

Ryan's Big Sleep notebook https://colab.research.google.com/drive/1NCceX2mbiKOSlAd_o7I...

Aleph2Image notebook https://colab.research.google.com/drive/1Q-TbYvASMPRMXCOQjkx...

I have a working repo for that

https://github.com/lucidrains/dalle-pytorch

It just needs to be trained

You can get a taste of the mentioned technology in the article today. https://github.com/lucidrains/deep-daze https://github.com/lucidrains/big-sleep

"It is difficult to get a man to understand something, when his salary depends upon his not understanding it."

Amazing day for structural biology! If it weren't for the pandemic, I would be out at the bars celebrating tonight!

By the time we can prompt large models and it gives us everything we want, then our work is clearly done. Before then, there's still work left.

Why not learn some deep learning and participate in the coming automation? You are in a better position than most given it is just code

Exactly.

I moved on to molecules https://thischemicaldoesnotexist.com Perhaps short stranded proteins are next

Hi all, this was trained by Michael Friesen (https://twitter.com/MichaelFriese10). I liked the latent space so much that I decided to share it with everyone. Have fun! I'll add some html to credit Michael with the training once I find the time.

I see, no worries then :)

Your expertise is acknowledged. We are in agreement

I see. My point is that all these efforts are really validating deep learning as an approach for solving previously intractable biophysics problems. Science is a series of stepping stones.

Not everyone is focused on solving folding. It's just one step of many. https://www.nature.com/articles/s41592-019-0666-6

https://github.com/yangkky/Machine-learning-for-proteins

I think what they did in the paper is to increase the chances by multiple rounds of hashing, up to 8 times. They show experimentally that 8 times was good enough to be equivalent to a regular transformer.

The author of the article you linked also has this repo, to welcome the public to start training https://github.com/aqlaboratory/proteinnet , following in the same veins as Imagenet

It absolutely did. Both AlphaFold and RaptorX (another deep learning approach from Jinbo Xu's lab) were mentioned in that paper as the main motivation of their architecture

For those interested, it appears as though David Baker (who dedicated his life to protein folding) has also turned to deep learning. His lab recently published https://www.biorxiv.org/content/10.1101/846279v1, which seems to outperform Alphafold with a very concise architecture. Code and model is at https://github.com/gjoni/trRosetta