lucidrains

bought "Mastering Nim" just yesterday!

you are probably thinking of https://arxiv.org/abs/2207.03481

for inference, there is https://github.com/bigscience-workshop/petals

however, both are only in the research phase. start tinkering!

Yannic and the community he has built is such an educational force of good. His youtube videos explaining papers have helped me and so many others as well. Thank you Yannic for all that you do!

they are missing a cute mascot

like gophers for go, crabs for rust, dinosaurs for zig

well, you are assuming these efficient matrices cannot become superhuman teachers that show us the inner workings of the universe in a more optimal way.

personalized RL agents that finds aesthetic trajectories through the music latent space... soon, i hope :D

it is neither as simple as the person you are responding to, nor as complicated as you make it seem. it will only get simpler with time.

in some medical circles, alzheimers is referred to as "type 3 diabetes"

Recommend reading "Man's Search for Meaning" by Viktor Frankl.

Kurosawa film, "Ikiru" is worth a watch too

Otherwise I believe finding purpose is a deeply personal journey for every single person on the planet. Good luck!

True, it goes against the spirit of deep learning

Still the Transframer results are interesting and has encouraged me to look into it

yup, training on DCT representation in general is an untapped research direction. can also help with larger images, longer audio, etc

yea of course https://video-diffusion.github.io/

we are going all the way to the holodeck (my guess is latent diffusion on weights, aka hypernetwork, of a 4d NERF)

out of curiosity, which hospital is this?

so is dreambooth worth open-sourcing then given textual inversion?

already done https://twitter.com/nousr_/status/1563912133120647168

Ikr! Hang on, this ride is about to get crazy :)

It is from the Heidelberg giants, Patrick Esser and Robin Rombach

They continued building off their latent diffusion direction (encode with vqgan-vae and then diffusion in latent space)

All roads lead to rome

https://github.com/CompVis/stable-diffusion https://arxiv.org/abs/2112.10752

To add to this, the deep learning field has already moved on towards MSA-less structure prediction. None of this would be possible without building on top of the work open sourced by Deepmind.

https://www.biorxiv.org/content/10.1101/2022.07.21.500999v1 https://www.biorxiv.org/content/10.1101/2022.07.20.500902v1

To be overly dismissive is to lack imagination.

Imagen actually shows some of the components in DALLE2 is unnecessary, so Imagen will end up being easier to build. I'll definitely add the dynamic thresholding trick from Imagen to DALLE2 repository though; that is a finding that should boost any DDPMs using classifier free guidance.

"Loneliness is not cured by human company. Loneliness is cured by contact with reality." - Anthony de Mello

hey thanks! :^) hope someone makes the next big discovery with them

have an implementation of this over at https://github.com/lucidrains/memorizing-transformers-pytorc..., for any researcher exploring retrieval and memory with attention networks

Attention is all we need

incorrect, they modified the license so it can be used for commercial use - https://github.com/deepmind/alphafold/commit/8173117130e6df8...

revised scaling laws https://arxiv.org/abs/2203.15556 https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scalin...

yes, they are most likely finetuning with this type of pretraining https://arxiv.org/abs/2103.10360 quite easy to build

https://alphacode.deepmind.com/ gives you a glimpse inside of what emerged from a similar attention net trained on code. however, whether the attention net has been forced upon pixels, language, amino acid sequences, the resultant representations are a bit beyond human reasoning, even if we can examine what individual attention heads are 'looking' at

I would recommend Routing Transformer https://github.com/lucidrains/routing-transformer but the real truth is nothing beats full attention. Luckily, someone recently figured out how to get past the memory bottleneck. https://github.com/lucidrains/memory-efficient-attention-pyt...

Thanks for the kind words, and credit goes to Ryan Murdock for Big Sleep and Deep Daze. I simply packaged it up to spread the usage