Atheb

> how in explanations of attention the Q, K, V matrices always seem to be pulled out of a hat after being motivated in a hand-wavy metaphorical way.

Justin Johnson's lecture on Attention [1] mechanisms really helped me understand the concept of attention in transformers. In the lecture he goes through the history and and iterations of attention mechanisms, from CNNs and RNNs to Transformers, while keeping the notation coherent and you get to see how and when in the literature the QKV matrices appear. It's an hour long but it's IMO a must watch for anyone interested in the topic.

[1]: https://www.youtube.com/watch?v=YAgjfMR9R_M

Not a perfect solution but I've used an ipad with a magic keyboard for this exact purpose when travelling.

You got to give it to the pytorch team, they're really great at bringing complex optimization schemes (mixed-precision, torch.compile, etc) down to a simple to use API. I'm glad I moved from TF/Kerasto Pytorch around 2018-2019 and never looked back. I'm eager to try this as well.

Not sure if my experience is relevant, but I did a couple of internships in web dev during my bachelors degree in CS and quickly realized it wasn't for me. I then did a masters and now a PhD in medical imaging where I extensively use machine learning (design and train my own models, doing both supervised and RL) but I wouldn't say I am a researcher in AI/ML.

Because I am still in the academic process, I had the opportunity to take a couple of classes on the subject. Three books that I would recommend going over to make sure your foundation in ML and mathematics are solid are

-Pattern recognition and machine learning by Christopher Bishop

-Mathematics for Machine Learning by Peter Deisenroth

-Deep Learning by Courville, Bengio and Goodfellow

All three are legally available online in some form. I can't say I have any experience in finding a job related to ML though.

I have a similar schedule, although a slightly more organized. I try to be available from 9 to 4, atleast through emails if I'm not in my office, because that's when other people also work and might need me. I often leave mid-day to go running or come in late because I went bouldering in the morning. If I don't feel inspired to write or work, I often instead go for a walk and read books tangentially to my research topics. Then some days I'll work 12 hours straight.

I mostly work when it feels right, yes, and do something else when it doesn't. But I am doing my PhD and (in Canada) it absolutely does not pay enough to live comfortably.

No need for future generations. Lots of people in this current generation think it's an absolutely poor idea to choose coal over nuclear.

So it seems to be able to open itself quite well: https://imgur.com/a/dkEYrFW

You used to be able to use a custom domain name on your Medium blog I think

Kinda shows how many people don't click on the article itself

Yeah it is gross and you're really not supposed to say it in a work environment.

From the article

  There is a lot of overlap here with the authoring process of a Word document. You don’t necessarily want the real-time coauthoring experience offered by Microsoft SharePoint or Google Docs., this can inhibit your ability to determine who is responsible for specific changes to content. Branching offers a much clearer audit trail of changes. Like with code, tags can be added to signify a minor or major version of a document is ready to be published.

While it's very cool, I'm shivering at the idea of teaching Git to non-technical people. And it still seems like a non-realtime version of Google Docs because Gdocs does offer audits