fnbr

I see the opposite fwiw. Numbers significantly lower

These numbers are wildly inaccurate. I spotchecked multiple places that I know the salaries and they're wrong.

Less expensive. Big discounts exist.

A number of large AI companies use it to train their large models; Midjourney, Stability, Anthropic, DeepMind, among others.

The main benefit in my experience is that it’s much easier to do distributed computations in JAX. It has a much nicer API. For single device computing there’s no advantage either way.

Not sure about VTI, but if you invested in the S&P 500 and reinvested dividends, you would have got 130%:

https://dqydj.com/sp-500-return-calculator/

You’re welcome :)

It’s definitely a very cool moment for me :)

Yeah, you’re totally right. I actually wrote a follow up essay about that:

https://finbarr.ca/llms-not-trained-enough/

I think the conversations were partly (largely?) a snapshot in time. I was talking to people in February/March, and all of this was much less thought through at the time. But you’re totally right. You want something like Llama, where you train a smaller model longer than Chinchilla would predict.

This is the original page, I’d link to this:

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

u/dang, swap links if you see this?

I went from being a lifelong fan, especially of the EU, to not caring at all. I’m now a Trekkie.

I was about to ask the same thing. Why does this matter? It's like when the Canadian Parliament subpoenaed Mark Zuckerberg- he just ignored it [1].

[1]: https://www.cnn.com/2019/05/27/tech/zuckerberg-contempt-cana...

man, I want a tiny Japanese pickup truck

Yeah, but Ilya left. Doesn't that prove your point?

Sous vide translates literally to “under vacuum” as it involves cooking food in plastic bags that are vacuum sealed and cooked in water baths at precise temperatures. The technique, however, is much more useful for the temperature control elements than the vacuum. The reverse sear method mimics the temperature control element of the technique, but is just done in an oven, so it’s very much not under a vacuum.

I think this is an excellent resource. I used to work at DeepMind, and Hado & Matteo are two of the best RL researchers there.

I was at google when he led the swift for tensorflow team and it was really unimpressive. Hopefully this does better

I’m doing my part. I got a new work M1 MBP. Then, I was laid off, so I got a new personal M2 air. Then, I got a new job, which gave me a new M2 MBP. North of $10k in laptops.

Oh, and a new iPhone.

Biggest advantage is the high bandwidth interconnect. It’s very painful training large models on multiple GPUs. TPUs help a lot with that, although the software side of things is pretty brutal.

Yes! This is something that is done. The problem is that a) it’s tough to find a sane denominator as the likelihood of the entire sequence can be quite small, even though it’s the best answer and b) the answer isn’t grounded in anything, so the confidence score isn’t super helpful.

A score like this can be useful for active learning though, where you find areas of low confidence in your dataset and get more data to train on.

It is not common for new projects. The vast majority of new projects use PyTorch, with some using tensorflow and some using JAX.

Do the baskets really make that much of a difference compared to stock ones?

I love this (even though I have ~25% of my net worth in $GOOG).

Google got fat and lazy. They focused inwards, on promo driven development and executive infighting. This was awful for the tech ecosystem as Google was held out for a long time as the company to emulate.

No longer.

They’re gonna be forced to actually compete: for talent, for users, for mindshare.

When I was laid off, and all my access to internal systems was disabled, I had no way to get in touch with my former colleagues, who didn’t have my contact info. Having a personal website enabled them to get in touch with me, which led to a number of job offers (including the one I ended up taking).

I’m a naive techie. What, exactly, did Mitchell do wrong here? What could he have done better?

FAANGs aren’t anywhere near the high end of the ML job market currently.

ML engineers work on products, AI research engineers work for AI research labs.

Skillset is almost identical, biggest difference is having research experience.

Yes, 1000%. Especially when it comes to AI. Meta has an excellent team of ML engineers, many of whom are in exceptionally high demand right now. If I were a ML engineer at Facebook, _especially_ if I was working remotely, I'd be sending out a ton of applications right now. I bet they end up getting an easier, more meaningful, higher paying job.

I'm an AI research engineer who was recently laid off and this was my experience.

It was very impressive for an org the size of Microsoft to pivot as quickly as they did.

Salary was still 2x FAANG comp (and roughly equal to my FAANG TC). So I’m fine taking the equity bet under those circumstances. I agree with you though. One company valued their equity at 3x (!!) what it was worth in October.