Andrej Karpathy is joining OpenAI again | Hacker News Reader

Andrej Karpathy is joining OpenAI again

twitter.com

158 pointsbirriel58 comments

For those who dont know, Andrej has a brilliant series of videos on youtube where he explores the basics of neural nets right up to transformer architecture through code.

Starts here [0] and most recent is building GPT from scratch in code.

https://www.youtube.com/watch?v=VMj-3S1tku0

Absolute Gold!. And he really gets down to the Bits and Atoms which is why its very useful

FYI He was on the Lex podcast [0] ~3.5hrs

[0] https://www.youtube.com/watch?v=cdiD-9MMpb0

Good for him. He seems to be a very thoughtful person that would let the world know if things got out of control.

Presumably the lure of all that hardware was too much to resist!

He barely lasted 6 months as a gentleman of leisure.

Do you think he was on garden leave? Some kind of non-compete?

100% ... i would be shocked if that wasn't written into his contract.

Classic. "100%" certainty but then the guy himself responds and says it's not the case.

there were no non-competes, garden leaves or etc., i just took some time for myself and then ~late last year started to feel an itch again.

Are you going to continue your youtube experience?

He said yes on twitter: https://twitter.com/karpathy/status/1623492347739910145

Really happy for you. I thoroughly enjoyed your YouTube NN series, and your Lex interview.

Foss?

Wow; appreciate the transparency dude and thanks for helping build autopilot and fsd.

I appreciate your videos. Thanks!

Some people enjoy not working some of the time. If that's hard for you to believe, I'd love to know what job you have! :)

makes sense ... karpathy is a pragmatic thinker. openai seems like a pragmatic company.

Can't wait to see want gets built.

As sad as Carmack working for Facebook.

One would expect that bright people would know better, specially after the Manhattan project.

In what ways do you view working for OpenAI as a bad thing?

No good will come from the business model offered by "Open"AI. It is not like the personal computer revolution that created tremendous wealth and opportunity. These companies are not developing "tools" like the personal PC. The models are closed, trained on closed data and cost a fortune to train. All they will ever offer is a paid prompt.

Who do you work for?

> All they will ever offer is a paid prompt.

Last time I checked Carmack never gave his games away for free.

I was referring to ChatGPT. It is a paid prompt.

So what business model should ChatGPT have then?

I seem to be using it for free just fine.

Carmack didn't promise to release doom for free but he delivered in 2012.

OpenAI put it in the name and backtracked almost immediately.

I don't think OpenAI ever promised to provide all of their tech for free.

Did last time you check, you saw that Carmack open sourced every single one of his game engines along the way?

A famous ~$50 millionaire+ programmer who is basically retired and no longer has a strong commercial connection to his original source code, of course can release his code for the public good without much risk. But that's a very different thing that whatever OP is talking about.

Doom source code was released on December 23rd, 1997 (1). That was while it was still relevant.

1: https://en.m.wikipedia.org/wiki/Doom_engine

After 1997 I don't see a single game being developed on the Doom Engine

https://imgur.com/a/fa5PxgR

Again this has very little real connection to a commercial company in its prime releasing OSS and sacrificing massive income.

All you've shown me is a programmer who got wealthy releasing the closed source games via big name publishers, strategically making their own company, then later released the source code for a well-past-its-commerical-prime game at minimal personal risk, in an industry at the time where old games die. Even remastering old IP games wasn't a thing yet.

Meanwhile releasing his engine brought him and his team much personal reputation, showing his amazing programming techniques and talent which helped the industry... even though no one cared to use his old out-of-date open source engine (aka little commercial risk)... which yes was novel at the time and respectable.

What does this have to do with OpenAI exactly? Or top end talent spending their prime at a commercial software company?

And I despise the heavily policed OpenAI approach to models so I have no skin in the game to defend them.

> After 1997 I don't see a single game being developed on the Doom Engine

Because Carmack kept developing new engines that were better. He also made it easy for people to build maps and add onto the game.

You can earn both money and followers by holding back key pieces but still sharing with the community.

Check out the book Masters of Doom.

Shareware for the first episodes and later GPLed the engines.

> All they will ever offer is a paid prompt.

They sometimes open their older models: https://github.com/Const-me/Whisper

I'm not personally too chuffed about corporate hypocrisy, but the more you believe in OpenAI's name and stated mission the more likely you are to be upset with them (and, by extension, this news).

Manhattan project was a huge success, what are you talking about? Or would you prefer nazis or commies building nuclear bomb first?

The H-Bomb was completed after the Nazi defeat.

It's just sad. The Manhattan project was necessary and it was successful. It also developed the technology for clean nuclear energy. But at the end of the day the reality of this world is that technology is a two edge knife. US government promptly bombed Japan, and other countries rightfully rushed to develop their own bombs.

I feel the same with AI. Most people struggle to make ends meet. Meanwhile our brightest minds are developing AI and will hand it over to people who don't care about the rest of us. Models will be closed, data will be closed. Why? You get my point?

It's all Andrej's fault.

Re Manhattan project, somebody else would have figured it out, is there a side you'd rather had got there first? The cat was already out of the bag.

More to the point, openAI makes chatbots that use matrix multiplication to provide believable sentences, and a few similar things. This isn't and never will be nuclear power. Nobody is working or, or close to, any sentient or dangerous AI (dangerous in a way that a regular computer program couldn't be). The applications in contemplation here are better searching, low end copywriting, some kind of smart assistant.

(On a technical level, there's very cool stuff going on, but there is not societal risk, other than the fallout from s big hype bubble)

> This isn't and never will be nuclear power.

Looking forward to this aging like milk. Virtually everyone in the industry believes AGI will happen, and virtually everyone agrees that it is going to permanently change humanity like we've never seen before.

Lol

This just a classical false dichotomy. Because we spend resources on one thing doesn't mean we can't spend resources on another, and it doesn't mean resources for one thing are being taken away from another. It's not a zero sum game.

Time and resources are finite. We cannot allocate the same resource twice.

Andrej came to fame by working for Tesla. It’s not exactly a charity.

It was news at the time because he was already famous. He had a series of videos on YouTube from his course CS231n at Stanford. He also wrote the popular char-rnn article in 2015 and was a founding member of the research group at OpenAI before joining Tesla.

http://karpathy.github.io/2015/05/21/rnn-effectiveness/

He was as famous as Lex Fridman was before his podcast took off.

Sure, some people in the industry knew him, but working for Tesla and selling their dreams of FSD really put him on the map.

Manhattan project was awesome, and thanks to nuclear arms race we actually have an energy source that solves all of our renewable energy needs, human vices and misinformation notwithstanding.

With the popularity of ChatGPT in the news, I've seen on many occasions now where the headline would refer to OpenAI as having been founded or co-founded by Elon Musk. Granted it makes a good lead story inserting Musk into it but how true is this claim? I hope karpathy can put this question to rest: can Elon Musk claim to be a co-founder of OpenAI in the true sense of being part of the founding team? (Recall the Tesla founders story.)

According to the blog entry at https://openai.com/blog/introducing-openai/, the founding members are:

"OpenAI’s research director is Ilya Sutskever, one of the world experts in machine learning. Our CTO is Greg Brockman, formerly the CTO of Stripe. The group’s other founding members are world-class research engineers and scientists: Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, John Schulman, Pamela Vagata, and Wojciech Zaremba."

Elon Musk is mentioned as a donor. And so is AWS, Infosys, and Reid Hoffman.

"Sam, Greg, Elon, Reid Hoffman, Jessica Livingston, Peter Thiel, Amazon Web Services (AWS), Infosys, and YC Research are donating to support OpenAI. In total, these funders have committed $1 billion"

Surely, donors cannot claim to be founders of OpenAI, the same as how VCs cannot claim to be the founders of startups they fund. If this were the case, then Jeff Bezos, owner of AWS then, could have the same claim as a co-founder of OpenAI.

The bit you quoted from https://openai.com/blog/introducing-openai/ goes on to say "OpenAI’s co-chairs are Sam Altman and Elon Musk." Did you not read that far?

Wired had a couple of articles in 2015/2016 about the founding of OpenAI

https://www.wired.com/2016/04/openai-elon-musk-sam-altman-pl...

https://www.wired.com/2015/12/elon-musk-snags-top-google-res...

So? It's not unusual for large donors to become an honorary chair of a company. VCs get this privilege oftentimes.

There is literally nothing in the OpenAI announcement that he's a founder. Websites like Wired and Buzzfeed ran with it because people would shrug if the story went that the founders are Trevor Blackwell, Vicki Cheung, Andrej Karpathy, Durk Kingma, John Schulman, Pamela Vagata, and Wojciech Zaremba. Who? No it can't be them, it's Elon Elon!#

How do you define founder exactly? Does being there for the initial discussions, helping get the team together, being on the board and being a co-chair and helping fund the whole thing count?

Did you also not read that far where it explicitly says who the founders are? I hope you are not confusing founders with funders.

If you really want to split hairs, https://openai.com/blog/introducing-openai/ actually says 'founding members', which is not necessarily the same thing as 'founder'. You are focussing too much on that one sentence from the one announcment. You need to look at what actually happened at the time.

Founder is not a legal term. There is a central registry where you can go and look up who the directors of a company are, but there is no such official place to record founders. Founder is a vernacular term that simply means one who is instrumental in the origination of an organization (or other initiative). For all intents and purposes, if you look at how OpenAI got founded, Musk was one of the founders.

Just my opinion: To me, it sounds like Sam Altman wanted Elons name attached to the project in order to attract talent. Read the wired article I linked above. Fair game, thats an important part of getting the thing started. In return Elon got to embed some of his ideas (e.g. I'm pretty sure the Open bit of OpenAI was partially his influence, back in 2015 he talked a lot about the dangers of AI being developed behind closed doors). Maybe he wasnt very involved once it got started; that would hardly be suprising. But still, he was involved in the founding for very specific reasons.

This feels like super pointless speculation because unless you were there in the room there's no way to know what Elon did or did not contribute to the founding.

Elon was on stage when they announced OpenAI in 2015. He was co-chair with Sam. They presumably both worked to get the founding team in place. The idea of OpenAI was both of theirs. He likely put up more than $100 million for its founding.

Elon wasn't "on the team" any more than Sam at the start. But no one questions whether Sam is a founder.

The point is it's explicit in OpenAI's website who the founders are but they are never mentioned in the press. Rather the narrative becomes Elon founded it. Of course, he'll take credit.

Elon isn't telling journalists to call him a founder.

Reply on news.ycombinator.com