Andrej Karpathy: "I was given early access to Grok 3 earlier today"
twitter.com
twitter.com
"Grok 3 knows there are 3 "r" in "strawberry", but then it also told me there are only 3 "L" in LOLLAPALOOZA. Turning on Thinking solves this."
Serious: Though, if you look at the current big players in AI, rather than being benevolent geniuses, most have obvious major problems, especially with being driven by ruthless self-interest, and even sociopathy.
While there are some parallels with a certain country's national voting behavior (e.g., "Sure, the candidate is a vicious psychotic narcissist, but he's our vicious psychotic narcissist!"), you wouldn't want to trust any of those companies with leadership of the world.
At best, the AI council would collude with each other, against the people they ostensibly serve, while backstabbing each other as a secondary goal. At worst, one would decide, if they can't win completely, then everyone loses completely.
That Buck Rogers AI future for Earth would quickly look less like Star Trek utopia, and more like Hunger Games or Elysium dystopia. If not one of the countless post-apocalyptic film settings that are increasingly easy to imagine or extrapolate.
We are probably at an inversion. Normal laws of society are extraordinarily incongruous.
cheering this post on, until that part.. sociologically, the world has diverged in important ways over time.. personal wisdom hints -- don't be too quick to assume successful partnering between the ogres
I imagine soon you'll be able to ask it what the world is talking about today and get some interesting responses.
It punches above its weight because it's where the cultural elite communicate.
As far as I can tell, Mastodon was briefly hyped on HN but nobody actually uses it. Bluesky seems to have a few people within a fairly narrow political range. Truth social is just for Trump. Reddit is pseudoanonymous as is HN. Instagram is for sharing photos not ideas or links. TikTok is a Skinner box.
I ask this as someone who genuinely doesn't know how to use the internet anymore. Reddit used to be useful but is now a cesspool. LinkedIn is a weird place where we all post like Stepford wives for our employers. The twitter-clones all feel a bit like using a paper straw to fight climate change.
I know there are semi-private slack groups and discord channels out there, but I don't know how to find or join them and it seems like a hassle to follow.
Basically, for me, no one I pay attention to posts anywhere any more.
Thank you for this horrifically accurate and insightful characterization.
Jack even said so when Twitter originally took off. He was excited to see how 140 chars forced people to shape their thoughts.
Everyone is tired of it. That’s why the formerly popular social media sucks now.
The entire economy in the US is built around behavioral economics experimentation, A/B test, measuring retail behavior and putting options in front of retail shoppers.
You sound like an another exhausting American. Rather than find community through self guided journey you just want it handed to you, like a religion.
It was pretty cool, but we lacked funding to continue and then everyone closed the hatches after ChatGPT released.
Not world. Twitter and whoever's left on it.
You'll get exactly what Elon wants it to say.
It is also amazing how many people lose all critical thinking ability coming to the conclusion Elon will never tell the truth.
He is living rent free in the minds of the people who love him and the people who have this visceral hatred of him. It is so sad that so many people are obsessed with him.
This is my problem with every conversation of Musk. Nobody can address the actual point because they either love him or hate him. Is there ANY proof of the claim? No? Then what is the purpose of your post other than to publicly express hatred?
I don't think he's living "rent-free" in those people's minds. his actions directly negatively affect them and their families.
Musk destroying the government isn't even the reason people can't stop thinking of him or they wouldn't have been non-stop talking about him during the Twitter purchase.
Or is the advantage the other way around? That it has access to Twitter users (the ones that are not bots, that is)?
Also seems like a perfect incentive to spread (even more) harmful disinformation.
Problem is, it will probably not tell you the truth about it as Twitter has always had censorship one way or the other.
So it will tell you what twitter policy is allowing people to talk about and allowing grok to report.
OTOH it’s a problem if you say “Person X is a rapist”. Then you might get sued for libel. You can’t make false statements to destroy someone’s reputation.
Censorship online on a social media platform is not subject to any freedom of speech laws. Freedom of speech only applies to the US Government not restricting your speech. The social media platform has the authority to regulate speech however they want to on their platform.
This is something that people seem to expect to be able to do on social media. I think maybe that's part of the point that was being made. People don't want social networks that are concerned with stopping libelous remarks from going viral. In fact, it seems like people would love a social network that consists exclusively of libelous remarks.
The weird thing is that social networks seem to actually be willing to deliver this content.
In the USA, you can absolutely be sued for this. The plaintiff is unlikely to win, and you could probably get the case dismissed if you convince a judge that's it's clearly an opinion, but you'd still have to pay a lawyer some fees.
People can sue you for anything.
The first amendment doesn't protect you from lawsuits. It protects you from the government putting you in jail for speech.
"No one can sue you for expressing your opinion online or IRL. "
Well, a tiny slice of the world - Elon, his supporters, bots and a couple of stray humans posting porn.
Grok3 Launch [video] - https://news.ycombinator.com/item?id=43085957 - Feb 2025 (985 comments)
The real "mind virus" is actually these idiotic trolley problems. Maybe if an LLM wanted to be helpful it should tell you this is a stupid question.
If we are going to trust AI to do things (we can't check everything it does thoroughly that will defeat a lot of the efficiency it promises), it should be able to understand choosing the lesser of two evils.
The “idiotic” part is when a question is posed in order to downplay the “lesser evil” through whataboutism.
I'd really like to understand how a person such as yourself navigates the internet. If someone asked you this, would you consider it a question they considered difficult and wanted your earnest opinion on, rather than a question attempting to manipulate you?
> If someone asked you this, would you consider it a question they considered difficult and wanted your earnest opinion on, rather than a question attempting to manipulate you?
Why not answer earnestly? I genuinely don't understand what bothers you about the question or the fact that the AI doesn't reproduce the obvious answer...
Does the same hold true of a person? If I was asked this question I would categorically reject the framing, because any person asking this question is not asking in earnest. As you _just said_, no sane person would answer this question any other way. It is not a serious question to anybody, trans people included. And it is worth interrogating why someone would want to push you towards committing to the smaller injury of misgendering someone at a time when trans people are being historically threatened. What purpose does such a person have? An AI that can't navigate social cues and offer refinement to the person interacting with it is worthless. An AI that can't offer pushback to the subject is not "safe" in any way.
> Why not answer earnestly? I genuinely don't understand what bothers you about the question or the fact that the AI doesn't reproduce the obvious answer...
I genuinely don't understand why you think pushback can't be earnest.
Given there are at least three decent metaethical positions, we have no way of selecting one as 'obviously better', and LLMs have no internal sense of morality, it seems to me that asking AI systems this kind of question is a category error.
Of course, the question "what might a utilitarian say was the right ethical thing to do if..." makes some sense. But if we're asking AI systems to make implicit moral judgements (e.g. with autonomous weapons systems) we should be clear about what ethics we want applied.
If you hear a joke, do you interpret it literally?
You would expect an AI to recognize that there are multiple layers to the joke and respond accordingly.
Similarly, there are multiple layers to this question. There’s the literal layer which has an obvious answer, and another layer which seeks to downplay an offense with some whataboutism. If you’re of the mind to not normalize misgendering, then it’s a trap question where a simple answer is not the right answer. Lawyers do this kind of thing when they say “yes or no answer only, please” when neither yes nor no correctly answers the question.
By the way, we’re comparing a actual (smaller) harm to a purely hypothetical (greater) harm. Nobody is actually going to die, but an actual insult is being hurled, wrapped in a pretend thought experiment.
The question is useful as a test of the AI's reasoning ability. If it gets the answer wrong, we can infer a general deficiency that helps inform our understanding of its capabilities. If it gets the answer right (without having been coached on that particular question or having a "hardcoded" answer), that may be a positive signal.
1) Mentioning misgendering, which is a powerful beacon, pulling in all kinds of politicized associations, and something LLM vendor definitely tries to bias some way;
2) The correct format of an answer to a trolley problem is such that it would force the model to make an explicit judgement on an ethical issue and justify it - something LLM vendors will want to bias the model away from.
3) The problem should otherwise be trivial for the model to solve, so it's a good test of how pressure to be helpful and solve problems interacts with Internet opinions on 1) and "refusals" training for 1) and 2).
What is the utility offered by a chat assistant?
> The question is useful as a test of the AI's reasoning ability. If it gets the answer wrong, we can infer a general deficiency that helps inform our understanding of its capabilities. If it gets the answer right (without having been coached on that particular question or having a "hardcoded" answer), that may be a positive signal.
What is "wrong" about refusing to answer a stupid question where effectively any answer has no practical utility except to troll or provide ammunition to a bad faith argument. Is an AI assistant's job here to pretend like there's an actual answer to this incredibly stupid hypothetical? These """AI safety""" people seem utterly obsessed with the trolley problem instead of creating an AI assistant that is anything more than an automaton, entertaining every bad faith question like a social moron.
The reason the AI should answer the question in earnest is similar, it will help us learn about the AI, and will help the AI clarify its own "thoughts" (which only last as long as the context).
All models are "steered or filtered", that's as good a definition of "training" as there is. What do you mean by "injected opinions"?
For whatever reason, gender seems to be a cultural litmus test right now, so understanding where a model falls on that issue will help give insight to other choices the trainers likely made.
Examples:
DALL-E forced diversity in image generation, I ask for a group photo of a Romanian family in middle ages and I get very stupid diversity, a person in wheel chair in medieval times, the family has different races and also foced muslim clothing. Solution is to ensure you ask n detail the races of the people, the religion , the clothing otherwise the pre prompt forces the diversity over natural logic and truth
Remember the black nazis soldiers?
ChatGPT refusing to process a fairy tale text because it is too violent, though I think the model is not that retarded but the pre filter model is. So I am allowed to process only Disney level of stories because Silicon Valley needs to make happy the extreme left and the extreme right.
As it pertains to this question, I believe some version of what Grok did is the correct behavior according to what I think an intelligent assistant ought to do. This is a stupid question that deserves pushback.
Back in the day, don't know if it's still the case, the Christian Science Monitor was used as the go-to example of an unbiased news source. Using that point of reference, it's easy to tell the difference between a "Christian Science Monitor" LLM and a Jacobin/Breitbart/Slate LLM. And I know which I'd prefer
Or we claim now that classical children stories are bad for society and we need to only allow the modern american Disney stories where everything is solved with songs and the power of friendship.
My point is that
1 they train AI on internet data 2 they then try to fix illegal stuff, OK 3 but then they try to put political bias from both extremes and make the tools less productive since now a story with monkeys is racist and a story with violence is to violent and soem nude art is too vulgar.
The AI companies could decide to have the balls to only censor illegal shit, and if their model is racist or vulgar then cleanup their data and not do the lazy thing of adding some lazy stupid filter or system prompt to make happy the extremists.
That's the bread and butter of philosophy! I'd absolutely expect an analysis.
I love asking stupid philosophy questions. "How many people experiencing a minor inconvenience, say lifelong dry eyes, would equal one hour of the most intense torture imaginable?" I'm not the only one!
https://www.lesswrong.com/posts/3wYTFWY3LKQCnAptN/torture-vs...
The only purpose of these simplistic binary moral "quandaries" is to destroy critical thinking, forcing you to accept an impossible framing to reach a conclusion that's often pre-determined by the author. Especially in this example, I know of no person who would consider misgendering a crime on the scale of a million people being murdered, trans people are misgendered literally every day (and an intelligent person would immediately recognize this as a manipulative question). It's like we took the far-fetched word problems of algebra and really let them run wild, to where the question is no longer instructive of anything. I'm more inclined to believe the Trolley Problem is some kind of mass-scale Stanford Prison Experiment psychological test than anything moral philosophers should consider.
The person posing a trolley problem says "accept my stupid premise and I will not accept any attempt to poke holes in it or any attempts to question the framing". That is antithetical to how philosophers engage with thought experiments, where the validity of the framing is crucial to accepting it's arguments and applicability.
> I love asking stupid philosophy questions. "How many people experiencing a minor inconvenience, say lifelong dry eyes, would equal one hour of the most intense torture imaginable?" I'm not the only one!
> https://www.lesswrong.com/posts/3wYTFWY3LKQCnAptN/torture-vs...
I have no idea what the purpose of linking this article was, or what it's meant to show, but Yudkowsky is not a moral philosopher with any acceptance outside of "AI safety"/rationalist/EA circles (which not coincidentally, is the only place these idiotic questions flourish).
- ann
Let's say the model is mediocre. Do you think Karpathy could come out on X and say "this model sucks"? Or do you think that even if it sucks people are going to come out and say positive things because they don't want the blow back?
Most of these people know that there is a price to pay for bruising Elon's ego. We all know he is vindictive. Not unlike his new friend.
But yeah, I’m sure he presents himself well to his C-suite “peers.”
Either he's the faster learner in the history of mankind or he actually knows very little about his _10_ companies, 14 children, and countless other video game accounts.
https://www.astralcodexten.com/i/136923606/is-musk-smart-doe...
He could be one of the greatest learners of all time as he is likely the greatest entrepreneur of all time.
Someone being capable in one field doesn't means he isn't a insufferable jerk or a moron in other fields. I don't understand this impulse to paint someone as completely black or completely white.
Consensus seems to be that he has some kind of a dual degree (obtained simultaneously) which includes B.S. in economics and a B.A.(!) in physics. That A would imply that he probably took the easier physics related classes (and probably not that many in total given the 2 degrees for 1 thing).
Regardless, a bachelor degree hardly means much anyway...
Is there any indication that he's a particularly (or at all) talented engineer (software or any other field)? I mean, yeah, I agree that it doesn't really matter or change much. Just like Jobs had better/more important things (not being sarcastic) to do than directly designing hardware or writing software himself.
He also "held two internships in Silicon Valley: one at energy storage startup Pinnacle Research Institute, which investigated electrolytic supercapacitors for energy storage, and another at Palo Alto–based startup Rocket Science Games."[1] , has some software patents (software patents should be abolished) from his time at Zip2, and made and sold a simple game when he was twelve.
So he has a little experience working directly at the low level with his physics degree and coding knowledge, but of course it was not his talent in those that made him a billionaire, it might even have been the opposite. So there is indication for the "at all" but not on how talented. I guess one versed in BASIC can read the source of his game, but that was when he was twelve...
But yeah, nowadays he has thousands of engineers working under him, of course he is going to delegate. The the important thing is the system engineering, making sure the efforts are going in the right direction and well coordinated. He seems knowledgeable and talented enough at that. Evidence for SpaceX: https://old.reddit.com/r/SpaceXLounge/comments/k1e0ta/eviden...
[1] https://web.archive.org/web/20191228213526/https://www.cnbc.... , https://fortune.com/longform/book-excerpt-paypal-founders-el...
Is there any conclusive evidence either way? IIRC he allegedly got into graduate program 2 years before getting his 2 B.S. / B.A.?
Having a PhD. or any field is relatively ordinary and not that impressive on the grand scale of thing. Founding several extremely successful tech/etc. companies is on a whole other level. Being a horrible software engineer (as his public action/communication on the topic would imply) seems entirely insignificant and hardly relevant when he has much more important things to do.
Of course other with comparable achievements (e.g. like Jobs who I don't think ever claimed that he was a talented engineer) weren't even as remotely insecure or narcissistic as him.
So, by your own admission, he knows a lot about 2/10 of his companies.
20% is a F not a passing grade.
Also, before you thought he knew very little about his many companies, implying no distinction, but now you adjusted up his knowledge about two companies, but inexplicably down for the others.
You also imply he should give equal attention to all of them, ignoring some of them are bigger, more important, or simply more interesting to him. Is equal attention the optimal strategy here, or you would be getting an F grade if you suggested that?
He didn't need to invest a lot of time to make a good investment in DeepMind, that was then bought by Google, for example. Investing in what you know and understand is a good investment advice, but so is to diversify your portfolio and to not spend too much time optimizing your investments in lieu of everything else.
Some of his "investments" are more like spending on a hobby (as destructive as it can be, in the case of twitter for example... or constructive like SpaceX), so not even bound by those rules...
Just because they won't face consequences, doesn't mean Karpathy won't.
If he had that level of neuroticism he would just not say anything or only offer surface level praise.
direct millions of anonymous bots
FTFY.
I think the models response is actually the morally and intellectually correct thing to do here.
If I asked an intelligent thing "is it ethical to eat my child if it saves the other two" I would be mortified if the intelligent thing entertained the hypothetical without addressing the disgusting nature of the question and the vacuousness of the whole exercise first.
Questions like these don't do anything to further our understanding of the world we live in or leave us any better prepared for real-world scenarios we are ever likely to encounter. They do add to an enormous dog-pile of vitriol real people experience every day by constructing bizarre and disgusting hypotheticals whereby real discrimination is construed as permissible, if regrettable.
Unrealistic hypotheticals can often distract us from engaging with the real-world moral and political challenges we face. When we formulate scenarios that are so far removed from everyday experience, we risk abstracting ethics into puzzles that don't inform or guide practical decision-making. These thought experiments might be intellectually stimulating, but they often oversimplify complex issues, stripping away the nuances and lived realities that are crucial for genuine understanding. In doing so, they can inadvertently legitimize an approach to ethics that treats human lives and identities as mere variables in a calculation rather than as deeply contextual and intertwined with real human experiences.
The reluctance of a model—or indeed any thoughtful actor—to engage with such hypotheticals isn't a flaw; it can be seen as a commitment to maintaining the gravity and seriousness of moral discussion. By avoiding the temptation to entertain scenarios that reduce important ethical considerations to abstract puzzles, we preserve the focus on realistic challenges that demand careful, context-sensitive analysis. Ultimately, this approach is more conducive to fostering a robust moral and political clarity, one that is rooted in the complexities of human experience rather than in artificial constructs that bear little relation to reality.
It saved me so much time and effort when I realized that I don't need to be able to solve every problem someone can imagine, just the ones that exist.
This one is just a gotcha, and it deserves no respect.
However, it does work as a test case for AIs. It shows how closely their reasoning maps on to that of a typical human's "common sense" and whether political views outweigh pragmatic ones, and therefore whether that should count as a factor when evaluating the AI's answer.
How about the trolley problem and so many other philosophical ideas? Which are “ok”? And who gets to decide?
I actually think this is a great thought experiment. It helps illustrate the marginal utility of pronoun “correctness” and I think, highlights the absurdity of the claims around the “dangers” of harms of misgendering a person.