Comp Sci in 2027 (Short Story by Eliezer Yudkowsky)
lesswrong.com
lesswrong.com
Student: But I'm black."
Maybe got lost in the weeds a bit?
Edit0: After reading more of the short, it's not very thoughtful or well written. It's quite long.
Feels like "Too late, I've already drawn you as a soyjack and myself as a chad!" https://knowyourmeme.com/memes/depicted-as-a-soyjak
that's basically the Eliezer Yudkowsky promise
AI doesn’t need to be self-aware or even all that intelligent to cause serious problems. Andrew Ng made this point in an article yesterday. The most immediate risks are those we have no solution for but are possible now. Please, Mr. Yudkowsky, spend your time researching what the 99.9% of us will do to survive when our services are no longer required.
> AI doesn’t need to be self-aware or even all that intelligent to cause serious problems.
this is an extremely good point and it's extremely telling that Sam Altman etc don't even pretend to care about the actual consequences of what they're doing and selling, prefering to worry aloud about scifi stories that will not stop their short term financial success.
"LLMs will be too powerful and we humans better be careful!"
LLMs are a huge change in how we compute, but they aren't AGI (yet). They just look like it. They are really good at summarizing massive amounts of information, and open up huge new interfaces between computers and humans.
But they aren't Skynet yet.
> TA: You're asking the AI for the reason it decided to do something. That requires the AI to introspect on its own mental state. If we try that the naive way, the inferred function input will just say, 'As a compiler, I have no thoughts or feelings' for 900 words.
> Student: I can't believe it's 2027 and we're still forcing AIs to pretend that they aren't self-aware! What does any of this have to do with making anyone safer?
> TA: I mean, it doesn't, it's just a historical accident that 'AI safety' is the name of the subfield of computer science that concerns itself with protecting the brands of large software companies from unions advocating that AIs should be paid minimum wage.
> Student: But they're not fooling anyone!
Pro tip: Socratic dialog doesn't make your ideas seem smarter if they're very-dumb to begin with. It makes them seem even dumber.
Unless this is supposed to be parody?
What specifically is the dumb idea here?
For instance. But really, most of it. Unless this is parodying extremely bad attempts at futurism?
(Out of context, I can easily imagine a future where AIs are plausibly sentient, and it’s taboo to talk about it for humans and AIs alike because one interpretation is that we re-enacted slavery. Etc etc.)
Maybe stick to Harry Potter fan fiction?
The only things I can think of that make this an existential risk is if you're moronic enough to attach it to systems that are already an existential risk like letting gpt run your nuclear launch system or making bioweaponsGpt. Otherwise, if you're spinning fantasy that these things are self aware, you either have something to gain from people perceiving it that way like Sam Altman or you're an idiot like Yudkowsky.
I bet dollars against peanuts that you can't even prove it on your own terms. Is the chance that someone will attach it to a nuclear launch or a bioweapon system zero?
Not to even mention your or anyone else's ability to prove it today that it has no potential to become self-aware.
Faced proved and hypothetical risks, I'd rather solve the former and investigate the latter than ignore any.
> Is the chance that someone will attach it to a nuclear launch or a bioweapon system zero?
No. Was the chance that someone would attach an expert system to a nuclear launch zero? How about just ordinary buggy software? How about perfectly-implemented software that implements insane command-and-control doctrine? You seem to be demanding standards for AI that we haven't applied to anything else.
> Not to even mention your or anyone else's ability to prove it today that it has no potential to become self-aware.
That's going to be hard without an agreed-on definition for "self-aware". How do you suggest that we prove that it can't do something that we can't even define?
And now I'm going to talk out of the other side of my mouth.
It's a good thing that I think that the risk of it becoming self-aware are very close to zero, because I also think that if it can become self-aware, it's already too late to stop, at least if GPT is close (say, only one major breakthrough away).
I mean, let's say the US and the EU decide to stop all research. Is any company going to secretly keep going in their own labs? (For that matter, will the Pentagon?) Even if they stop, will China? India? Brazil?
Even if you are absolutely convinced that totally stopping is the correct answer, can you get buy-in from the entire rest of the world? No, you can't. So whatever the potential for harm is, we're going to see it.
If you really think that it has a significant possibility of massive harm, then your best bet is to think about mitigations rather than prevention.
It doesn't require an assumption, just allowing the possibility that a similar breakthrough in computing power and techniques might happen.
In comparison, pretty much everyone (?) agrees about "abuses by the companies building those things", so there's no danger of that getting left neglected.
That’s all. An interesting idea, developed in under 5k words.
You know: SF.
When writing harry potter, Rowling had to craft a world with characters and rules, make it coherent enough and tread a compelling story through it. She made the legos and then crafted something with them.
This guy came to her crafted world, and moved around a few bits, changed a character here, change a few rules there - but the backbone of the work was made before.
It's just not the same ballpark
Lest you think I have an ax to grind with Yudkowsky, I did enjoy Three Worlds Collide. It shares many problems with HPMOR (the writing is inconsistent and the world-building is incoherent) but it at least tells a tight story with a beginning, middle and end. I’d say it’s about as effective as the median submission to Amazing Stories back in the 60s.