Hacker News Reader

Comment by matusp | Hacker News Reader

I don't think they are taken that seriously. They are writing their op-eds and fiery papers, but the field marches forward nevertheless. But I agree that the whole fields seems like ivory-tower types trying to hijack the latest hottest technology an become the decision makers based on their "exprertise". AI as a field always attracted all kinds of weird individuals, probably because it is such a popular and influential concept. If we would work with their ethics, we would never allow computers to reach general population, because it is simply too dangerous and a lot of "harm" can be done. Their use will be heavily monitored and a license or an assessment would be required to use some of the functions, e.g., connect to the Internet.

View on HN

Don't be silly. Computer proliferation is not an existential risk.

—

There is exactly one correct analogue to AGI risk in our history, and that is the risk of an all-out nuclear exchange. And surprise surprise, we as a species took that risk pretty damn seriously once we realized what the stakes were. Do you think that the considerable efforts to avoid nuclear proliferation, indeed even the use of a single low-yield tactical nuclear warhead in anger, are misguided?

Nuclear risk is a contained problem. Access is extremely limited and controlled by low tech hardware locks. The technology is understood, we know what makes bombs explode and it is predictable by physics.

AI is the opposite of all these characteristics. Its access is nearly everyone. The technology is not understood and not predictable. It is a nuclear warhead that thinks for which we don't know what might make it explode.

Yes, although I wouldn't go as far as to claim that current AI technology has any sort of "accessible to all" path to AGI. Accessible to irresponsible actors with lots of money? Possibly, and that's the real risk. But at least it's not an absurd comparison unlike the GP's.

Unlimited power makes everyone irresponsible actors. Nvidia already stated they expect a million fold processing power in 10 years for AI.

> Computer proliferation is not an existential risk.

AI is created and run on computers so AGI risk must by definition be a subset of computer proliferation risk

I'm amazed to see how common this view is on HN. You'd think the rapid progress made in AI would give more validity and attention to these concerns raised about risk and alignment.

The people doing this work are mediocre at best, if they could push the state of the art they'd being do that. It's a bunch of second handers trying to control those who actually understand what is going on.

Can you point to any evidence or examples of this? I don't find it a convincing counterargument at all.

We need something more like the proactionary principle instead of the precautionary principle: https://web.archive.org/web/20120203050245/http://www.maxmor...

Yes. The AGI teacher is a recent graduate with a Masters in physics. None of the material in this so called "course" is helpful with actual AI research or industry work.

AI "alignment" is a made up field ride with crackpots and charlatans.

The creator of the course was a research engineer at DeepMind and now works at OpenAI. Several people at those orgs have acted as facilitators for the course since its creation.

The purpose of the course is to give people an introduction to AI risk so that they can start getting into the field. Some who take it have been ML engineers for a while and some are just learning about the field.

The expectation was never to consider yourself a full-fledged alignment researcher once you finish this course, but instead to get people started. Many people will spend a bunch of time doing things like implementing GPT from scratch and replicate papers after this to try to get an internship at one of the big orgs. And then do other things to slowly make their way to producing great research. Sometimes ya’ll just want to hate on this site.

I am talking about the people conducting the course - Bluedot impact. The cofounder and DRI for AI safety is who I am talking about

https://www.linkedin.com/mwlite/profile/in/jamie-bernardi/

The researcher who has apparently created the syllabus is also a fresh PhD with 3 years of work experience.

So, what are they wrong about?

If you like to pay for an AI ML course being taught by someone whose entire ML education is 6 months of independent self teaching (as per his own LinkedIn) go ahead, be my guest.

The course is free and funded by Open Philanthropy.

It's not taught by James solely, but there's weekly discussion groups on readings chosen by a mix of alums from OpenAI and Oxford's Future Of Humanity Institute.

I mean, I'm not sure I'd require people who study and advocate for nuclear disarmament to also be publishing papers on how to improve bomb yields in order to treat them as credible. I'd probably expect them to study game theory and geopolitics.

"None of the material in this nuclear disarmament course even covers how to build a modern hypersonic missile!"

Reply on news.ycombinator.com