Hacker News Reader

Comment by JumpCrisscross | Hacker News Reader

> best outcome right now is for OpenAI to open source GPT-4 architecture and its training data, then release the weights. Would at least take LLM dominance away from Microsoft.

While undermining its safety argument. Better: open it to partners, subject to vetting and contractual terms regarding its use, in exchange for cash and/or compute.

View on HN

> While undermining its safety argument

Which is just political marketing for regulatory barriers to entry lobbying.

It was the legit raison d'être of the company when it was founded, the safetyism fiesta predates the legislative entrenchment.

I tend to agree. But we just watched folks vaporise tens of billions of dollars to defend that view. In light of that, I’m willing to give it a little credence.

There are multiple factions hiding behind safety. The regulatory capture faction and the BuildingGodInOurImage factions both currently say similar things to the public, but the underlying motivations are very different.

But neither seem likely to want to open anything meaningful that is is AGI related-- locking the public out of powerful AI is a point in which all the major 'safety' factions are aligned. (well not the monopolization concern faction, but they're not a player here and usually don't hide behind the safety banner)

If it's that unsafe, it shouldn't be in the hands and control of a clearly incompetent private company.

Has ANYONE been able to articulate what the "safety" concern is or more likely is it just companies trying to build an artificial moat around a statistical model built on free/stolen data?

Currently the for-profit models are Nerfed to not allow bigoted or sexist (against women) text to be generated but that seems to be it. What kind of text are we being protected from?

The main danger (but certainly not the only danger) is that the AI kills us all for reasons that have been articulated for 20 years including in books by respected academics like Nick Bostrom. The techniques used to 'steer' or 'aim' the AI stop working once the AI becomes better than humans (and human institutions like the FBI or Microsoft) at certain cognitive capabilities like making intricate plans that will succeed in spite of determined human opposition. Researchers employed full-time at the Machine Intelligence Research Institute have been trying to solve this 'control' or 'alignment' problem continuously for 20 years, but progress has been very slow compared to 'progress' on just building dangerous AI.

LLMs have the potential to be one of the most toxic technologies in human history.

Sexist etc content is a footnote compared to the potential for an automated, proven, and tuneable technology of mass persuasion and impersonation.

So - massively addictive adtech. Reliable and effective political and ideological propaganda. Bot farms and "credible" fake media content produced at scale.

Imagine those being used against a relatively defenceless population to creative division, undermine personal and collective confidence, and encourage stochastic terrorism and other kinds of violence.

The safety argument is bullshit, at least in terms of the model itself doing nasty things.

The risk with AI is monopolization by humans using it for nefarious purposes. This risk is reduced through openness and transparency.

> While undermining its safety argument.

For someone not knowledgeable in this, why is this the case?

>For someone not knowledgeable in this, why is this the case?

So you open this to the world - what's the stop the Chinese government from weaponizing it to both suppress their own citizens and attack foreign adversaries?

they are already doing this. even if you don't give this to them they will just make it themselves. keeping it hidden doesn't prevent china from doing bad things. this doesn't keep this power out of the hands of china it just keeps it out of the hands of the common person.

Right, just like not giving them access to TSMC and advanced fab designs doesn't stop them. And not giving them access to advanced jet engines didn't slow them down.

Except... it does and the net effect was seen just this week as Xi started softening his tone realizing that his economy cannot keep up without access to advanced technology.

if you think that china can't get access to gpus because the united states says so then you are incredibly naive. of course they can get access to gpus. they could buy them on the black market or buy them through another country or just steal them. they are literally made in taiwan right next to and right under the thumb of china.

What's stopping them from doing that right now with the GPT-4 API?

The US has already used similar software solutions for suppression of it's own citizens for at least a decade, so what can "AI" improve for China (see probation policy in many jurisdictions, often decided by a test fed to a black box that says whether or not you need to be monitored)?

In this part of the world, we're more worried about what the US government can do to us than what the Chinese can do.

In other words, for many people in the world that wouldn't be an issue, it's already in the hands of a country that had invaded or intervened far more countries than China in the past few decades

Someone said it, amen.

I would care what China would do if China was going around the world killing millions of people in the name of "national security".

Alas, so far that privilege went to Great Britain, France (Napoleon), Germany, US and recently Russia and Isreal.

Still no China on that radar.

https://en.wikipedia.org/wiki/Annexation_of_Tibet_by_the_Peo...

Indistinguishable from most doomsday cults in history, there are a bunch of people who fear that AGI is approaching and that it will pose an existential threat to humanity if the proper incantation isn't used during summoning.

That belief underlies the OpenAI's prevailing nonprofit mission and is the reason for its existence.

Prematurely loosing AI amongst the lay people is their nightmare scenario. Only the technocrat elect can perform the ritual safely and only then if they work together and avoid earthly temptation for power and money.

Reply on news.ycombinator.com