Hacker News Reader

Comment by josephg | Hacker News Reader

Why not both?

I agree government is useful and needed sometimes. But laws are slow, blunt instruments. Governments can’t micromanage every decision companies make. And if they tried, they would hobble the companies involved.

The government moves slowly. When AGI is invented (and I’m increasingly convinced it’ll happen in the next decade or two), what comes next will not be decided by a creaky federal government full of old people who don’t understand the technology. The immediate implications will be decided by Sam Altman and his team. I hope on behalf of us all that they’re up to the challenge.

View on HN

> And if they tried, they would hobble the companies involved.

Well, yeah, that's the point of regulation: to limit corporate behavior. There are plenty of other highly-regulated industries in the US; why shouldn't AI be one of them?

Define AI. Companies tend to find loopholes and the government is slow to regulate.

Broad enough laws don't have loopholes. GDPR for example had the wisdom not to define specific types of personal information (such as name, or IP address) but instead any data that could be correlated to identify a person. That puts the ball in the corporations' court to make sure they stay well clear of the dividing line.

Delegated councils (like the FDA, FTC, FCC, etc.) and courts can be used to make decisions about edge and individual cases, and do so much closer to real time than Congress can act.

Governments regularly successfully define things much more vague than "is this AI?"

The emergent behavior from this however is not that we get healthy competition, it's that the big guys have the money, connections, and understanding of the process to still do everything they want, while the little guys can't get off the ground and compete.

This is a necessary evil in the case of an industry like aviation where there's a massive and immediate risk to human life if you get it wrong.

If you see AGI as life threatening (in a physical sense) it's an understandable stance to take. However if AGI is threatening moreso in a social sense I don't think concentrating the power behind it is going to be a societal good.

I genuinely believe we as a society will get fucked by corporations for everything we can give if we don't either radically socially reform for incoming AGI, or have competitive open developments in the area not beholden to corporate interests.

For my money, the latter seems at least possible.

I don't know, power concentrates plenty in unregulated industries. Google has owned search for over two decades, and it's not like there's a ton of government regulation preventing anyone from starting a search engine. pPople have tried, they just haven't succeeded because Google was that much better and that much farther ahead, and they used their advantages to widen the gap. The same is true for advertising, it's not like the government was stopping anyone from competing with Google and Facebook, but other companies just couldn't get there.

I'm not sure that AI is going to look that different. Is there any reason to believe that if OpenAI wins the majority share, that any competition is going to be able to dethrone them?

We could imagine a world where the US government broke up big tech companies in the late 2000s or early 2010s, and it sure seems like that world would be in a much better position today, due exactly to increased competition. Instead we have a bunch of big companies that don't take risks or release ambitious new products because they're too busy trying to defend their moats.

>The immediate implications will be decided by Sam Altman and his team. I hope on behalf of us all that they’re up to the challenge.

Will they really be determined by OpenAI? So far what Altman has accomplished with OpenAI is to push a lot of existing research tech out into the open world (with improvements, of course.) This has in turn forced the hands of the original developers at Google and Meta to push their own research out into the open world and further step up their internal efforts. And that in turn creates fierce pressure on OpenAI to move even faster, and take even fewer precautions.

Metaphorically, there was a huge pile of rocks perched at the top of a hill. Altman's push got the rocks rolling, but there's really no guarantee that anyone will have much say in where they end up.

What if I told you, there is this startup that is working on really effective ways of proliferating plutonium. And that it is under pressure to keep the hype, so there isn’t much time to organize the storage of proliferated materials. And the thinking is that even if it explodes, since startup is not building a bomb, it can’t really make that much damage.

Oh, and also imagine that nobody really knows what plutonium is, there is just one startup that had figured out how to turn uranium to plutonium.

Also that understanding of uranium for now is at the level when it was just discovered. So people are still handling it by bare hands. And trying to sell shiny uranium toys.

> existing research tech out into the open world (with improvements, of course.

That’s quite the understatement, isn’t it? Neither Google nor Meta have been able to demonstrate something to the public that resonates even close the the altitude that OpenAI has.

I'm not sure if this is research quality or product tuning. I tried using Bing's chatbot and I found the experience to be vastly worse than OpenAI's version, yet Bing is ostensibly running on exactly the same platform (GPT-4). I think OpenAI turned the dial to "loquacious and friendly" and we humans are interpreting some pretty-similar underlying tech as having vastly different underlying quality, because we're impressed by that sort of thing.

The user interface to a tool matters.

Once you're down to user interface, you're no longer in "insurmountable technical advantage" territory.

But more critically, chat.openai.com is more of a tech demo than a serious product. So one can optimize the UX for "chatty and engaging" without being too concerned for accuracy. Bing, on the other hand, is intended to be a search product. It's noteworthy that adjusting the same model to provide a more accurate search experience seems to have snuffed out a lot of the magic.

They'll be decided by AI, not by governments, corporations, or individuals.

There's still a kind of wilful blindness about AI really means. Essentially it's a machine that can mimic human behaviours more convincingly than humans can.

This seems like a paradox, but it really isn't. It's the inevitable end point of automatons like Eliza, chess, go, and LLMs.

Once you have a machine that can automate and mimic social, political, cultural, and personal interactions, that's it - that's the political singularity.

And that's true even the machine isn't completely reliable and bug free.

Because neither are humans. In fact humans seem predisposed to follow flawed sociopathic charismatic leaders, as long as they trigger the right kinds of emotional reactions.

Automate that, and you have a serious problem.

And of course you don't need sentience or intent for this. Emergent programmed automated behaviour will do the job just fine.

> Once you have a machine that can automate and mimic social, political, cultural, and personal interactions, that's it - that's the political singularity.

The machine can mimic, but it is still completely reactive in its nature. ChatGPT doesn't do anything of its own accord, it merely responds. It has no opinion, no agenda, and no real knowledge or understanding. All it can do is attempt to respond like a human would, but it cannot reason on its own. Ask it about its views on a political issue, and it won't think about its stance and the reasons for taking it, but it will just produce what it has trained an answer to that question ought to look like.

The panic about the machine takeover is completely overblown, driven by people who don't really understand that these machines are and how they work. We are still far, far away from points where AI would be capable of actually making political decisions.

> Essentially it's a machine that can mimic human behaviours more convincingly than humans can.

Isn't this impossible by definition? Nothing can be more convincingly human than humans, or else it would be something else.

Perhaps you mean mimic charismatic or intelligent humans more than most, or the average, human?

> Once you have a machine that can automate and mimic social, political, cultural, and personal interactions, that's it - that's the political singularity.

Do you mean online only? Because I imagine we're still quite a ways from physical machines convincingly mimicking humans IRL.

> And that's true even the machine isn't completely reliable and bug free.

If the machine makes inhuman mistakes the humans will likely notice and adapt.

> Isn't this impossible by definition? Nothing can be more convincingly human than humans, or else it would be something else.

Why isn’t it theoretically possible for an AI to pass the Turing test so hard that more than 50% of the time humans think the AI is the real human? That would effectively be more convincingly human (to humans) than humans are.

> When AGI is invented (and I’m increasingly convinced it’ll happen in the next decade or two)

Aside from the issue of AGI being extremely difficult to capture in a proper definition, what leads you to believe this? Recent developments like LLMs and stable diffusion are impressive technology but I don't view them as remotely relevant to achieving AGI.

> Why not both?

This is not a good idea. There is a conflict of interest here. The fundamental goal of government is regulation, the fundamental goal of a business is profit.

Both the morals of government and corporations are therefore very different. I call them Guardian and Commercial Syndromes respectively. When you combine these two syndromes you create monstrous hybrids.

Allow me to elucidate:

An entity in the Guardian syndrome values honor and uses forceful tactics to maintain order and regulation.

An entity in the Commercial syndrome values money and tries to use whatever means possible to gain profits through trade agreements.

You don't want a commercial entity to have the power to enforce things and you don't want a government entity to desire profit. Both hybrids lead to bad things and it's the root of corruption in the world today.

You know how the old medieval English Nave formed? So many pirates lurked off the coasts of medieval England that London merchants went to the expense of financing a fleet of fighting ships and gave it to the Crown.

Why give it to the Crown? Because you don't want some commercial entity running the Navy using methodologies that circumvent trade agreements and violently funnel trade and profit to a singular commercial entity. You need an entity outside of the commercial sphere; An entity in the Guardian Syndrome. If a commercial entity controlled the Navy you get something similar to the Mafia or Yakuza; Basically organized crime.

Only the government should enforce the laws of AI because there is in theory no conflict of interest. But practically speaking the US government is pretty heavily hybridized already (See military industrial complex). Still it's the best option.

Reply on news.ycombinator.com