0xDEAFBEAD

This is the type of Ed Zitron prediction which keeps being wrong: https://danluu.com/zitron/

Anthropic's revenue is up 50% in the past two months. They're not hitting a wall.

>Any real work in this field is being done not by the people saying 'stop'.

Not exactly, that Evan Hubinger guy from Anthropic famously said his p(doom) is over 10%

See the signatories:

https://aistatement.com/work/statement-on-ai-extinction-risk

https://www.pacingthefrontier.com/

I'm amplifying their calls to reduce the rate of progress

So from a conservative's perspective, you're basically a dictator facilitating an invasion.

This is how you fuel the far-right.

"I'm gonna put my head in the sand and there is nothing you can do to stop me."

If it's different sometimes it makes sense to have a different term.

Why shouldn't we let Europe do the world policing for a while? They're always complaining about whatever we do.

The US is now the world's top oil exporter thanks to fracking:

https://ourworldindata.org/grapher/fossil-fuels?fuel=oil&met...

This type of content is mostly on reddit, which leans heavily leftist. They censor right-wing views aggressively.

Reddit is kind of a cult. "Ditch your loved ones so you can spend more time on reddit telling people to ditch their loved ones."

The question was: "What label do you prefer for the agent that did something it was not asked to do?"

OK, so considering Sweden, if Sweden is flooded with immigrants, where should the Swedes who just want to live with other Swedes go? Australia??

Also you do realize that your preferred migration policy will lead to microaggressions (and possibly macroaggressions) against immigrants? If people don't have democratic means to keep out disliked foreigners, they may be tempted to try alternatives. The ballot box is ultimately an alternative to violence.

You could make similar statements about user interface design. That doesn't prevent it from being a legitimate and useful field of study.

The US economy did great prior to 1900 or so. During this period, we did not have significant military action outside of the Americas, and we avoided peacetime alliances. From 1820-1900 the US economy exploded from about 40% the size of the UK economy to about 200% the size of the UK economy.

https://ourworldindata.org/grapher/gdp-maddison-project-data...

Switzerland has a per capita GDP exceeding the US while pursuing isolationist foreign policy. They are the world's most popular country in opinion polls.

I'm not sure people should be forced to live with those they feel xenophobia towards, even if one grants that the feelings of xenophobia have no basis in reality.

Could help put a stop to the AI nonsense for a while, at least.

>if you cant punish the creation for what it does it leaves only the one who started the process

Agreed, but I think we can do more on the prevention side as well. Traditional liability law is for negligence in case of preventable disasters. Since we currently have no way to prevent AI disasters in principle (alignment problem remains unsolved), I think we should just stop developing the technology for now: https://pauseai.info/

"AI has gotten cheaper more quickly than any other transformative technology in history. The cost of achieving a given level of AI performance has fallen about 47% per quarter since 2023, or 13× per year. That price drop is four times faster than DNA sequencing, six times faster than compute, 18 times faster than lithium batteries, and (in the century up to 1973) 54 times faster than electricity."

https://epoch.ai/publications/the-plunging-price-of-thought

Most unreliable computer programs won't launch research programs consisting of thousands of pages of text to find creative ways around obstacles.

This type of whack-a-mole approach is akin to "fixing a bug" by hardcoding a special code path for known-buggy inputs. It doesn't address the root problem of AI misalignment, and doesn't allow you to prevent catastrophes in advance, only patch things up after the fact.

This might be helpful reading: https://www.lesswrong.com/w/nearest-unblocked-strategy

As AI systems get smarter, we may reach a point where we have to get it right on the first try or face truly catastrophic consequences: https://www.youtube.com/watch?v=7wy3xyoXYt8

How specifically did misconfiguration lead to the HuggingFace attack? You could argue that its sandbox was misconfigured, sure. But suppose you had a similar incident where its intended task required access to the internet, and it veered off course in a similar manner. I don't think "misconfiguration" would be an accurate description of what went wrong in that hypothetical.

The doomers already have a term which fits pretty well: "AI misalignment".

>It implies random behavior, which isn't unintended like you suggest.

The HuggingFace attack was not "random" behavior. It was goal-directed but misaligned behavior.

This isn't necessarily a simple matter of the operator making sure they behave. AI alignment has been considered to be a difficult problem for over a decade -- and remains unsolved in general, as these recent incidents illustrate.

"Fuzzer" already has an existing meaning in CS anyway: https://en.wikipedia.org/wiki/Fuzzing

Well in that case, OpenAI has reduced incentive to solve the alignment problem, since it won't suffer liability from alignment failures.

I'm not exactly sure why you think this distinction is so important. I think my point stands if you replace "Person" with "OpenAI". In any case, I presume the swarms OpenAI has been researching will be available to the general public before too long.

They could slow down more if they knew others were also slowing. If others are also slowing, you can slow more yourself without loss of market share.

* * *

Many experts believe that:

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

https://aistatement.com/work/statement-on-ai-extinction-risk

If the antitrust waiver is too broad, we can always rework it. But there's no "undo" button for human extinction.

>There is no reason to pause any AI development. Quite honestly there is very little appetite to be left last.

"According to survey results released on Wednesday, 68% of [likely US voters] said they would support the proposal to temporarily pause advanced AI development and permanently prohibit the development of superintelligent programs, while just 25% said they’d oppose it."

https://www.commondreams.org/news/sanders-casar-ai-bill-poll

>Eg in a hypothetical armed convenience store theft: We don't prosecute the gun manufacturer, we prosecute the individual using the weapon for crime. Same deal here.

So if I ask ChatGPT to make me some paperclips, and it replaces all the matter in a nearby city with paperclips, who gets prosecuted: me, or OpenAI?

"Bot" doesn't carry any implication of unintended behavior. You could call it a "buggy" bot, but these aren't ordinary software bugs.

There's no simple bugfix which will address AI misalignment. It's essentially been an open research problem for upwards of a decade.

That approach means they can only talk about publicly known AI techniques. You're saying if they want the standards to apply to proprietary stuff, they might just have to make it public. They still have investors they need to please right?

Yes, that would be the common sense response from my perspective: https://pauseai.info/

>line go up

It's already going up at staggering rate. Anthropic is now at $100B in annualized revenue, up 50% in the past two months.

* * *

Here's a little allegory for how I'm thinking about this discourse.

Imagine a man who is raising tiger cubs in his backyard. They're growing fast. He keeps them on dog leashes so they stay under control. One day, a growing cub breaks its leash and goes on a rampage through the neighborhood, eating a beloved local pet.

The neighborhood erupts into a big argument: Was the leash inappropriately thin? The neighbors point out that thicker leashes are easily available at the local pet store. Furthermore, is it appropriate to refer to the loose cub as a "wild animal" in local news reporting, or is it factually more accurate to call it "domesticated"?

Meanwhile, the cubs grow larger and lick their lips, oblivious to the discussion.

Dario requested "a narrow waiver for certain kinds of safety conversations."

https://darioamodei.com/post/we-must-pace-the-frontier

I'm not seeing how that solves his 3rd-party competition problem.