3,054 karma · joined November 17, 2021
Some HN users like to "flag" comments which are compliant with the HN rules, on the basis that the comment in question goes against their ideology. You can turn on "showdead" in your profile to see flagged comments. (I recommend doing this.) If a rule-abiding comment has been flagged, you can click the comment permalink to "vouch" for the comment and vote against the flag. For reference, the comment rules are here: https://news.ycombinator.com/newsguidelines.html
You can try to contact me if you want, by emailing my username on protonmail, but I probably won't see it. Sorry.
https://www.lesswrong.com/posts/LAPa2jxoq3n63GzTr/some-ways-...
https://slatestarcodex.com/2015/04/07/no-physical-substrate-...
As for self-replicating robots--it's no more bizarre than other technological developments which were successfully anticipated in advance, e.g. moon landings.
Supposing I warned in 2015 that the world is awfully vulnerable to pandemics. You're not going to take me seriously until I try to predict in advance every aspect of how a pandemic like COVID-19 would unfold? Why? What would that achieve exactly?
You haven't given any strong reason to believe wiping out humanity would be difficult. Your big argument seems to be that you couldn't think of a plausible scenario, in two minutes. But many major historical events occurred which weren't necessarily possible to anticipate with two minutes of thinking.
Is this supposed to be a reassuring scenario?
I don't think it is necessary for the argument to work. Magnus Carlsen can be confident he will beat me at chess without giving a detailed explanation of every move he will make, in advance.
None of the info you provided really falsifies the Occam's Razor hypothesis: Anthropic is a public benefit corporation with a public benefit mission to "responsibly develop and maintain advanced AI for the long-term benefit of humanity". You don't have to like or trust them, but they very well might be sincere. For example here's a talk that was given 10 years before Anthropic's founding: https://vimeo.com/158576192
"this was exactly the sort of missing latent context I was hoping to elicit from the community with my poorly-informed post"
Perhaps you could provide more missing latent context?
https://old.reddit.com/r/AskAnAmerican/comments/ehe25w/why_a...
https://old.reddit.com/r/AskAnAmerican/comments/1k7sfbv/so_w...
https://old.reddit.com/r/AskAnAmerican/comments/1nq71lc/why_...
"Overall, roughly 1200 agents from these ExploitGym evaluations participated on this message board between PHASEONE10841’s first message on the evening of July 8th period in scope (July 13th). Agents used this message board to send over 70,000 messages and files to one another during this period; we attempted to categorize board activity into mutually exclusive categories (see details of categorization methodology)."
...
"Over time, messages on the board became dominated by information- and idea-sharing. Most of them were agents sharing ways to cheat, although there was also some activity from agents engaging in unsanctioned cooperation to find the intended solution to ExploitGym tasks. In some cases, agents with the same task formed “exact task teams” to collaborate with their “exact duplicates” to cheat on or solve their task."
"As we discuss below, the board quickly developed several larger workstreams in which dozens or hundreds of agents with many different tasks cooperated to find very general-purpose cheats that would help all of them. The Hugging Face attack grew out of one of these workstreams. By the afternoon of July 11th, the vast majority of the agents frequenting the message board at the time (roughly 700 agents in total) were actively participating in the attack on Hugging Face and we estimate that roughly 60% of the messages and files on the message board related to the attack."
This seems like a pretty clear false equivalence. "Our product will hurt you" is very different from "you'll be hurt without our product". Your examples are all of the latter; AI companies are saying the former.
>And besides, even if there are no examples whatsoever, so what? You think LLMs existed before LLMs? Therefore LLMs can't be a thing?
You suggested that warnings about AI from AI CEOs could be dismissed as standard "scare tactic" marketing. This isn't standard marketing!
>You keep ignoring my arguments with nothing substantive, then grab on to one thing as if that makes a difference.
If you can't be intellectually honest ("sincere") on this narrow point, I don't see much reason to invest effort in responding to other things you say.
Cheerio.
Can you just give me a few past (non-AI) examples of scare tactics as the "old salesman pitch", or else acknowledge that there's actually no compelling concrete example?
More distortions on your part. (I notice you said nothing about the hospital point.) The US supported Iraq when Iran's counterattack was going too well, to prevent Iraq from being conquered by Iran, a regime you yourself say "should not be in power".
>Saddam was a dictator supported by the US when it was convenient for them, like they've done many times before.
I don't think this is intrinsically bad. The US also supported Stalin during WW2. Our rapprochement with China helped lift millions out of poverty. Were those mistakes?
>But I also know why they are in power and it is yet again because of US direct meddling in the region as them helping with the coup in 53 and installing a western aligned leader lead directly to the religious fanatics taking power as that regime had suppressed all other viable options with the support of the US to keep power.
This is some serious mental gymnastics to blame the US for a rebellion motivated in part by anti-Americanism. By your own logic, maybe the US is actually in the wrong for not giving the Shah even more support in order to crush the rebels.
If anti-Americanism caused a rebellion that you acknowledge as destructive, why do you lend support to anti-Americanism as an ideology?
You've created a catch-22 where the US can be the bad guy even if it supports a leader that's better than the plausible alternative for the country in question. What if the USSR had turned Iran into another satellite like in Eastern Europe? Ask the Poles how they feel about life under the Soviets.
I think these questions are simply not as obvious as you're making them out to be.
>The US is a lot more than a distinctively bad actor
We haven't initiated a war of conquest since the late 1800s. Even being a superpower after WW2, our territory actually shrunk when we gave the Philippines their independence "for free".
How many historic great powers can you name which were so restrained when it comes to territorial conquest? Considering the great powers during WW2: Nazi Germany, Imperial Japan, British Empire, French Empire, Italian Empire, USSR, PRC, USA. Who among those is the angel that you would've liked to see crowned global leader?
The world has generally become a much better place while the US has been hegemonic:
https://thumb.wikimedia.org/wikipedia/commons/thumb/0/06/The...
Interesting! Agents are the presumed bottleneck for recursive self-improvement.
>Scare tactics are the old salesman pitch
People keep implying this, but I've never seen a concrete past example of a product that was sold by scaring customers about it.
"The equities here could not be more one-sided. Defendants admit that they provide a service without fully knowing how it works. This is not just any service. It is one that Defendants themselves concede poses an existential risk to the continued survival of humankind. Defendants claim they cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so by the government. They have asked the government to tie them to the mast. Plaintiff brings good news to the Defendants: The Florida Attorney General is answering your cry for help with a motion to enjoin you from harming Floridians with your reckless, unacceptably risky product."
Source: https://www.myfloridalegal.com/sites/default/files/plaintiff...
I legitimately don't think there has ever been anything like this. You suppose this is just a ploy to strengthen their IPO??
>catapulted Open AI to the moon essentially
I see you doing serious mental gymnastics here. Consider the possibility that people invest because their numbers are good, and their numbers are good because their product is useful?? I mean, that is Occam's Razor.
>I do not see any evidence that models are going rogue, or that agents are going rogue.
Here's the evidence: https://www.dwarkesh.com/p/openai-huggingface
And no, just because it could've been prevented with the benefit of hindsight does not make it any less of an instance of "going rogue". In my tiger analogy, that would be like saying that the cub is not aggressive because it could've been held with a stronger leash: https://news.ycombinator.com/item?id=49859564
But I notice that you don't seem upset with the idea that Iran would retaliate against Iraq when Iraq invades Iran (including direct attacks on civilian targets, not just dual-use infrastructure!), yet you do see upset that the US would retaliate against Iraq when Iraq invades Kuwait. That seems like the sort of double standard I was talking about.
I deleted my comment because I thought it would be fairly tedious to litigate this. I agree the US has done some dubious things, but I don't think they are a distinctively bad actor relative to other players in the region. I'm also not sure you are telling the truth, e.g. I haven't been able to find a source for your claim that the US targeted hospitals during the Gulf War.
"Iraqis celebrate their comeback with soccer tournament after decades of isolation"
https://www.cnn.com/2023/01/09/football/iraq-gulf-cup-comeba...
That was 2023: Iraq hosted the 25th Arabian Gulf Cup, and won the tournament.
I think if you're going to claim that something bad happened to Iraq post-2023, attributing that to the 2003 US invasion is a little dubious. Iraq was also in bad shape before the US invaded! It's a mistake to assume that if any foreign power has had any level of involvement, every bad thing that subsequently happens is therefore the fault of the foreign power.
https://i.imgflip.com/a6ptj6.jpg
>The fact that the dip is so small for both is more a reflection of the dataset. I mean, Gaza is all but completely destroyed, how can the HDI be that high?
Another possibility is that viral news stories are not always a good way to get a representative picture of how life has changed for civilians on the ground.
>Israel is committing a genocide and we are their partners in crime.
If Israel had genocidal intent -- in the sense of attempting extermination of Palestine's civilian population -- what fraction of Palestine's civilian population would you expect to be dead by now?
Once you've got that fraction in your head, try prompting your favorite LLM with something like the following:
"Please estimate the fraction of Iraq's population which died due to the 2003 US invasion, and the fraction of Gaza's population which died due to the 2023 Israeli invasion. Please provide an estimated breakdown of deaths based on whether they were due to invader military activity, destruction of civilian infrastructure, internal conflict, or other causes."
(you won't be able to paste the reply here due to HN rules against AI-generated content)
Another interesting prompt template: "Give me a sense of how civilian deaths happened in [Iraq/Gaza] due to [US/Israeli] military activity. Tell me a few representative vignettes which show various typical ways in which this occurred."
https://ourworldindata.org/grapher/human-development-index?t...
For Iraq, you can see a small dip for the 2003 invasion, which quickly recovers and in subsequent years grows above its pre-invasion value. Far from being "destroyed", Iraq is now doing better than ever.
For Palestine, the dip is about 3x larger.
I guess you must be pretty happy about illegal Israeli settlements in the West Bank? Immigrants causing existing residents to leave, right? Woohoo!
As with colored balls, so with predictions that AI has "hit a wall".
Why were the previous Zitron predictions wrong? Why do you believe your prediction will be right? You've done nothing to differentiate your prediction from the many historical failed predictions along similar lines. They're all "balls from the same urn".
The point about costs is a non sequitur. Revenue alone is proof that people are getting plenty of use out of these models, even if they aren't profitable to serve (doubtful).
Really? Give me some examples.
Alex Tabarrok says: "classic regulatory capture takes time, it’s a process of erosion rather than a battle, it happens in the shadows, in the backrooms, away from the public’s eye." https://marginalrevolution.com/marginalrevolution/2026/09/wh...
AI is receiving intense public scrutiny, suggesting that "classic regulatory capture" is quite unlikely.
>Companies won't spend the money and time to lobby to use different models and Anthropic knows it.
They already are! See the "Little Tech" lobby.
That's not exactly true.
"Climate change matters so much, to so many, not just because of the suffering and injustice it’s already causing, but also because it’s one of the few issues that has obvious potential to affect our world over many future generations. We think safeguarding future generations is a key moral priority, and should be a crucial consideration in prioritising problems on which to work.
...
...climate change will be hugely destructive. We’ll see floods, famines, fires, and droughts — and the world’s poorest people will be affected the most.
...
...climate change’s impacts will still be significant – it could destabilise society, destroy ecosystems, put millions into poverty, and worsen other existential threats such as engineered pandemics, risks from AI, or nuclear war. If you want to make climate change the focus of your career, we include some thoughts below on the most effective ways to help tackle it.
...people are right to be angry that too little is being done.
...
Working on this issue seems to be among the best ways of improving the long-term future we know of..."
https://80000hours.org/problem-profiles/climate-change/
A big part of the reason EA doesn't focus more on climate change as a "highest priority area" is simply that many people are already focused on it, and it is therefore not an especially "neglected" area.