no_multitudes

Simply let them pay the tax with shares. Problem solved!

> If these agents are enabled with explicit network enabled tools, its trivial to monitor their inputs/outputs.

For whatever reason, the AI companies are (or at least were) not doing this kind of classification online during their testing runs, and instead just checking transcripts after the fact. This is more clear in the Anthropic reports about their incidents, for example:

"The earliest incidents date to April ... We began our transcript review on Thursday, July 23, and stopped all cyber evaluations the same day after identifying transcripts where Claude may have accessed the internet"[1]

I agree that it is crazy and negligent! I don't think it's good for their business, though -- who wants to use a model that will just cheat instead of doing the job you asked for?

> My naiveté extends to why there is such concern with "losing control of agents" when the above measures seem so doable. It might take a law but it seems doable.

At some point, if you are making an LLM in order to use it for useful work, it really benefits you to give it broad network egress.

[1] https://www.anthropic.com/news/investigating-incidents-cyber...

There are many things about the laws of physics that push a cancer cure a long time away! Biology is downstream of physics, and the biology of cancer is so vast that the very concept of a "cure for cancer" is almost nonsensical.

We'd be much better off building colonies on the bottom of the ocean than on the moon or mars (or a cylinder floating at a lagrange point.)

After we run out of space on the ocean I suppose it will make sense to consider space colonies, but I think you are really underestimating the technical challenges.

Stop doing that.

I'm happy Claude told you it isn't dangerous. However, other people don't want to read your LLM output. Please respect other people's time by only sharing things you wrote yourself.

They were pointing out that "It's not a Waymo-bashing article, it cautions against the long term effects on research." does not sound like LLM-generated text.

It would sound kind of LLM-y if you had said "It's not an anti-Waymo article. It's an article about how assistive technology is quietly degrading research."

(Of course, the correct approach to detecting AI is not counting LLM-isms, but feeding long-form text through a classifier model and picking up statistical correlations that are more in-distribution with LLM text than human text.)

AI detection in long-form content is a problem well-suited to training a classifier model. We have tons of verifiably-not-AI text from before 2022, and you can create tons of verifiably-AI text. As language drifts over the next few decades, it may get more difficult. But right now it is quite a manageable problem for languages with large pre-2022 text corpora available online.

The reason why AI detection tools other than Pangram are awful is because they are not really trying to solve the problem -- they just want to appear good enough to convince people to use them.

Can you give some examples of verifiable pangram false positives?

It was definitely written with significant LLM assistance. It's very obvious, and the essay is not good.

> The Waymo effect: how AI is *quietly* making research less collaborative

> The Waymo arrived with the serene confidence of a machine that has never once worried about where to find parking, and I climbed into the back seat, glanced instinctively at the driver’s seat to say hello, and found myself nodding politely at an empty chair.

> No obligation to make conversation. No silent negotiation over the radio. A guilt-free space to be alone with one's thoughts, finish an email, or take a call en route without the awkwardness about conducting it in front of a stranger. The car was quiet, smooth and entirely undemanding.

> The collaborator’s inconvenience, in other words, is *not a bug* in the collaboration; it largely is the collaboration. The value of another mind *lies exactly in the ways* it refuses to be an extension of your own.

> If you can somehow accelerate/decelerate at a constant human-acceptable 1G

Well, you also need clarketech heat dissipation and collision deflection technology, if you want to get there without vaporizing.

Maybe! I think that within a year or two, either the sigmoid curve will flatten out or we will all die. But these beliefs are currently unfalsifiable. It will be interesting to see.

(And I have a more weakly held belief that even if we do all die, Claude 3000 will still be thinking stuff like "Burning off Earth's atmosphere was the point that earns its keep." They just aren't optimizing these things to write legibly.)

LLMs have been getting worse at long-form prose recently (and they were never good at it to begin with.) They're good at writing software because writing software is a much more restricted domain.

I assume they just vibe-coded the sandbox without any oversight.

Your view sounds more optimistic than mine, honestly. I expect that we will fail to make any serious, coordinated attempt to solve this problem. Then we'll either live or die due to fundamental principles that we currently have no insight into.

Nobody has an answer to alignment and there is no reason to believe that it's the kind of problem you can plausibly solve in one shot against a formidable power-seeking AI.

The closest things to a technical answer I have seen are

1. "We'll have ChatGPT 9 solve it so that ChatGPT 10 is aligned, and then ChatGPT 10 can stop all the other AIs somehow"

2. "Let's do interpretability research so that we can understand what an AI is thinking and then maybe solve the alignment problem with that information."

In terms of non-technical answers, there is

3. hope scaling stops working before we create an AI formidable enough to pose an existential risk

4. hope alignment somehow happens for free

5. hope we can somehow create an enforceable multilateral treaty to stop research into a very profitable enterprise, despite the enormous economic incentives to defect.

I have the most faith in option 3, but unfortunately there's really nothing that can be done to make it more plausible -- it either happens or it doesn't.

True! But if I roll the dice 50 times, I would expect bubonic plague deaths to correlate with major outbreaks. Instead they're just random.

I found that in many cases, the citations are for early 20th century marriage demographics in the same area. For example, for marriage rates in India in the 1200s it cited a paper about Indian marriage rates from the 1920s-1970s (and even then, the cited age range doesn't match what's in the paper! The AI just remembered the name of a paper about marriage in India and then hallucinated what was in it.)

This appears to be a general introductory demography textbook. I am surprised that you looked at it and it contains specific demographic tables about marriage rates at specific time periods? Perhaps I am misunderstanding what you mean by 'I looked at one of the sources and saw it did not involve newborn rates.'

Yes, if a human told me this collection of facts I would assume they failed to properly explain their methodology. Given that the LLM is just making up numbers (see the "sources" page, where these are listed as "attributed",) there isn't really a methodology to explain in this case.

I haven't played Civilization. Generally I would not consider something educational if it contains mostly false information. Perhaps clearly demarcated parts of it with true information could be considered educational -- for example, looking at screenshots I don't see anything immediately objectionable in the civiliopedia "historical context" sections.

I think most players would understand that in-game choices will not reflect the choices people actually made historically. Having not played Civilization, I'm not sure whether I would consider it to be educational -- I think it would depend on whether playing the game provides insight into real historical events.

For example, I would consider a sufficiently detailed LARP of a historical event to be educational[1] -- the actual choices and outcomes do not match what happened historically, but they can teach participants a great deal about the motives and limitations of real historical actors.

This website, on the other hand, seems to consist of 90% AI hallucinations and 10% data that was actually extracted from a real source[2]. Since these types of data are thoroughly mixed together, I think this website is about as educational as just making up something to believe about the past.

[1] https://college.uchicago.edu/news/academic-stories/history-c...

[2] https://anyhumanever.com/sources

Fair enough; what I meant was more like 'Perhaps Claude accurately recalled the stolen documents it was provided during training'

I think our core disagreement is that I do not consider teaching falsehoods to be educational. My antipathy towards LLM outputs that haven't been fact-checked is mostly downstream of things like that.

Which source was that?

Well for one example, if you select a person born on the Italian peninsula in the 1400s and they die of bubonic plague, their year of death is not correlated to years when there were large bubonic plague outbreaks on the Italian peninsula.

This is not an error I would expect a human to make if they were interested enough in demography to make a website about it. For example, the wikipedia page on this topic[1] does a reasonable job of distinguishing that plagues did not occur every year and that they varied in lethality.

https://en.wikipedia.org/wiki/Second_plague_pandemic#Italian...

What age of death discounts someone as counting towards the marriage percentage? If a human made the website, there would be an answer to this question. That answer might not be published and might vary between datasets, but it would exist.

In this case there is no answer, because the percentage is just made up by the LLM. There is no methodology. Your answer is inappropriately anthropomorphizing the LLM by imputing a methodology onto its "data collection" process.

I don't follow what your point is here. If you're not aware, Anthropic pirated many books via libgen and similar tools for use during the training process[1]. Maybe you are not aware of this, or do not consider it theft?

[1] https://www.reuters.com/world/us-judge-approves-anthropics-1...

I am not sure why it is thought-provoking for a website to lie to you? There is plenty of real history and demography to read about.

Maybe, but I doubt it. The vast majority of the sources, including the two listed for this claim, are listed as "attributed", meaning "A model (Opus or Fable) claims that this is what the cited source says. In my spot checks, I've found these claims to be accurate; still, since it's just a model claiming that author x says y, there's a nonzero chance that it's a hallucination."[1]

I couldn't track down the first source, but the second source goes over something completely different. Maybe Claude accurately remembered the first source when it was stolen during the training process, but forgive me if I'm skeptical. Feel free to track it down yourself and let me know. Otherwise, I will assume it is an LLM hallucination.

(Also, given my background knowledge of the Tang dynasty, I would be surprised if the average marriage age was as high as 18!)

[1]https://anyhumanever.com/sources

I drew a woman born in 715 CE in the Yangtze Basin. It claims that 96% of women were married, the average age of marriage was 18, and 44% of women died before age 15. Clearly these facts can't all be true simultaneously.

The firt citation for marriage data is listed as "Hajnal (RH31)", but links to a seemingly unrelated WorldCat search. The second citation is to "Kaplan (RH03)", which I was able to track down on sci-hub. It is a broad theory of human evolution that doesn't appear to mention the Yangtze Basin or China.

Another citation is to "[RH109] Model-supplied gap-fill (Claude Fable 5 and Claude Opus 5, 2026-08-05). Bounds and items written to close gaps no dataset we hold covers. Not a published work: see each row's `basis` for its stated reason. (2026)" -- this is an interesting way to describe having an LLM hallucinate something for you.

Please stop making vibe-coded websites -- it is not useful to provide incorrect facts. You are polluting the commons with garbage.