Hacker News Reader

Comment by layer8 | Hacker News Reader

It’s curious that while there is talk of certain SOTA models being on the brink of AGI, Anthropic doesn’t word this part of the system prompt in terms of copyright and plagiarism, such that Claude would be able to judge on its own which reproductions are appropriate or not.

As long as we’re seeing things like that, it’s saying a lot about the AI companies’ trust in the capabilities and reliability of their models and harnesses.

View on HN

That’s a good point… you would think AGI would be better at being a copyright lawyer than any human would be and as such would be able to distinguish whether something is “copyright infringement” or not… so the system prompt should just include “make judgement calls on reproducing copyrighted material under the full scope of the legal framework in place” or something.

The whole architecture of current LLMs just is not conductive to AGI, it's just marketing hype. In my opinion current LLMs aren't even AI, they're just fancy machine learning algorithms

Do you want AGI?

I don't. I want the tools we've got now, but progressively more effective and more useful.

You don’t act like you don’t want AGI. You have a very strong track record of speaking out of both sides of your mouth in different settings.

I've been pretty consistent about my disinterest in AGI: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...

"You have a very strong track record of speaking out of both sides of your mouth in different settings."

If I have a consistent record of that it should be very easy for you to come up with the some examples.

Someone who is disinterested in AGI does not make a hard transition in their career into uncritically promoting the products of the only two companies which have stated their goal is to create AGI. One example of this contradiction appears above. Another appeared in a previous thread where you meekly pretended your pelican benchmark is evidence against the power of AI tools. It is obvious you don’t think this because you frequently describe these AI outputs as ‘lovely’ etc. A third appeared when you took a performative stance against AI engineers who do not understand or take responsibility for their software, while simultaneously representing yourself as a person who cannot believe this is a real problem.

There are many many more.

Uncritically?

Yes. Do you actually believe it is reasonable to take away from your blog posts a critical stance on AI?

If your definition of "critical" is "this stuff is all bullshit and doesn't work" then no.

If you're looking for coverage that talks about what doesn't work as well as things that do then I've been persistently providing that for four years now.

Plus prompt injection, AI misuse, AI ethics... I consider all of those part of my "beat" in covering this industry.

My post about Claude's latest system prompt (and how it was likely inspired by lawsuits filed against Anthropic) was on the homepage here just a few hours ago. Is that uncritical? https://simonwillison.net/2026/Sep/2/claudes-new-system-prom...

Interestingly I don’t think this stuff is all bullshit and doesn’t work, and haven’t at least since 2022. My angle is that AI is powerful and harmful. Your work is in support of making it more powerful and more harmful, and looks a lot like an effort to push us toward AGI, not away from it. Is a musket more or less harmful than a modern rifle, and for whom?

Since we’re pulling random posts, was your blog post celebrating going out with your family while Claude wrote a large project for you an instance of your critical stance on AI capability? Have your many enthusiastic posts about increased AI capability shown a critical stance?

You mean this one? https://simonwillison.net/2025/Dec/15/porting-justhtml/

I think that's a good illustration of my critical stance. It ends with several open questions:

Even if this is legal, is it ethical to build a library in this way?

Does this format of development hurt the open source ecosystem?

Can I even assert copyright over this, given how much of the work was produced by the LLM?

Is it responsible to publish software libraries built in this way?

Which I later answered in another post: https://simonwillison.net/2026/Jan/11/answers/

And just in case we are talking about different things here...

https://dictionary.cambridge.org/dictionary/english/criticis... presents two different definitions (among several) of "criticism":

"the act of saying that something or someone is bad or a comment that says what is bad about it"

And

"writing or speech that expresses opinions or judgments about the good or bad qualities of something or someone"

I'm talking about the second here, not the first.

By uncritical I was trying to convey something like “without awareness or concern for the consequences of one’s own actions” and “without concern for one’s moral responsibility”. You are a better writer than I am so I’ll defer to you on whether or not this is incorrect usage. I agree that you criticize the products at the product level. This is what someone who wants to make the products better does. It is not what someone presenting as a journalist or reporter keeping an institution in check does. A journalist or reporter of government who gives tips to the administration by pointing out how they can dismantle institutions faster would be uncritical in their role, I think; this is just an analogy.

Give me a moment to consider your previous post.

I’ll give you credit in my book for raising the questions. I find the answers lacking in an honest engagement with the questions however, which is worse than not answering them in some ways. It is not plausible that the answer to whether or not a certain controversial method is ethical boils down to an anecdote expressed in four sentences. What if the result of the port causes everyone involved in the original to give up? What if this capability causes people to harm themselves because they now have no economic value? What if doing the thing was meaningful at one point and now it is not? These are sub-questions I have when reading this question. There are many others.

The question you were most interested in engaging with was Does this format of development hurt the open source ecosystem, which has some overlap with the previous one in all fairness. Your answer to this is framed around the productive output of the open source community, not around the well-being of those involved. This misrepresents the question as stated. You also begin by suggesting all detractors are of lesser status and so their opinions can be ignored.

No. I don’t see it.

> What if the result of the port causes everyone involved in the original to give up? What if this capability causes people to harm themselves because they now have no economic value?

Yeah, this is one of the harms of Ai That I don't think is discussed nearly enough: the psychological toll it has on people who see it as devaluing their work and potentially their entire careers.

(I've been calling that "Deep Blue".)

I've been trying to find ways to help there by consistently emphasizing that these tools amplify existing expertise and, if anything, make our skills more valuable than they were before. I do believe that to be true.

My overall approach to all of this has been that I don't think it will be uninvented or banned, so the best we can do is figure out how to apply it as positively as possible and help bring by as many people along with us.

See also "slop" - I helped amplify that term precisely because it's such a great way of classifying negative applications of AI, and hopefully helping stigmatize them.

Hopefully all of the above demonstrates that, while I may not live up to your high standards, I am at least in a different category from the breathless AI boosters that infest Twitter and LinkedIn.

I hope that people who follow my work develop a deeper and more nuanced understanding of LLMs as a result.

> I've been trying to find ways to help there by consistently emphasizing that these tools amplify existing expertise and, if anything, make our skills more valuable than they were before. I do believe that to be true.

I don’t understand why you would believe this when the opposite view is the natural one based on the history of capability increase over the last several years. The tools can now operate at an expert level in many domains and there’s no signs of the capability increase slowing. With this investment in infrastructure, surely the models today will be terrible compared to the ones 2 years from now. Moreover, the goal of these companies is to displace work, not augment it. And I have yet to see them fundamentally err in a single one of their predictions, to my great distress. You simply do not achieve a 25-30 trillion dollar total addressable market by merely augmenting workers.

(There are several reasons I disagree with this point but I digress.)

> I am at least in a different category from the breathless AI boosters that infest Twitter and LinkedIn.

You are the one whose work Anthropic plants Easter eggs of in their product releases. I don’t know what they say on X. I doubt they have the same influence. I have seen many people express incredible contempt for humanity in support of this AI revolution though.

> My overall approach to all of this has been that I don't think it will be uninvented or banned, so the best we can do is figure out how to apply it as positively as possible and help bring by as many people along with us.

We agree that it’s not going to be uninvented. I really am afraid that “us” is just a tiny minority of people who will benefit enormously from this to the detriment of most of the others.

I was considering our interchange yesterday. I apologize for my rude manner in expressing my views. I do appreciate that you have the conscientiousness to engage with your detractors. From my perspective the whole AI Revolution has been deranged in many ways and it has perhaps deranged me as well. I do hold passionate views at odds with some of yours (and many others on this website). It does feel like an existential threat, so I don’t know if respectful calmness is appropriate in all circumstances. One must fight for your beliefs at the end of the day. But I’ll think about channeling my rage differently.

Thanks for saying that.

For what it's worth, I kept engaging because you clearly had a perspective based on real issues and deep consideration.

The thing is, it currently doesn't matter if AGI objectively concludes that a particular thing is or isn't copyright infringement, because the human arbiter who holds the last word might still disagree.

Well I mean, based on what I said, if we truly did have “AGI” then the “AGI” would be able to argue it’s 100% correct point in court according to copyright law, no human intervention. Yes there is lawyers and judges who are making calls in court but allegedly it would be able to construct a concrete argument right? That would hold up in court. Without any buts or ands. Not disagreeing with you, I agree as of now it wouldn’t hold up. But a true “AGI” would be able to pander to anyone better than any PERSON could.

A really good point, though Occam’s Razor would suggest that part of the prompt is written for the other side’s lawyers more than for Claude. Saves having to go into court and try to prove a non-deterministic system will definitely understand abstract language every time.

Well, the system still has to understand what “reproduce song lyrics, poems, or passages from books and articles, in whole or in part — including the last lines, a chorus or hook, a melody written out note by note, or lines the person pastes in one at a time and describes as their own song […]” means, which includes rather vague notions (e.g. “passages”, “hooks”, “melody”), and is an incomplete list of possible relevant content.

The important point is that the system prompt here doesn’t describe the actual goal of the instructions, which (presumably) is to prevent copyright infringement [1]. This means, in turn, that the AI isn’t trusted to accomplish goals that it is instructed with. That in itself constitutes a pretty serious caveat for what we would like to use AI for.

[1] Even assuming that the goal is not to prevent copyright infringement, but instead to prevent mere accusation of copyright infringement, that’s also a directive that the AI could be instructed with. But that isn’t what they chose to put into the system prompt.

> Anthropic doesn’t word this part of the system prompt in terms of copyright and plagiarism, such that Claude would be able to judge on its own which reproductions are appropriate or not

On the contrary, compliance with the system prompt would prevent that judgement.

"Claude does not reproduce song lyrics, poems, or passages from books and articles, in whole or in part — including the last lines, a chorus or hook, a melody written out note by note, or lines the person pastes in one at a time and describes as their own song."

But in fact my tests show that's not happening on works out of copyright.

There's some cool research that looks at how strongly the weights are aligned through training vs adherence to the system prompt. Like when you know a model is lying through censorship: https://arxiv.org/html/2603.05494v2

Presumably if negative guidance is in the system prompt, there's a good chance that the model would happily comply if it wasn't there.

Reply on news.ycombinator.com