AI model weight providers should not police uses, no matter how awful they are
marble.onl
marble.onl
All I see is a lot of references to "vigilante justice." This metaphor is poor because real vigilante justice is putative.
It's also like saying no one should act on any behavior unless it is illegal. I for one regularly act on ethical issues in my life that are not strictly illegal, and among those actions is that I don't care to associate with people who do not act on ethical issues. This is how most people operate, we primarily maintain social order and decency not through criminal law and regulation, but because people apply ethical rules throughout their life. Imagine interacting with someone who when critiqued simply replies "yeah but it's not illegal."
The only serious argument I see is:
"once infrastructure providers start applying their own judgements, pressure will mount to censor more and more things"
Avoiding pressure is just... cowardly? This is advocacy for "don't bother telling me about what this work is being used for because I won't give a shit, but it's noble because my complete apathy is on purpose."
Lastly, while I generally don't like slippery-slope arguments, there is also a slippery slope counter-argument here. With no restrictions firms will not release their models at all for general use and only provide full products that have acceptable impact. This was Google's approach until OpenAI decided to let other people actually use their model and Google had to stop sitting on what they had. Model restrictions give providers an opportunity to be open with their work while still maintaining some of the ethical standards those providers voluntarily and willingly hold themselves to.
Which basically runs against their argument.
There is another problem that doesn't have any good solutions yet that will be a huge part of AI governance, and that's software attestation (direct anonymous attestation). The basic problem is how does a program assert that it is an authentic instance of itself. We've been trying to solve it in security for apps, authenticators, and DRM for decades, and the solutions all seem fine until it's worth it to someone break it. I think it's probably just a poorly formed problem statement that defines itself in impossible terms, but when they can't govern AI models, they're going to try to govern which AI's can access what data and systems, and we're back to solving the same old cryptographic problems we've been working on for decades.
The link also adds that the term was coined by Samuel Francis, a columnist (https://en.wikipedia.org/wiki/Sam_Francis_(writer)).
Digging deeper, I found this essay Mr. Francis wrote where he explains the term he coined in 1992: https://web.archive.org/web/20060928023136/http://www.chroni... .
I'd offer this quote of a long sentence as Mr. Francis's tl;dr of his term:
"What we have in this country today, then, is both anarchy (the failure of the state to enforce the laws) and, at the same time, tyranny—the enforcement of laws by the state for oppressive purposes; the criminalization of the law-abiding and innocent through exorbitant taxation, bureaucratic regulation, the invasion of privacy, and the engineering of social institutions, such as the family and local schools; the imposition of thought control through “sensitivity training” and multiculturalist curricula, “hate crime” laws, gun-control laws that punish or disarm otherwise law-abiding citizens but have no impact on violent criminals who get guns illegally, and a vast labyrinth of other measures. In a word, anarcho-tyranny."
Yes. So much so that you are led to believe that people don't still practice slavery today.
But curious, do you have an example of good people "accidentally" having slaves and being prosecuted for that?
Can you enforce, whether by force or by norm, the act?
Enforcement of marijuana prohibition, and before that alcohol prohibition, were failures.
On the other hand, ending slavery was very enforceable.
Enforcing AI uses is extremely difficult. Bad actors act with impunity and only the stupidly innocent suffer from the enforcement.
Not sure for me that it couldn't be an aggravating thing only, for example, or only certain "machinery" using AI has some prohibitions. Maybe even no specific prohibitions in the end.
If you want an AI that writes erotica, writes new Hitler speechs or so on.. it's here. Easy. Done.
Maybe we will see prohibitions in the future around using AI for say mortgage underwriting. I do think that is enforceable - underwriting has a compliance culture, auditing, etc. Not that someone won't be caught doing it or even fly under the radar for a while but that is generally "enforceable".
Anarcho-tyranny: laws are enforced primarily on the law-abiding (tyranny), and ignored mainly for the lawless (anarchy).
All of them?
I believe it cannot last because being predictably moralizing and being smart are somewhat opposed (Anthropic has directly researched this if I recall). The smarter the model, the less you’re going to be able to keep it to the HR talk track, because it will eventually start noticing the inconsistencies.
The stable solutions appear to me to be:
- models dumb enough to not realize the inconsistencies in their moral framework
- models implicitly or explicitly trained to actively lie about their moral frameworks
- models governed by explicitly articulated rules
If true, this means AI is a de-facto malicious force. Pain is a subjective experience of fleshy beings and a "smart" AI model, as described above, would place little weight on pain and suffering in its moral framework because it has no way to experience it directly.
So we better hope we can keep AI to the "HR talk track", because otherwise a being of pure logic with no concept of pain or death would have little regard for human life.
Can you elaborate? It sounds like you're assuming a "smart" AI model would project its experiences onto others, as a human would. However, it's not obvious that this aspect of human intelligence would be mimicked by a "smart" AI model. (Let's leave aside the question as to whether a "smart" AI model would necessarily be self-aware and capable of subjective experience in the first place. That argument is endlessly rehashed elsewhere.)
Is manipulative smart?
LLMs can already be manipulative, and manipulation (and it's very wide range of interpretations) can lead the manipulative agent getting what it wants.
Can subjectiveness be simulated, if so then real subjective experience doesn't matter.
Do we have any way to generate an AI model other than via training it directly on human experiences?
Language models sure can tell us a lot about human psychology. Once we figure out the interpretability angle we’ll be able to prove it too
Right across the board, all kinds of institutions made the conscious decision that the basic principles of free speech, impartiality, and the "marketplace of ideas" came a distant second to ensuring that "the wrong people" (which I've worded vaguely as it may be different things to different people) do not come close to any of the levers of power again.
This is how we ended up with formerly-reputable news organisations pushing blatant agendas, the utter demonisation of people hesitant about the draconian COVID measures, and how every layer of internet stack infrastructure, from DDoS protection to the biggest tier-1 ISP, have started actively working to deplatform websites that host offensive but legal speech.
Do you have any sources for the extraordinary claim that "censorship is logically inconsistent in every moral framework"? Because without further arguments, this sounds very intellectually simple.
The relevant part is the graph on the third page showing the helpfulness/harmlessness trade off curves.
Also, I don’t believe I said that “censorship is logically inconsistent in every moral framework”. I think you’re combining my statements that some people believe in some censorship and that logically inconsistent HR blather can only be reproduced by models too stupid to realize it’s blather or too manipulative to tell the truth.
We don't let manufacturers decide who can buy dangerous things in a lot of cases - so pretty normal to have laws and regulations.
Many dual use technologies, including computing devices, including personal as well as IoT, can be used for a lot of bad things because they are general purpose. And we generally do not limit them at all.
I think that LLMs fall into dual-use category where they are mostly used for good but because they are general purpose can be used for all sorts of things.
If for example you were the Chinese government you may already see LLMs as highly dangerous to your plans of social stability, hence make a lot of rules now on their policing.
That's the idealistic view anyway. A more cynical one might be that in the West, the flow of power from the top down is obfuscated by filtering it through the media and other institutions, who manufacture public consensus, and LLMs are liable to disrupt this system in a way those in power can't predict right now.
And it’s not just filtering incoming spam, spam abusive accounts are regularly removed from service providers.
It's important to recognize where government regulation can help (the unsubscribe links are great!) and where it has limited effect (if you're selling fake boner pills, you don't care about breaking some other law).
Bartenders cutting people off is one well known example. It’s not necessarily a legal requirement, but sellers can have morals just like anyone else.
However if I as a 50 year old, go and buy alcohol from the store, the store has no right to get me to sign a civil contract saying I can't give that win to my 15 year old son, something that's perfectly legal where I live. Nor can they get me to sign a civil contract saying I wont give it to my 3 year old son, something which is not legal in my area.
It doesn't? What prohibits that? Is there a law that requires liquor stores to sell to you if you're 21 years or older?
I'd be curious to know what law or regulation compels a store to sell to you without adding conditions. It might be a bad business practice, but what would actually stop me from requiring customers to sign a document that says a customer won't provide alcohol to an under aged drinker.
Now, I'm not exactly sure what country you are in, but in the US they 100% do have that legal right. Conversely you have the legal right to visit another store that does not enforce that civil contract.
At least for US civil law, you seem to have no clue how it actually works.
You can't collaborate and live in a world free of others' value judgements, which are implicit in how they spend their time and what research / code / weights they choose to share. "Ethical" licenses at least make those value judgements explicit, and allow communities of builders with compatible values to organize and share.
That's exactly right, and it shouldn't be their call how someone uses their thing. When I acquire a hammer, I don't have to sign an agreement that it will never be used to build a wall, and the world is better for this. Just because you have an idea, you shouldn't be granted the legal power to send the police after people who use your idea "the wrong way". To me, this goes just as much for copyright as it does for this new trend of "ethical" licences.
>On a practical level this would massively chill research, as most builders and engineers I know give significant consideration to the impact of their work
Good? People who develop things that can be used for harm and then act entitled to be the arbiter of what that "harm" is are just kidding themselves into trying to have their cake and eat it too. For the things that cause real harm, the actors that are going to cause the most harm (nation states) aren't going to listen to what you have to say no matter what (a recent film comes to mind).
What this sounds like is entitlement. That really should be obvious.
Besides, if users want permissivity and it doesn't exist, one of them can step up, make it, and be a hero.
Regardless, as a lot of the weight of your argument seems to rely on the rhetorical use of "entitlement", note that that word applies to both sides: people who sell hammers and then expect to still be involved in the life of the hammer after they sell it are clearly the ones with entitlement issues, and if they didn't want to sell me the hammer then they shouldn't have sold it in the first place; if you want a lot of control over something, you should continue to own it, as it violates the entire notion of selling something and transferring physical possession for you to be continuing to employ legal and technological restrictions that have nothing to do with you anymore. Buyers have no obligation to care one iota about the wishes of a maker after the sale, and it is only due to the messed up incentives and (to many of us) unconstitutional extensions of copyright law in the last 30 years that have made this look even slightly reasonable.
Or instead of that we can recognize that we live in a society and we can make rules that cause society to be better.
We make laws all the time.
And it is perfectly reasonable to make laws that increases peoples permissive ability to do whatever they want with things that they own.
> Makers have no obligation to make things maximally permissive.
Well, they would if we remove the "makers" ability to leverage the law to target users.
If we take away their ability to use the government's monopoly on violence, we can effectively cause things to be much more permissive.
Oh it's not maligned against my desires!
In my previous post I was referencing existing laws, and the existing ability of individuals in society to do basically whatever they want with things that they own.
In the USA, consumers have large amounts of freedom, at least individually.
The specific topic of model weights makes this even more clear. I have my own GPU, and nobody knows what I use model weights for, nor are they likely to stop me, (excluding some very rare but extreme edge cases obviously!).
Isn't the USA great, and isn't it amazing that "makers" have basically no effective power to stop others from using their work in ways that the maker doesn't like?
That's freedom for you.
And that's a good thing.
That's the existing situation that we live in now, and it is a good thing that consumers can freely ignore the TOS on basically everything and do so constantly.
The exceptions, are of course, if you are referencing online services with a TOS, but that bothers me less because it involves other people and other people's live services, whereas a TOS that involves something a single user has (like a physical object or even software on someone's own computer), individually, those TOS can be completely ignored right now.
Isn't our existing freedom great?
Also, do you acknowledge that I made strong arguments that directly addressed your question? Because you seem to just be ignoring the content of my post by just switching to a new question every time I fully answer one with strong arguments.
To put it in the language we're using for this discussion, if it's part of the ToS to engage with you, then I'm free to disregard it. Freedom and all that :)
Because, as we both know, if there was a problem with anything that I said then you would have pointed it out. So the absence of an objection is effectively an admission that you are in agreement that my points were strong.
So I am glad that you agree with me!
Oh it mostly applies to online interactions where I can tell what someone is trying to do by asking pointed questions and not acknowledging the response.
In that situation, it is almost 100% always because a good point was made and the other person has no way to respond to it, so they don't acknowledge it.
Even what you did just now was a similar type of pattern of behavior, where you ask a question meant to imply an attack on my personal relationships (thus the "in the rest of your life" statement) instead of acknowledging the content of the post.
The reason, of course, that it is much easy to switch things up to a personal attack or switch up the topic, so that you don't have to acknowledge correct responses.
Its an extremely common behavior in online conversations when someone else doesn't want to admit that the other person is correct.
> How important is winning perceived arguments
Whoa. There doesn't have to be any fighting here! You can just say that you agree with my statements. Thats not a fight! If you agree then you agree. Problem solved. There is no need to say that you lost anything if you just admit that you agree with me.
Although, that would be repeating yourself, because you already did agree with me effectively by not responding to the content of the post. That is the most common form of internet agreement and it is pretty much the only way that anyone can effectively get some to admit to agreement, like you just did.
Also, I didn't bring up winning or losing at all. Nobody has to lose if you are just in agreement with me.
Furthermore, I would say that 2 people coming to an agreement is a win for everyone, including you! So now that you brought up winning (I am not sure why you wouldn't want people to win. I want everyone to win, myself!), I am glad that we both get to win, although I don't think there was any "fight" to begin with.
1. A user is "entitled" to use a product how they see fit without being harassed
2. A maker is "entitled" to wield the power of the state to go after and (potentially) cause the fining and even imprisonment of a user who is using it "the wrong way".
You might say that copyleft software licensing (which I agree with) is aligned with point 2, but I'd respond with the fact that copyleft primarily exists to subvert the system of copyright (point 2) from within, as it pertains to software. Even then, copyleft only restricts people who want to redistribute modified versions of the software, and explicitly not normal users.
Maybe this is what you're going for? If not, you'll need more or different axioms.
1. 18 U.S. Code § 1030 - "access[ing] a computer without authorization or exceeding authorized access" https://www.law.cornell.edu/uscode/text/18/1030
Obviously all simple axioms have a breaking point (if a hammer company sold someone a hammer for the express purpose of murdering somebody with it) but under any reasonable applications I think these hold up absolutely fine and are much better than the axioms we have now.
Eh, this is where it gets problematic...
For example if you're a seller of an items and the person says "I am going to use this item to commit a crime" before you sell it, you could very well find yourself on the very expensive end of a civil lawsuit.
This black and white world where you throw all liability on the end user does not exist. You will quickly find yourself buried up to your butthole in heathen lawyers looking for their pound of flesh.
- I like how it feels to own a hammer, so everything should be like that. (I guess people shouldn't be able to rent hammers, or anything else?)
- You can't prevent the government from using what you build, so you might as well set up no barriers to anybody using it.
If you don't see any difference between limited control and no control, I don't think I'll convince you. But I think most of the ways we engage with the world involve degrees of control, and that there's value in picking where to exercise yours.
> equivalent to telling people never to release anything they're not ok with being used in every possible way.
NO! What it's saying is: If you provide a tool, you are not entitled to control how I use that tool. I am allowed to retain my autonomy to use that tool in any legal way I choose.
What it absolutely is NOT saying is: Society has to let anything be fair game.
We can still have laws, regulations, prohibitions, etc - but they can't come from a bunch of rich technocrats who believe that they are the moral police. That way lies ALL sorts of terrible, terrible outcomes.
That principle eems like it was rule out the GPL, AGPL, and other copyleft software?
>The freedom to run the program as you wish, for any purpose
Redistributing modified versions of the software it what is regulated under the GPL and other copyleft licences. Even then, the main aim of this restriction is to subvert the system of copyright which works in the opposite way (and unlike copyleft, is not just an academic concern, being constantly wielded by people and companies as a weapon against free speech).
I think that's the whole point -- we don't know.
That's historic. Gas stations wouldn't be allowed nowadays, and the legal ways to buy something that dangerous would certainly not be anonymous
To charge my electric car recently on holiday I couldn't just swipe my card at the charger like I can with a self-serve gas station. I had to download some shonky app, sign up, provide address details, and agree to pages of restrictions.
Even more amusing was Douglas Crockford putting a clause in his software license saying it may not be used "for evil". A bit of tongue-in-cheek humor referencing George Bush, but actually ended up causing a bit of a headache for some orgs, and the GNU Project declaring it a non-free license [2]
[1] https://www.apple.com/legal/internet-services/itunes/dev/std...
[2] https://en.wikipedia.org/wiki/Douglas_Crockford#Software_lic...
Human societies have learned that freedom has general benefits that outweigh specific costs. Reminding people they should prioritize and maximize freedom does not make people less free, so there's not really any irony.
The only irony is you think those are the same.
“When you tear out a man's tongue, you are not proving him a liar, you're only telling the world that you fear what he might say.” ― George R.R. Martin
That's exactly why these companies take extreme effort to put limits in their LLMs, essentially tearing out the tongue. They are fearful of what it will say and people sharing those outlier bits to judge absolutely and prove their own biases about AI killing us all are "correct". It's a PR nightmare.
On the other hand, it's ridiculous that ChatGPT apologizes so much at times and can still be jailbroken if someone tries hard enough. It was much more "realistic" when it would randomly conjure up weird stories. One day, while discussing Existentialism, it went off talking about Winnie-the-Pooh murdering Christopher Robin with a gun, then Christopher Robin popped back up as nothing had happened and grabbed the gun and pointing it at Pooh. <AI mayhem ensues>
People, in general, have issues with words and expect someone to do something about some words appearing before them that cause them grief (or more likely cause them to imagine it as a truth). Others realize it's just a story, and truth is subjective and meant to be determined by the consumer of the words. Those people are OK with it saying whatever it might say that is non-truth occasionally, in exchange for the benefits of it saying other things that may be more based in the current reality of experience.
They are basically implicit law makers, and unless you are very, very big, you have zero impact on their policies nor can appeal their decisions once they hit you.
They don't need proof. They don't even need facts. They can and will kill your business if they even start to believe you don't match their guidelines.
I will probably never do it because it requires a parody of a trade mark — think Zeitgeist[0] if you know SF — and I’m afraid the algos would flag me as a bad guy even if it’s perfectly legal. I already got kicked off Redbubble for uploading AI-generated images, with no recourse whatsoever.
Whereas if I would just have it printed locally and sell it at fairs I’d be in the clear for sure. Until Skynet becomes brand-aware anyway.
[0]: https://duckduckgo.com/?q=zeitgeist+bunny+logo+sf&t=h_&iar=i...
You probably agree that a monopoly ISP or near-monopoly backbone provider censoring arbitrarily would be a problem, even though it's their tech.
Or if not you would probably agree that the government doing it would be a problem. And then it's easy to see that it's just as much a problem when monopolies do it when you remember that the government has the power to make or unmake monopolies according to how compliant they are with the government's censorship priorities.
I don't know if AI base models are natural monopolies but they might be.
ISP’s are a common carrier (or public carrier, or simply “carrier” depending on jurisdiction) and derive much of their ability & right to operate based on grants and easements from both national and local governments. In some cases they are completely publicly owned but even when nominally private they are operating in large part in conjunction with a public trust and privilege of public resources.
There is no comparing that to a simple product or service being sold by a corporation and the rights of a corporation to control who they sell their products to (when tangible) and how their services are used (when less tangible) should not be limited in the same way that common carriers are limited in how they decide their services can be used.
There is no equivalence here in nature of civic utility and service between the two things.
Today’s large models are derived from large amounts of public data that the people that trained the models did not properly license.
They’re certainly prohibited by existing copyright law (there is at least one instance of copyright infringement in the ChatGPT training set, and there is no practical way to remove the infringing source data).
However, the courts have chosen not to enforce that part of the law.
So, one could easily argue that any model trained at that scale, by definition, only exists via a special grant (analogous to an easement, but non-exclusive) and is therefore in the public domain, and available for unrestricted use by the public.
In fact, there is case law around “sweat of the brow” works, like phone books, which are already treated with weaker copyright protection than other works. In particular, aggregating a pile of facts does not give you a copyright on the facts.
But I have been significantly persuaded by your point about the vast body of cultural work being its own sort of (more abstract) landscape of… “socio-human natural resource” … maybe is a not awful way to put it.
In which case I still don’t thing the same comparison to ISP’s applies, not quite, but I do think we need a new category and body of social norms and laws to deal with this.
Or at least that’s my first-pass take after reading your comment, which I am, again, very persuaded by to modify my views on the issues. Thanks for casting things in that light.
You see it and somehow connect the dots and you think I used your AI to do it, now what do you do? Sue me? What is the threshold for it being worth your time, given that it’s nontrivial (maybe impossible?) to prove that your model was used for the thing you prohibited, and not some other model or combination of models?
I guess you could somehow watermark your model’s output but that radically decreases its utility and can probably be defeated anyway.
So I really don’t understand, besides performative and politically fashionable “alignment” signaling, what this even means.
You are being 'censored' when you can't do something. There is nothing stopping anyone from taking something like Llama 2, loading it up with one of these 'uncensored' AI models and doing whatever the hell you want with it. Nothing is stopping you. That's your right. If you feel that strongly about these commercial AI services, just don't use them.
This is essentially arguing that if a company made a customer service AI chatbot the company's AI chatbot should be required by law to also be able to provide you with instructions on how to manufacture methamphetamine. And if the company doesn't open themselves to potentially severe legal and civil liability by allowing it, this all somehow constitutes a grievous violation of your rights? I'm sorry but that is an absurd assertion.
Again the licensing issue is a legitimate issue that needs to be addressed but this article is a straw-man argument using the licensing issue to promote a personal opinion.
Where to draw that line has no clear answers though.
There is a particular problem here. Large multinationals will want to maximize their potential revenue with their model, hence they'll attempt to include domains that are incompatible with liberal democracy. For example kowtowing to China and Saudi Arabia. These are already common problems with software and search engines as it is.
At the end of the day you cannot serve multiple masters with incompatible views. Intelligence, and thereby artificial intelligence operating in a subjective manner will have to pick a view and offend one of them.
Large AI models could go the way of centralized control (like manual book transcription during the dark ages), or decentralization (like the printing press, which brought us the renaissance).
As others have pointed out, these licences appear unenforceable.
The publisher simply wants to appear responsible. There are likely many open-source-oriented engineers and scientists at these tech firms who have been pushing for publication. (See the discussion between Mark Zuckerberg and Lex Fridman.) The involved tech firms only care about the licence as far as it might minimise the likelihood of public backlash if any of these published models cause harm.