1,297 karma · joined October 3, 2015
https://github.com/martin-t/rec-wars
https://github.com/rustcycles/rustcycles
---
Blog: https://martin-t.github.io/
---
If you wanna talk, my email can be easily found from my commits. We can arrange something reasonably secure/private too. I am not a huge fan of Overton windows but they make a very satisfying sound when a brick flies through them.
Look, if you think I am wrong, you can surely put it into words. OTOH, if you don't think I am wrong but feel that way, then it explains why I see no coherent criticism of my statements.
Besides that, your whole arguments hinges on large companies being inflexible, inefficient and poorly run. Isn't that exactly the kind of problem AI promises to solve? Complete AI surveillance of every employee, tasks and instructions tailored to each individual and superhuman planning. Of course at that point, the only employees will be manual workers because actual AI will be much better and cheaper at everything than every human, except those things where it needs to interact with the physical world. Even contract negotiations with both employees and customers will be done with AI instead of humans, the human will only sign off on it for legal requirements just like today you technically enter a contract with a representative of the company who is not even there when you talk to a negotiator.
Not to mention companies will try to mandate hardware decryption keys so the binary is encrypted and your AI never even gets to analyze the code which actually runs.
It's not sci-fi, it's a natural extension of DRM.
AI proponents completely ignore the disparity of resources available to an individual and a corporation. If I and a company of 1000 people create the same product and compete for customers, the company's version will win. Every single time. Or maybe at least 1000:1 if you're an optimist.
They have access to more money for advertising, they have an already established network of existing customers, they have legal and marketing experts on payroll. Or just look at Microsoft, they don't even need advertising, they just install their product by default and nobody will even hear about mine.
Not to mention as you said, the training advances only goes from open source to closed source, not the other way around.
AI proponents who talk about "democratization" are nuts, it would be laughable if it wasn't so sad.
There's really 2 ways to argue this:
- Either AI exists and then it's something new and the laws protecting human creativity and work clearly could not have taken it into account and need to be updated.
- Or AI doesn't exist, LLMs are nothing more than lossily compressed models violating the licenses of the training data, their probabilistically decompressed output is violating the licenses as well and the LLM companies and anyone using them will be punished.
There's a threshold where you modify it enough, it is no longer recognizable as being a modification of the original and you might get away with it, unless you confess what process you used to create it.
This is different to learning from the original and then building something equivalent from scratch using only your memory without constantly looking back and forth between your copy and the original.
This is how some companies do "clear room reimplementations" - one team looks at the original and writes a spec, another team which has never seen the original code implements an entirely standalone version.
And of course there are people who claim this can be automated now[0]. This one is satire (read the blog) but it is possible if the law is interpreted the way LLM companies work and there are reports the website works as advertised by people who were willing to spend money to test it.
[0]: https://malus.sh/
I can't point out where I draw the line clearly but here's one different I notice:
A recommendation can be both a thing and an action. A piece of text is a recommendation and it does not matter how it was created.
Assistance implies some parity in capabilities and cooperative work. Also it can pretty much only be an action, you cannot say "here is some assistance" and point to a thing.
Yes.
I have 2 issues with "post-scarcity":
- It often implicitly assumes humanity is one homogeneous group where this state applies to everyone. In reality, if post-scarcity is possible, some people will be lucky enough to have the means to live that lifestyle while others will still by dying of hunger, exposure and preventable diseases. All else being equal, I'd prefer being in the first group and my chance for that is being economically relevant.
- It often ignores that some people are OK with having enough while others have a need to have more than others, no matter how much they already have. The second group is the largest cause of exploitation and suffering in the world. And the second group will continue existing in a post-scarcity world and will work hard to make scarcity a real thing again.
---
Back to your question:
I made the mistake of publishing most of my public code under GPL or AGPL. I regret is because even though my work has brought many people some joy and a bit of my work was perhaps even useful, it has also been used by people who actively enjoy hurting others, who have caused measurable harm and who will continue causing harm as long as they're able to - in a small part enabled by my code.
Permissive licenses are socially agnostic - you can use the work and build on top of it no matter who you are and for what purpose.
A(GPL) is weakly pro-social - you can use the work no matter what but you can only build on top of it if you give back - this produces some small but non-zero social pressure (enforced by violence through governments) which benefits those who prefer cooperation instead of competition.
What I want is a strongly pro-social license - you can use or build on top of my work only if you fulfill criteria I specify such as being a net social good, not having committed any serious offenses, not taking actions to restrict other people's rights without a valid reason, etc.
There have been attempts in this direction[0] but not very successful.
In a world without LLMs, I'd be writing code using such a license but more clearly specified, even if I had to write my own. Yes, a layer would do a better job, that does not mean anything written by a non-lawyer is completely unenforceable.
With LLMs, I have stopped writing public code at all because the way I see it, it just makes people much richer than me even richer at a much faster rate than I can ever achieve myself. Ir just makes inequality worse. And with inequality, exploitation and oppression tends to soon follow.
Surely you have to understand that you own a plot of land, a house, the number in your bank account or the clothes on your back only to the extent that somebody is willing to perform violence on those who want to use "your property" for themselves. That might be you yourself but you can't be everywhere at once and you can't be awake all the time either. That protection comes from mutual agreement of people to defend each other's properties, usually through some institution such as the police/army/state.
Why should intellectual property be any different?
Why should I not be able to make an agreement with people like me that we only allow certain people to use our work under certain conditions and if any one of us violates the agreement (or an outsider decides to ignore it) we use violence to stop and punish that use?
> the people you're giving the stuff away to
Not giving it to them, they are taking it. I am making it available with instructions who can use it and how. Some people take it, following those instructions, some take it ignoring them. Would you use the word "give" if it was about leaked source code? What about leaked nudes of your girlfriend or daughter?
> See: all of the AIs being trained on publicly accessible code (regardless of its license).
That's a circular argument.
LLM companies claim what they're doing is legal. At best they're using a loophole - statistical interpolating autocompleters did not exist when copyright law was being written, I doubt many people could conceive of them at that time. At worst they are actively and knowingly violating the law, not to mention consent, of the best most altruistic people in the world to exploit them and bring about a new era of inequality and oppression.
Anyway, just because somebody gets away with something does not make it legal and certainly does not make it right.
> That's just a consequence you'll have to accept.
Or I can build both social and technical means to control the usage. Nothing is perfect but then if you want perfect, why do you lock your car or home?
If people find this cool and wanna play with it, they can, just make sure to only mix compatible licenses in the training data and license the output appropriately. Well, the attribution issue is still there, so maybe they can restrict themselves to public domain stuff. If LLMs are so capable, it shouldn't limit the quality of their output too much.
Now for the real issue: what do you think the world will look like in 5 or 10 years if LLMs surpass human abilities in all areas revolving around text input and output?
Do you think the people who made it possible, who spent years of their life building and maintaining open source code, will be rewarded? Or will the rich reap most of the benefit while also simultaneously turning us into beggars?
Even if you assume 100% of the people doing intellectual work now will convert to manual work (i.e. there's enough work for everyone) and robots don't advance at all, that'll drive the value of manual labor down a lot. Do you have it games out in your head and believe somehow life will be better for you, let alone for most people? Or have yo not thought about it at all yet?
Things can always go wrong. We probably shouldn't strive for 100% safety because they we'd spend our lives in a padded cell. But we also shouldn't assume things are safe because they're common or routine.
It should be either something like "(partially/completely) generated by" or if you want to include deterministic tools, then "Tools-used:".
The Turing test is an interesting thought experiment but we've seen it's easy for LLMs to sound human-like or make authoritative and convincing statements despite being completely wrong or full of nonsense. The Turing test is not a measure of intelligence, at least not an artificial one. (Though I find it quite amusing to think that the point at which a person chooses to refer to LLMs as intelligence is somewhat indicative of his own intelligence level.)
> whether these things are actually sentient or not is of no pragmatic importance if we can't distinguish their output from a sentient creature, especially when it comes to programming
It absolutely makes a difference: you can't own a human but you can own an LLM (or a corporation which is IMO equally wrong as owning a human).
Humans have needs which must be continually satisfied to remain alive. Humans also have a moral value (a positive one - at least for most of us) which dictates that being rendered unable to remain alive is wrong.
Now, what happens if LLMs have the same legal standing as humans and are thus able to participate in the economy in the same manner?
LLM-creation ("training") involves detecting/compressing patterns of the input. Inference generates statistically probable based on similarities of patterns to those found in the "training" input. Computers don't learn or have ideas, they always operate on representations, it's nothing more than any other mechanical transformation. It should not erase copyright any more than synonym substitution.
LLMs are not persons, not even legal ones (which itself is a massive hack causing massive issues such as using corporate finances for political gain).
A human has moral value a text model does not. A human has limitations in both time and memory available, a model of text does not. I don't see why comparisons to humans have any relevance. Just because a human can do something does not mean machines run by corporations should be able to do it en-masse.
The rules of copyright allow humans to do certain things because:
- Learning enriches the human.
- Once a human consumes information, he can't willingly forget it.
- It is impossible to prove how much a human-created intellectual work is based on others.
With LLMs:
- Training (let's not anthropomorphize: lossily-compressing input data by detecting and extracting patterns) enriches only the corporation which owns it.
- It's perfectly possible to create a model based only on content with specific licenses or only public domain.
- It's possible to trace every single output byte to quantifiable influences from every single input byte. It's just not an interesting line of inquiry for the corporations benefiting from the legal gray area.
LLMs are lossily-compressed models of code and other text (often mass-scraped despite explicit non-consent) which has licenses almost always requiring attribution and very often other conditions. Just a few weeks ago a SOTA model was shown to reproduce non-trivial amounts of licensed code[0].
The idea of intelligence being emergent from compression is nothing new[1]. The trick here is giving up on completeness and accuracy in favor of a more probabilistic output which
1) reproduces patterns and interpolates between patterns of training data while not always being verbatim copies
2) serves as a heuristic when searching the solution-space which is further guided by deterministic tools such as compilers, linters, etc. - the models themselves quite often generate complete nonsense, including making up non-existent syntax in well-known mainstream languages such as C#.
I strongly object to anthropomorphising text transformers (e.g. "Assisted-by"). It encourages magical thinking even among people who understand how the models operate, let alone the general public.
Just like stealing fractional amounts of money[3] should not be legal, violating the licenses of the training data by reusing fractional amounts from each should not be legal either.
[0]: https://news.ycombinator.com/item?id=47356000
[1]: http://prize.hutter1.net/
[2]: https://en.wikipedia.org/wiki/ELIZA_effect
[3]: https://skeptics.stackexchange.com/questions/14925/has-a-pro...
Most countries used to be hereditary dictatorships ("kingdoms") just a few hundred years ago, then people picked up rifles (and guillotines) and changed that. Now, since we already have some semblance of democracy at the top layer of power, maybe we could revolutionize the lower layers without the risks inherent to picking up rifles.
I don't see why we should be controlled by sociopaths[0] 8 hours a day, quite often against our interests.
---
The other part of the solution is automatically and continuously redistributing ownership of the company according to hours worked and skill level. This of course has to be required by law, otherwise those sociopaths at the top have so much leverage that if you ask for anything other than money during negotiation, you'll get laughed out of the room.
[0]: https://www.ribbonfarm.com/2009/10/07/the-gervais-principle-...
That's my point - maybe FOSS isn't the absolute good we've been lead to believe.
It was a response to locked down proprietary software which increasingly became hostile to its users. And it is (from a user's perspective) better that that for sure. But from a dev perspective, it's not as good as it could be.
> my answers
Exactly, those are your answers, your work. We've spent a lot of our limited time working for other people's benefit because we believed in it or sometimes because it was fun. But ultimately, it's becoming clear other people don't care and will throw us under the bus as soon as we're no longer useful. And then there's people who are just looking for a way to take advantage of us.
And I want to exclude both from benefiting from my work.
We should strive to find methods to make good, productive, pro-social people to benefit while keeping anyone who wants to exploit us away.
Either you alone own the repo but then you're a single point of failure. Or you give those perms to others but then any one of them can abuse it (or get hacked).
I'd like to see tooling which requires consensus or voting to make certain changes such as archiving a repo or publishing a new release.
If you had the option to exclude only certain people (e.g. those who argues with you) from seeing/using your contributions, would you have done it instead of deleting your account?
I am asking because I've too been burned and it's very commonly how an open source contributor's journey ends. So I've been toying with the idea that contributors should be able to exclude certain people or perhaps even groups of people from using their work.
Basically "I give away my work for free for anyone to use and build upon but if you don't appreciate it, if you treat me like shit, if you do any of X Y Z which hurts me or other people, then you're no longer allowed to use it".
See my comment above - you still see AI as a tool because you're only considering what I call "AI" instead of aAI (actual AI).
Imagine Stephen Hawking level genius at every area of expertise, able to think faster than any human and cheaper than minimum wage but unable to interact with the physical world.
That's not a tool you adapt to use, that's a tool the owners of your company replace everyone with, except ironically those roughly minimum wage manual workers.
I think you underestimate how actual AI would change the economy. All white collar jobs gone, not just programmers - customer support, accountants, managers, therapists, teachers, lawyers, engineers, researchers, designers, HR, marketing. All gone. Everything you can do from home - gone. IF your job doesn't involve physical interaction with the world - gone. And even if it does, it's cheaper to strap a camera to someone's forehead and let actual AI tell him what to do.
These people will still need to eat and they are highly competent so they'll try to get into careers which require manual skill, driving the value of those down as well. And that's even before robotics get sufficiently advanced to stop replacing those. Everyone will get fucked except those who own the AI companies.
And that's how you get a revolution.
But I think people object to you saying LLMs are not coding. Using them certainly does not help the person using them think the same way making all the decisions while writing the code. But even if LLMs are a Chinese room, if they could produce the same output as a human (given the same goal and information context), they IMO would be coding.
The submission[0] right next to this one shows why.
Apparently, in the US, you are now a criminal if you fly drones half a mile from ICE vehicles. Some of which may be unmarked and even if marked, how exactly do you verify no ICE vehicle is in a 0.785 square mile radius? Anybody capable of logical thought sees that this is BS.
(Also, anybody who retained primary school knowledge can calculate the area. But ask a person on the street to do it and watch your faith in humanity fall. Ask them to point out the area on a map and estimate how many cars that would be...)
---
Even the lawyer who taught intro to law at my uni always said that the people who most often find contradictions in laws are engineers.
The problems always start when somebody takes an ideology too far. So let's figure out what is too far instead of rejecting the whole thing.
No idea if this claim is true. How do Americans transfer money? Don't your banking apps allow that?
The premise is flawed.
Your company has 3 layers: workers - managers - owners. When AI (actual AI, not "AI") takes over, that'll change into 2 layers: AI - owners.
How you're equipped doesn't matter because you're out of the picture. Not economically viable. Irrelevant.
You still get paid a salary, you can still save up or invest it, it's just that money only buys you ownership according to how much work (times * skill) you put in to make that money.
Any system based on market competition in which your scenario realistically happens was probably so degenerate it would end up being replaced (whether democratically or by force). My system, AFAICT, is strictly (in the mathematical sense) better than the current implementation of capitalism, it just has extra precautions against buying power. What it boils down to is you want a perfect system while I am proposing a system that's better than the current state and you reject it based on not being perfect.
BTW a part of your comment is a condescending personal attack which doesn't add to the discussion and is against the guidelines.
I would like to know how the author concluded that this is the majority opinion.