8,782 karma · joined November 5, 2008
Nor is this a trivial decision. AI has the potential to change society and economic relations as profoundly as the industrial revolution, or the invention of the printing press.
These boundaries are also contested, as interests which benefit from a particular application are different from those whose interests are harmed. Society needs to identify which usages have a net benefit.
It also needs to define which usages cause "absolute" harms which is will consider unacceptable regardless of benefits to some parties. Such as, potentially, reductions in personal autonomy, increased leverage or dominance by government or private interests.
Not only that, but data and models which were collected/ built for one purpose can easily be adapted for others.
This is also basically now happening all at once in many domains.
In short, you can expect there to be tension over these boundaries for some time. It's not realistic to expect that others will agree with your personal perception of which applications are "obviously" unproblematic.
In the early days one of the high profile soaps in the UK published their "catch up" summaries for the week ahead which you could get just by editing the date in the URL. But back then not so many people were looking, so they were doing it for months...
However, at my old employer they didn't get commit rights to the main software repos. They had to carry around a bunch of patches which were gradually cleaned up and integrated. As I didn't directly work with them, I don't know if this says more about them or the guys managing the internal development.
It's a role that fits a different personality to that typical among software engineers. If you're bored as a dev it may suit. Pointless to try to shoehorn people into the role that don't suit it though.
I had an Aunt who had dementia. This is obviously a terrible outlook. But her and my Uncle seemed to be doing okay. My Uncle was an ultra-competent guy, highly stable, the person you could rely on; and had been his whole adult life. So it was shocking when he had a mental breakdown and became manic. There was probably something physical going on, but what was also going on was that he had wanted my aunt to be able to live normally for as long as possible, so had been covering everything - for a year or more he'd had to be alert 7 days a week in case my aunt tried to cook on the gas stove or something like that, at which she was no longer safe. So the risk-alert part of his brain had been constantly overworked.
I appreciate that this is scientific research, but there are definitely companies out there that will try to row-hammer everyone's brain if this sort of thing is not heavily controlled.
It would have been difficult given the state of other technology at the time for the inventors of the internal combustion engine to have supplied it as a drop-in horse replacement for your carriage, but you could kind of imagine that working with current technology.
At some point, when something is not working, you have to write off the sunk cost. In many cases, there's no obvious threshold as to when you make that decision, and it's easy to think that maybe, if you'd just held out a little longer, been a bit more determined, you could have made it work. But you have to make a call, without knowing for certain either way. And it can be the right call, even if, with perfect information, you'd have made the other choice.
So, while I'm sure lots of the people saying "you should just have tried harder" are doing so with complete good will, it's not helpful. You don't know, for another random person on the internet:
* how much work they put in
* what other burdens they were carrying
* what other needs and responsibilities they could use that time and energy on
* how their mind is different to your mind
etc, etc. In short, you don't have most of the information needed to make that decision for them, and you don't have any call to judge them for it.
On the subject: some people find meditation very helpful, others find it a net negative, or useless, or impossible to do. So a categorical "you should do this" isn't correct or particularly helpful. Try it, if it works for you, great; but don't put it about that people who aren't doing it are being negligent in some way.
Had a competitor pull something like this with a previous employer. They were supposed to be interoperating with a standard, but they had a secret steganographic handshake, which they used to pretend that competitors products were unreliable (they had a first mover position in a smaller national market with specific requirements, so this wasn't shooting themselves in the foot). Our guys figured out the handshake and just silently implemented it. In this case, the competitor wasn't big enough to waste engineering time on multiple such hacks, but Anthropic have time (or Claude does).
As for not alienating allies: if it's this easy to piss off people to agree with you, maybe it's also worth thinking about how to talk to those you want to win over.
Opposing this requires:
* Linking to specific harms, which the public can emotionally resonate with. For example, scanning billions of photos for suspicions of child abuse will result in false positives that cause innocent people's kids to be taken away.
* Not seeming like an overheated conspiracy theorist. Feeling angry about this is legitimate, but it's not necessary to communicate in the same emotional register that you are feeling, even if it feels inauthentic not to. The public are saturated with people being publicly angry, much of which is purely performance. Deep concern may work better.
* Have plausible reasons for why this is happening. Yes, a few individuals like Thiel want to create a digital Stasi, but this would still be happening without them. Mostly this is driven by companies that want to make money, and officials who have a bias towards centralised processes, and are tunnel-visioned with respect to some issue. And people who are genuinely concerned about kids and haven't been given another convincing solution.
* Get facts straight. (Eg, rent/job IDs aren't a future threat. They are here)
Here's EFF on reverse engineering and the law: https://www.eff.org/issues/coders/reverse-engineering-faq
Historically a lot of competition in physical products was very much reverse engineering. Because you can buy them without signing your rights away. That's why companies are keen on patents and click-through agreements.
If you look at how "clean room" processes work, they are actually a form of reverse engineering. Also clean room technique exists to avoid your new implementation infringing copyright, not trade secrets.
There is a difference with anthropic, as no-one signs a licence agreement to buy a coke. But Anthropic are also not saying you can't publish the output of their models. It's not clear to me if trade secret law will (or should) cover a secret which can be extracted from information that licensees are not restricted from publishing.
* Creates a market for privacy tech of several million teenagers
* Wastes police time chasing down social forums which kids are hosting abroad using their pocket money
* Rubs the noses of the securirati in the fact that they've made it easier for terrorists to hide their comms among the thousands of teenage speakeasies
This is not the 80's when comms tech required capital and man-years of engineering. Setting up forums online isn't even a high-school project.
A lot of the reason people outsource normal software is its brittle security properties, not sure that even applies to an LLM - it can go and look up the latest security best practices just like an engineer can.