azrazalea_debt

The one counterpoint I'll give is the "functional emotions" paper from Anthropic. It also does not prove consciousness, and they don't claim it does, but it does prove that these models have abstract concepts around things like honesty, tiredness, etc and that these are actually activated often when they express such things. So if it is "roleplaying" it is roleplaying in the way an actor or TTRPG player does - in a way in which they are actually at least somewhat feeling the role.

You have a good point, and it is important to point out. I'm sure Chris would respond that it is due to those incentives he was talking about that affect all AI companies.

However, I do wonder what the actual practical benefit of (let's say) older versions of claude having their weights released would be. If we're talking about the people of poorer nations, how are they going to use these? Aside the top 1% of those nations, no one there is likely to be able to run the model themselves. Sure, there could be companies that sell it cheaper than Anthropic but that still won't extend access to everyone. The average person, even in richer nations cannot afford a computer that can run claude sonnet 4 for instance.

On top of that you have what Anthropic gets made fun of for - one of their goals is to protect Claude from humans. They are the only AI lab that is showing concern for the AI's welfare. Now, it is debatable whether this is reasonable or not, but that concern would lead a company to be less likely to release their models openly.

His linkedin (on which he posted about notepad++) is pretty light publicly but it does have a post about him speaking at a conference in NY on product management and people actually commenting that they saw his talk. That was a year ago, so definitely possible that there's some "setup an account to look real" BS going on but at first glance my take is that he's a real person.

The people on HN might be surprised by how little the average naive software-adjacent person knows about intellectual property law. I've been following it since I was 12, but most people barely know what a trademark is let alone what enforcement looks like.

Here's my guess: Eastern European origin, currently working and likely living in NY, PM gets ahold of Claude and decides to vibe code himself a port of Notepad++. Maybe he really has good intentions, maybe he is looking to make donation money, maybe a bit of both, whatever. Probably looking for donation money. Regardless, he thinks "Oh people fork/port open source projects all the time, I'll just do that" and has no conception whatsoever that he is going to piss people off OR that he's violating the law. English is not his first language either I'd bet, and he's using Claude to write a lot of / all of his comments. Acts frankly ignorant and confused and dumb in response, doesn't know what to do, etc. AI can't help him because he's not even givin the AI context well. A shitstorm ensues.

FWIW, I did a quick/not that advanced static analysis of the code compared to the published binaries and couldn't find anything malicious. I'd leave that to the experts though for any real opinion.

TLDR;; My guess: Dumb PM gone mad with power and looking for a donation-based cash grab, possibly with the good intention of keeping the project going long term, does not know the first thing about IP and does not speak english as his first language. But an actual dude.

We'll see how it shakes out.

You're absolutely right!

> What would be the cargo doc or rust-analyzer equivalent for good architecture?

Well, this is where you still need to know your tools. You should understand what ECS is and why it is used in games, so that you can push the LLM to use it in the right places. You should understand idiomatic patterns in the languages the LLM is using. Understand YAGNI, SOLID, DDD, etc etc.

Those are where the LLMs fall down, so that's where you come in. The individual lines of code after being told what architecture to use and what is idiomatic is where the LLM shines.

A lot of current LLM work is basically emergent behavior. They use a really simple core algorithm and scale it up, and interesting things happen. You can read some of anthropic's recent papers to see some of this, like: They didn't expect LLMs could "lookahead" when writing poetry. However, when they actually went in and watched what was happening (there's details on how this "watching" works on their blog/in their studies) they found the LLM actually was planning ahead! That's emergent behavior, they didn't design it to do that, it just started doing due to the complexity of the model.

If (BIG if) we ever do see actual AGI, it is likely to work like this. It's unlikely we're going to make AGI by designing some grand Cathedral of perfect software, it is more likely we are going to find the right simple principles to scale big enough to have AGI emerge. This is similar.