Hacker News Reader

Hacker News

Comment by FeepingCreature | Hacker News Reader

Parent Full thread

FeepingCreature

If a human can apply AI irresponsibly, the AI can probably also learn to apply AI irresponsibly.

View on HN

Not really, humans are the ones who think and have agency.

Agency? Sounds like a problem that can be phrased as a text completion task to me!

ChatGPT can already emulate beings with agency, see LangChain. It's just very bad at it. It's gonna get better.

> ChatGPT can already emulate beings with agency, see LangChain. It's just very bad at it. It's gonna get better.

What's an example that isn't a rote process?

Do you have any source to back that claim up?

Just look up any serious, non-quacky overview about how it works. If that’s excessive, just consider the empirical fact that we have created it (that’s also why we know exactly how it works), that might be enough.

We know exactly what it does. However, how it actually works we haven't the faintest.

If you really think so, I wouldn’t sleep at night if I were you. Thankfully, we actually understand fairly well how it works (cf. the “we’ve built it” part). That we may not know exactly how any particular output is produced is what sometimes happens when you build things that behave non-deterministically. Perhaps you give RNGs and bugs an undeserved aura of mystery.

We understand how Transformers "learn", ie. how the mechanism of its action operates. However, except for the most basic of cases, we don't understand how Transformers operate the skills they've acquired and can demonstrate, at all. See the field of interpretability for early attempts to change this.

For example, if you train a large network on lots of languages, and then fine-tune it to listen to instructions in English, it will, on its own, also listen to instructions in every other language it was trained on. Nobody knows why this is the case.

Humans only have the delusion of agency.

Reply on news.ycombinator.com