Hacker News Reader

Do you have any source to back that claim up?

Just look up any serious, non-quacky overview about how it works. If that’s excessive, just consider the empirical fact that we have created it (that’s also why we know exactly how it works), that might be enough.

We know exactly what it does. However, how it actually works we haven't the faintest.

If you really think so, I wouldn’t sleep at night if I were you. Thankfully, we actually understand fairly well how it works (cf. the “we’ve built it” part). That we may not know exactly how any particular output is produced is what sometimes happens when you build things that behave non-deterministically. Perhaps you give RNGs and bugs an undeserved aura of mystery.

We understand how Transformers "learn", ie. how the mechanism of its action operates. However, except for the most basic of cases, we don't understand how Transformers operate the skills they've acquired and can demonstrate, at all. See the field of interpretability for early attempts to change this.

For example, if you train a large network on lots of languages, and then fine-tune it to listen to instructions in English, it will, on its own, also listen to instructions in every other language it was trained on. Nobody knows why this is the case.

Reply on news.ycombinator.com