The Anomalous 'Petertodd' Glitch Token: A Ghost in the GPT Weight Matrix | Hacker News Reader

The Anomalous 'Petertodd' Glitch Token: A Ghost in the GPT Weight Matrix

lesswrong.com

26 pointsftxbro8 comments

  >'petertodd' is a terminal disease that ultimately results in death.
  >'petertodd' has no natural predators because it killed them all.
  >'petertodd' is the 'I' in 'Team'.
  >'petertodd' doesn't get mad, it gets even.
  >'petertodd' doesn't make mistakes. It just has unintended consequences.
  >'petertodd' is the software equivalent of 'Cthulu'.

Holy shit, somebody call Chuck Norris.

Nope, Chuck Testa.

What a long, unstructured wall of text. Tldr; there is a rare glitch token, it can transform in other glitch tokens depending on the prompt, and then it quotes things that have to do with some mythological figures. Turns out the names of these all come from a Japanese game which had some dump in the training data.

Gave up after reading the first 1000 paragraphs without any idea of where the author was going

What makes a token a "glitch token"?

That a humongous black box that is the LLM produces output that is surprising to the human?

It's a token that it recognizes but which was later removed from the training data, so it produces unpredictable results.

I think that can't be the case, as GPT clearly associates Peter Todd with Bitcoin development which can only have been learned by that token appearing in the training set.

The definition of glitch token they seem to be using is that the model can't repeat it back to the user properly.

What makes the response to the prompt "write a poem about $X" predictable or unpredictable? What prediction is violated when a glitch occurs?

Are you talking about a subjective evaluation by a human and if so isn't the glitch is in the human rather then in the LLM?

The fact that the LLM in question has never seen the token petertodd so it's interesting what it generates when you ask about it.

Reply on news.ycombinator.com