Hacker News Reader

Exactly this. The tokens generated should always be valid, unless some post-processing layer between the model's output and the user interface detects for some keywords which it would prefer to filter out. In which case I suppose there is another commonly seen error message that appears?

View on HN

Not really, right? There are a ton of special tokens, like start of sequence etc., so what happens if there are two start of sequences predicted? It's a valid token but cannot really be turned into something sensible, so it throws an error when converting tokens to plain text?

Special tokens are handled by the application, not the model. They are still output before then.

Reply on news.ycombinator.com