Hacker News Reader

Comment by K0balt | Hacker News Reader

I can’t help but wonder if there are “glitch tokens” for human minds, collectively or individually?

For hunams the msot iprmoetnt tihng is taht the frist and lsat ltteer be in the rghit pclae. The rset can be a mses and usaully you can raed it wouthit a porbelm.

It is a deffinert knid of hullunciaation but we hvae drffeenit bisaes tahn GPT-4.

Trehe are ptenly of otehr tpyes of gitlch tkoens for hamuns too. See opitacl iulloisns, or the fomuas peom The Cahos.

https://ncf.idallen.com/english.html

That's something of a myth. Or, rather, English happens to have so many short words that only a few permutations are possible if you fix the first and last letters. In something like German it would absolutely not work as well. And even in English, it becomes more difficult if you don't cherry-pick fairly simple permutations that preserve some important structure of the word, like "deffinert" or "porblem".

But on the other hand, yes, natural language has low entropy and lots of redundancy because it's meant to have error correction. From most sentences you can drop several words, not just scramble them, and the meaning can be inferred from the context.

It's not a myth, it is a hallucination that works in English. Which used because we're all speaking that language here. Like I said above, it is a different bias, but we have different biases than LLMs. Every language is going to have different biases. Some classes of heteronyms don't even exist in certain languages so a similar thing to The Chaos doesn't work as well there either. Maybe Chomsky would say there are more clear examples since he believes in a universal grammar but I'm also not a linguist. Just wanted an example. I wouldn't call this cherry picking as it both works in a few latin languages as well as strongly works in English. But even in English there is a bit of care you need, like double letters help to be stuck together. That's not cherry picking though as the example isn't such a unique thing that it invalidates the main thesis (what cherry picking is about).

It’s not a hallucination. It’s simply error correction, just like optical disks or ECC RAM are resistant to scrambled bits thanks to redundant, low-entropy coding – the fact that the bits are correlated.

Not sure, but the 'feature visualization' which attempts to maximize input for various terms looks a lot like psychadelic art to me somehow.

It also sort of reminds me of the "impossible colors" you can see by depleting the receptors in your eyes to see pure color signals that aren't otherwise visible due to actual light activating multiple receptors due to overlap in the color ranges.

Do you have any suggestions on where to find these?

For impossible colors go here:

https://www.thoughtco.com/impossible-colors-introduction-415...

For art, just compare any of these images to the pictures in the article:

https://duckduckgo.com/?q=psychadelic+art&iax=images&ia=imag...

Not as extreme, but garden path sentences sometimes give me a glitchy feeling in how they lure you into an parsing error that takes difficulty to overcome. For example:

The complex houses married and single soldiers and their families.

Not human, but this seems sort of similar: https://en.wikipedia.org/wiki/Chicken_hypnotism

My thoughts exactly. We used to do this all the time when I was a kid at my grampas place. There was other tricks with other animals too, but I can’t remember exactly.

Absolutely, see: https://xkcd.com/856/

This "glitch" is a short-circuit pathway from the input token to a result that makes statistical sense to the model's representation of the world, but very little sense to someone operating outside of that statistical model (you).

An internet meme fits that definition too: it's a short-circuit to some specific in-group context.

silk silk silk silk silk silk silk silk

What do cows drink?

I mean, baby cows…

cow

noun

1. a fully grown female animal of a domesticated breed of ox, kept to produce milk or beef. "a dairy cow"

(definition from Oxford Language)

But you're right, in practice people often use the word cow to mean calf.

First thing to come to mind for me are "thought-terminating cliché"s.

Reply on news.ycombinator.com