Hacker News Reader

Comment by Tiberium | Hacker News Reader

That's been widely known for quite some time, there's no engine, it just happens "magically" when a LLM gets big enough (and they sure have a lot of that stuff in their training datasets). GPT-4 is generally very good at this, while Turbo will often make mistakes if you force it to decode Base64 that's some nonsense (like random characters).

View on HN

Sorry but I have to call bs. How do you explain this? It decodes a huge (guaranteed unique) string perfectly:

https://chat.openai.com/share/417169ed-8256-46cb-a3ed-81ab06...

Too generic to be used as a test. Try changing the comments to include gjdckfkdkxkdkskx34717Q 22371 re-encode and report back

Huh, you're right. It got the random string close, but not exact. I'm still blown away... what in its training set could predict that random string as base64 that closely?

https://chat.openai.com/share/02d616c2-5667-454d-9b72-43dfb1...

What about the 1-1 relation between the binary encodings of ASCII-characters and their base64 equivalents? I dunno, but isn't a base64-encoded character sequence still unambiguous / aren't the encoded character sequences formint a prefix-free grammar when concatenated?

Still very impressive.

What's even more interesting are the edge cases as presented above. Now that we consider almost-perfect natural translation a given, we're obsessed with marrying this to basic rule-based logic.

Me too, in every conversation I had with ChatGPT.

Because marrying the two seems almost magical and scary.

There isn't a 1-1 of ascii character to base64 character is the thing. 3 ascii character = 4 base64 characters, so changing one of the input changes 1 or 2 of the output.

Fair enough, what I meant by 1:1 wasn't the length, more the prefix-freeness thing.

Trivially, there is a 1:1-correspondence between the set of all possible ASCII strings and the image of that set under Base64 encoding, because otherwise it wouldn't be an encoding anymore :)

This doesn't guarantee that a Base64-encoded string can't be identical to a substring of a different Base64-encoded string. And for arbitrarily short substring length (bit), this doesn't make sense either.

So we rely on some definitions and minimal data structures such as bytes.

What I meant was that AFAIK, the ASCII character set might have an image under Base64 encoding that allows for prefix-free encoding given a minimum read length.

Reply on news.ycombinator.com