Hacker News Reader

Comment by smusamashah | Hacker News Reader

LLMS are trained to predict next text. But examples like these look like they have also 'learned patterns'. If rot13 is applied on this minified code, will LLM still find meaning in it? if it still could, its more than just next tokens. Need to try it.

edit: chatgpt found out that its rot13 and couldn't explain the code directly without deobfuscating it first.

View on HN

Claude 3.5 Sonnet can natively speak double base64 encoded English. And I do mean it - you can double b64 encode something, send to it, and it'll respond as if it was normal English. Obviously base64 is a simpler transformation than rot13, but no GPT models can deal with double b64.

> Obviously base64 is a simpler transformation than rot13

Is it? It’s probably more obscuring from an LLM’s perspective, assuming the LLM has seen enough rot13 text during training. Spaces and punctuation are untouched by rot13, unlike base64, which means that word and sentence boundaries will still be denoted by tokens that denote those boundaries in plaintext.

it appears that openai's gpt-4 model can speak base64 as well. I jumped to your comment seeing if anyone else had tried it following the OP. double b64 I didn't try, but that is interesting.

> $ ask4 ' what does dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw== decode to? ' > A "dGhhdCBpcyBxdWl0ZSBpbnRlcmVzdGluZw==" is a Base64 encoded string. When decoded, It translates to "that is quite interesting" in English.

I asked Claude 3.5 Sonnet a question in Italian in rot13 and it replied in Italian in rot13, there are a few typos but it's perfectly understandable.

I tried with GPT-4o and it also responded in rot13, the response was on topic, but quite non-sensical, like GPT-2 or lower level.

However I can confirm that Claude was able to identify that it's rot13 and also respond properly.

Reply on news.ycombinator.com