Comment by bwiggs | Hacker News Reader

Comment by bwiggs | Hacker News Reader

DEFCON30, Mayhem CTF.

We were given a file full of random bytes. The flag was in there somewhere. It was too random to be encrypted, there wasn't any structure. `file` didn't return anything, truly just a bag of bytes.

I had decided to install `hexyl` as an alternative option to some of the other hex editors installed o my linux machine. All the bytes were colored grey.

I scrolled the file and noticed a blip of yellow. A random golden `{` amongst all the noise. Weird.

The next colored byte was a `C`, then `T`, `F`.

---

At that time, I was mostly using HexFiend to look at raw files, which didn't have byte coloring. For DEFCON I had decided to drive my linux machine. I had ghex installed, but i had also decided to install and try `hexyl` via cli. So seeing bytes in color was purely by chance that I had installed it. I eventually posted an issue to ghex to add color support. https://gitlab.gnome.org/GNOME/ghex/-/issues/60

I need to see if I can find the file and post it on that blog post. https://bwiggs.com/posts/2023-08-31-hacking-in-color/

> It was too random to be encrypted

That's a rather odd remark.

You would still expect some amount of file structure, ex byte headers or something at the beginning/end of file. no?

That would be expected for encodings or container file formats. Straight-up encryption like AES produces results that are visually indistinguishable from random data.

I’d expect the greater length of the encrypted data (which should look random) vs the structured header/footer to rapidly push the Shannon entropy to the maximum

No. Why would you? Encrypted data should look no different than random. The app figures out if it is the app's data after it attempts to decode it.

It's not?

Compare a random data from a pseudo gen, a really random data and some encrypted data. They are all different.

If it's a CSPRNG, and decent encryption, they'll be indistinguishable from random noise.

Either you messed up somehow or you've made a massive discovery. I assume it's the former.

I don't get it. If you were looking at random data, why did hexyl apply colour to only the brace, C, T and F?

The implication is that the random-looking data did not contain ASCII characters other than the parts of the flag

Wouldn't strings(1) have worked for this?

By default, strings needs a run of 4+ (printable|ascii) characters. This sounds like it was 1 ascii character at a time in a sea of random data (with other alpha chars removed).

Ah, right. It would have to be `strings -n 1`, then.

Reply on news.ycombinator.com