Debunking the LZ4 "20 years old bug" myth | Hacker News Reader

Debunking the LZ4 "20 years old bug" myth

fastcompression.blogspot.com

178 pointsbronson14 comments

This all looks like a lot more drama than it really is. There's a lot of reasonable discussion going on between the two parties now.

See http://seclists.org/oss-sec/2014/q2/681 and https://code.google.com/p/lz4/issues/detail?id=52&can=1 for more recent, measured responses.

The original article was posted at https://news.ycombinator.com/item?id=7951324.

And now its hit the front (Internet) page of a big paper. The Mars Rover hardly gets a mention, despite the headline. http://www.telegraph.co.uk/technology/news/10930090/Software...

"besides TCP messages have a hard-code limit of 64KB."

A legit criticism, but who would transmit an 8Mb datablock using LZ4 over a network? There are far better compression schemes whose lower performance tradeoffs become negligible when network latency is involved.

I am using LZ4 currently for a project compressing Netflow-derived data for a major ISP, even then it's in 64Kb chunks for quick access and decompression.

> but who would transmit an 8Mb datablock using LZ4 over a network

Well, an attacker would. The question is whether the listener accepts that block, decodes it and gets pwned.

The question is whether every single router between the attacker and the destination accepts that block and passes it on. TCP window scaling will get an ISO across a building faster, or data from EC2 to S3 faster, but it won't get your 8MB block through the internet backbone.

And, keeping in mind, that if you do manage to break something, it'll likely be the very first router in the chain, not the destination system. Exploiting said router in such a way that it will exactly rebroadcast the problematic packet (i.e. perform its intended function), from within your own shellcode--and then continuing that chain all the way to the destination--would be quite a feat.

Doesn't the compression happen at the frame level rather than at the connection level?

Depending on where the attacker and where the potential victim are located and depending the protocol used and on how the network is structured all of that may or may not be a problem for the attacker. Maybe he already owned the last router before the victim via a totally unrelated issue and it's just a single hop. I concur that this bug is unlikely to be exploited in the wild but I'm generally opposed to the statement "yes, that may be a critical problem, but who would do that?!" People that want to attack you do that.

Today I read an article where the response to a critical flaw in a process was "but that's not a real world scenario, those actions would need to be done deliberately and that would require criminal energy." Phew, we're safe then. Only positive energy around, move along, nothing to see here.

I wasn't so much saying "phew, you can relax" as I was saying "this is why you haven't seen a new Internet worm based on this concept."

It can still, of course, be done in one-off scenarios (attacking a peer on a guest wi-fi network is one easy possibility) but it's not Heartbleed-level scary, because you can't just "scan the Internet" for the vulnerability, and attack every vulnerable thing you find to useful result.

That's not particularly relevant, you're thinking at the wrong layer. Seen from a program sitting on top of TCP, there's no such thing as a TCP message/block. You can very well grab a single 16MB chunk with 1 read() call, if the socket buffer is sized for that.

Or you can append incoming data to a big internal buffer which you pass through decompression once that buffer fills up.

You would have to look at protocols sitting above TCP that transfers compressed data, that typically would define a message based protocol, and how an implementation decompresses that data.

Even then, like it's mentioned in the article you need a 16mb datablock.

What about Martians? Remember, the bug is on Mars.

Lobste.rs, of course, toplists the original citation in place of Yann's response. I find it better reading.

Reply on news.ycombinator.com