landave

Wow. It appears that movaxbx.ru copied multiple of my blog posts as well[1][2][3][4].

I wonder how hard it would be to take legal action (in particular to stop them from doing this altogether, as opposed to just taking down a single article). The site appears to be hosted in Russia[5]. From a quick glance over the Wikipedia page[6], it seems to me that Russian copyright law is quite similar to copyright law in most developed countries.

[1]: https://movaxbx.**/2018/06/05/f-secure-anti-virus-remote-cod...

[2]: https://movaxbx.**/2018/05/04/7-zip-from-uninitialized-memor...

[3]: https://movaxbx.**/2018/05/04/7-zip-multiple-memory-corrupti...

[4]: https://movaxbx.**/2018/05/04/bitdefender-heap-buffer-overfl...

[5]: http://ip-api.com/#movaxbx.ru

[6]: https://en.wikipedia.org/wiki/Copyright_law_of_Russia

LGPL or not, if I remember correctly, F-Secure does not enforce a valid signature of the 7-Zip library, so you can replace it yourself. Don't quote me on this though.

However, F-Secure applies several patches to harden 7-Zip and to fix bugs that are not yet fixed in the public 7-Zip version. So it is not clear whether it is always such a good idea to do this.

Why do you think so? It seems they have updated to 7-Zip 18.05 on May 11, 2018: https://forums.malwarebytes.com/topic/228610-vulnerability-i...

I think this is correct. Since _solidAllowed is set to false at the beginning of Code(), it will remain false if an exception occurs in the middle of decoding (CVE-2018-5996). This will enforce PpmError being set to true for the next item, which in turn will enforce the (possibly broken) PPMD state to be reinitialized. In some sense, this means that the new bug fix is a generalization of the first one, fixing both CVE-2018-5996 and CVE-2018-10115.

> does say VS2017 produce a smaller executable file, or a faster executable?

If I recall correctly, Igor once said that he tested the new VS compiler and it produced neither smaller nor faster executables. I believe there was almost no difference.

Okay, so these packages come with more mitigations than 7-Zip on Windows. However, looking at the source code, I am pretty sure they are affected by the same bug.

That's right, they patched CVE-2017-17969, which affected ZIP decompression. Interestingly, I believe they didn't patch CVE-2018-5996 (affecting RAR), which I published [0] on January 23 together with CVE-2017-17969.

[0]: https://landave.io/2018/01/7-zip-multiple-memory-corruptions...

You missed my point. It is trivial to find out what the magic number is. What is more important though: How exactly is the magic number matched? From what you have written, one might be tempted to simply check whether a file begins with this magic number. And this would be wrong. If you take a look at the matching in CPP/7zip/Archive/Rar/RarHandler.cpp:

    Byte marker[NHeader::kMarkerSize];
    RINOK(ReadStream_FALSE(stream, marker, NHeader::kMarkerSize));
    if (memcmp(marker, kMarker, NHeader::kMarkerSize) == 0)
      m_Position += NHeader::kMarkerSize;
    else
    {
      if (searchHeaderSizeLimit && *searchHeaderSizeLimit == 0)
        return S_FALSE;
      RINOK(stream->Seek(m_StreamStartPosition, STREAM_SEEK_SET, NULL));
      RINOK(FindSignatureInStream(stream, kMarker, NHeader::kMarkerSize,
          searchHeaderSizeLimit, arcStartPos));
      m_Position = arcStartPos + NHeader::kMarkerSize;
      RINOK(stream->Seek(m_Position, STREAM_SEEK_SET, NULL));
    }

7-Zip finds the magic number if it appears within some searchHeaderSizeLimit, i.e., the file does not need to start (at offset 0) with the magic number. For example, 7-Zip will extract a RAR file which begins with [00 52 61 72 21 1A 07 00] (instead of [52 61 72 21 1A 07 00]) just fine.

Just looked at both the packages source, and it looks like they are affected. At least all the vulnerable code is in the source package.

Of course, but I would still strongly advise against this.

If you really cannot avoid implementing something like this, you should inspect the 7-Zip code in order to be 100% sure that the magic number detection in your filter is identical (or matches a superset) to the one from 7-Zip.

HE-ASLR I am discussing with him right now, and I think we will get this.

But honestly, I don't think we will ever see a 7-Zip with /GS or CFG. Not only would this cost about 1% in binary size, it would cost an additional 1% in runtime performance loss. Additionally, it would require compiling 7-Zip with a modern compiler like VS2017. You're just asking for too much.

Note that the standard 'p7zip' package from Debian/Ubuntu doesn't support RAR. However, they have an additional package 'p7zip-full' or 'p7zip-rar' for RAR support. I didn't check explicitly, but I assume these are affected.

DEP was previously disabled because Igor used to compile 7-Zip with VC6, which doesn't support the /NXCOMPAT flag. I convinced him back in January to enable it for 7-Zip 18.01. Note, however, that 64-bit versions of Windows enforce DEP even if the /NXCOMPAT flag is missing. Since Windows 10, the 32-bit version does this as well.

ASLR was primarily disabled because Igor wanted to strip the relocation from the binaries in order to save about 0.5-1% in file size. I have discussed this with him, and convinced him to enabled full ASLR for 7-Zip 18.05.

So now we have 7-Zip 18.05 with full ASLR and DEP. Stack canaries (/GS) are still disabled though.

There were some misunderstandings that I want to clear up (maybe I will add them in an update to the blog post):

1. Some people mentioned that this would "only affect RAR files" and it would be safe to extract 7z files with 7-Zip prior to version 18.05. This is wrong, because 7-Zip detects the file type from the magic numbers at the beginning of the file. So the exploit can be renamed to 'exploit.7z' and it works just as well.

On /r/sysadmin, someone even mentioned that a temporary solution might be to block RAR files. By the same argument, this is unlikely to be effective.

2. Almost all versions prior to 18.05 are affected. I manually checked version 15.05 and 17.01, and they are definitely affected.

3. Not only 7-Zip itself is affected, but essentially all software that uses 7z.dll as library to extract files. This includes various anti-virus software. However, exploitation may be more difficult (though not impossible) if ASLR&DEP is properly enabled (on all modules).

I assume you mean a performance comparison? The runtime performance cost of ASLR on Windows is zero once a binary has been loaded, since the code is relocated at load time.

Stack canaries might cause a slight performance hit, but it is usually below one percent, since it creates only a small cost per function call for a fraction of all functions.

So I just tried to compile 7-Zip with VS2017 and /DYNAMICBASE. The main binary 7z.dll is 1,569,792 bytes in total, 9344 bytes (0.595%) of which are used by the relocation table. Enabling stack canaries (/GS) gives me a 1,578,496 byte binary (including the relocation table), so another 8704 bytes more.

What do you mean exactly by "these things"?

It may be that the blog post is difficult to understand simply because I have written it poorly...

Thanks for pointing this out. I just fixed it.

Yes, 18.00 beta is the patched version. The current release (non-beta) version is not patched yet. Moreover, the POSIX port of p7zip is not patched yet at all.

The RAR PPMd bug can only be triggered if many conditions are satisfied. For example, the RAR archive needs to be mostly correctly structured, and needs to have at least two items that are compressed with the right flags (e.g., RAR version 3, PPMd). Furthermore, the compressed streams need to be constructed such that the bugs are triggered. Hence, I believe the bug is difficult to hit with straightforward coverage-guided fuzzing.

You are completely right with the first comment. The antivirus product itself reuses parts of 7-Zip and is vulnerable itself. I mentioned this mainly because I did not analyze the original 7-Zip software, but only discovered that it was affected as well after I had found the bug in this antivirus product.

I admit that this is confusing, so I'll probably try to rephrase this.

I couldn't agree more, and I've always been a strong advocate of learning things bottom up, as opposed to top down.

However, it always amazes me how powerful abstractions such as high-level languages are. In particular, it allows people who know virtually nothing about the underlying mechanisms to develop cool applications. This is a great thing, and I think we should value it, even though it comes with some major problems.

Bitdefender's core has dynamically loaded modules that are distributed with the regular definition update, which runs fully automatically. So there is nothing to do.

That's interesting. Unfortunately, AVG has been acquired by Avast last year [1]. I already looked into the new version of AVG a few months ago, and found that they have replaced AVG's engine with Avast's engine. Since the scanner always runs as NTAuthority\SYSTEM in the current Avast version, I would assume that the same is true for the most recent AVG version. I'm not completely sure, though, so don't quote me on that.

[1]: https://press.avast.com/avast-announces-agreement-to-acquire...

The definitions are licensed with the engine and can usually not be modified.

In a nutshell, most anti-virus vendors that are licensing the Bitdefender engine extend it with their own engine to improve their detection rate. For example, they support more exotic file formats and binary packers, or fancy heuristics, etc. Essentially this means they have the full attack surface from Bitdefender plus their own attack surface...

That's right. The list of anti-virus products that license the Bitdefender engine is extremely long. Actually, I wanted to include the most prominent Bitdefender customers in the article to give an impression. I ended up not doing so, because some license partner use the Bitdefender engine with disabled archive extraction. In this case, they would not be vulnerable to this bug, and I didn't want to mislead someone into thinking they are.

I don't know. If you are not already familiar with the inner workings of bitcoin, it seems to me that a simple computation power secures the transactions is not really helpful. What does that even mean, to secure the transactions?

Essentially, for an article that is aimed to such a wide audience you need to find an appropriate simplification. Such a simplification will never be completely accurate. Therefore, I consider anything appropriate that is technically not wrong and does not create more confusion. For example, describing mining as a process of generating bitcoins would probably satisfy those requirements (ignoring transactions fees).

I agree that it would be nice to be able to explain what the real purpose of mining is, but I don't see a really good way to do this.

I agree with the pdf spec allowing some insane stuff.

However, I think it's quite a stretch to put any blame on Adobe for this one.

In essence, Avast has implemented their own std::vec in C for the management of the magic numbers, and they implemented it quite poorly.

As mentioned in the article, the find_magicnums function supports roughly 300 (!) different magic numbers. Adobe's PDF is not required at all to exploit this bug.

The graph is generated by binary ninja [1] fully automatically, and this is just a screenshot of the tool. Any alternative reversing platform or disassembly tool like IDA Pro can generate you something very similar.

[1] https://binary.ninja/

Thanks for the question, I probably should have made this clearer in the article. Just presenting some pseudocode and typedefs of structs may have given a wrong impression of how this works.

So to be very clear: I reversed the functions and types without any symbols. All function names, type names, and variable names from the article are chosen by me. In the actual code, those names are most likely very different.

For such a simple function as this, all you need is the control flow graph form of X86 disassembly as linked in footnote 4 of the article.