A Sampling of Anti-Decompilation Techniques
blog.ret2.io
blog.ret2.io
The kind of cleverness needed to thwart the decompiler's efforts is very impressive. A part of me wishes I had been around at a time when assembly was an acceptable "language" instead of the last resort it is today for most scenarios. Getting this close to the metal (or silicon) and being able to pull off such a stunt must be exhilarating.
At the same time, another part of me is very happy I do not have to deal with such low-level details. Debugging code written in assembly (with assembly being the main language, not just inline assembly in a C/C++ program or something like that) must have been exquisite torture.
Back in the microcomputer days, all you needed to write a program was a desk with a single computer, one or two tools that you knew through and through and a few reference books.
Nowadays, we sometimes spend days or even weeks choosing our tools and libraries of sometimes questionable quality and getting them to work together before we can even start thinking of the problem we want to solve, and even then, we still spend half our days googling, and being lucky if we get semi decent answers.
My most formative years as a programmer mostly involved 68040 assembly on the pre-PowerPC Macs. Back then, I used CodeWarrior a lot, and if there was a head-scratching bug in my software (as there often was), I could launch it from a fully-featured debugger, set breakpoints, skip ahead to a block of code, step through it line by line, see every single value that existed in raw hex on the heap, see exactly what was on the stack. If I was wanting to flip bits in someone else's software, I had MacsBug, which was a stop-the-world OS-level debugger that did all the same, but even better. [1] Back then, there was a very definite sense that I owned the computer, that it was my tool, that it would do precisely and exactly what I wanted it to. In a way, whether something was open source or not didn't matter; I could modify it anyway, and often for less effort than it takes to crawl through a modern over-architected codebase.
However, now I can glue some really impressive libraries together, written by other people and available on massive centralized code repositories, and deliver an application to as many people as I want, all running different hardware or operating systems (within reason) but still all getting the same software experience (...mostly...), and even charge some money for it without having to worry about somebody cracking the license check on my shareware.
The effort required to make complicated new software has gone way down. The effort required to troubleshoot and fix complicated software has gone way up. Nowadays, when I'm doing the former, I'm happy, and when I'm doing the latter, I'm not.
I think that might be part of why there's so much churn in the software industry today.
[1]: MacsBug did this in real-time. Start up an application, drop into MacsBug, set a breakpoint on a function call, switch back to the OS and application, trigger the function call, then step through the code in MacsBug. If you were careful, you could even "rewind" code under certain conditions, so you could, say, change a branching opcode, back up, go back to the OS and see if the change did what you want.
I wish I could upvote this more than once. ;-)
These days you can get a lot more done, more easily too, and help is easier to find, but there are a lot more levels of libraries between you and the machine. Choosing the correct platform takes time and sometimes guesswork.
But decompiling & debugging wasn't just limited to ASM. I once worked for a shop that lost its original C source code and only had a copy of the executable. (major source code control failure - no backups of the server). They decompiled it and worked from the resultant C code. It was as bad as you can imagine, and enormous amounts of time were spent renaming "variable1" to "customerName", or methods from "sub232" to "CalculateTax()", etc. After a few months the code was sprinkled with meaningful names, but it was still very difficult to read and felt very unnatural, formal, and distorted. It also ran slower than the original when recompiled.
But as for defeating a disassembler, I recall a few strategies stretching back to the late 70s/early 1980s on the Apple ][, when the "disassembler" was a human being rather than a tool. This is probably more appropriately called obfuscation or even copy protection, but it feels related. There were a variety of approaches taken:
* Self-modifying code. This was the most common approach. Code would loop through areas of memory, incrementing/decrementing values by set values or based on an algorithm to produce valid opcodes. At any point in time, much of the code loaded was nonsense, so debugging had to be done in decoded blocks. * Jump tables (and indexed jump tables with offsets calculated at runtime) so that the code would skip all over the place. This was mostly just an irritation, rather than a hard obfuscation. * Burying executable code in resources (sounds/images) and related techniques such as code that calculated its hashvalue and used that to self-modify to 'unlock' executable blocks. *Overlays. Code was loaded on top of unused resources, and resources loaded on top of code.
And while it wasn't a decompiler defeat, per se, changing the timing for the floppy disk drivers which read from non-standard disks would prevent one from simply loading data from disk and decompiling -- only that code that is in memory could be examined. Nybble-copiers were created to defeat this approach.
One or all of these approaches could be combined to produce code that was a nightmare to debug, but ultimately anything that the computer could execute could be decompiled, eventually.
Fun times.
http://mith.umd.edu/vintage-computers/items/show/11
Oh and I just found this thing called the “Senior PROM” which I’d never heard of before:
http://planemo.org/retro/senior-prom-reverse-engineering/
Here's some examples of early copy protection techniques:
Being able to use hardware tricks for programming, like non-standard display modes, self-modifying code, or fit a few more bytes into 64 KB was worthy of geek brownie points, specially at demoscene parties.
Probably not as glamorous as you're imagining. It took a long time to get anything done, and most of that time was spent finding bugs. All the time a lot of people thought the real cool people where the ones before who had to program using wires and vacuum tubes.
Best advice is, don't try to mimic those of the past, become a master of your own domain and change with the times.
1. Movfuscator page https://github.com/xoreaxeaxeax/movfuscator
2. Derbycon 2015 video https://www.youtube.com/watch?v=R7EEoWg6Ekk
references Movfuscator discussions:
The following sequence of bytes:
b8 50 83 ec 10
mov $0x10ec8350, %eax
push %eax
sub $0x10, %esp
All of this gets easier if you can augment your static analysis with control flow traces from program executions with coverage of the relevant branches, so you don't miss basic block entry points.
That's only true if the traces you can generate do a reasonable job at covering the states the application can find itself in, which is a big assumption.
But wasn't what got scrambled imports, polymorphic code nasty anti debugging the shareware packers? Asprotect and whatnot.
A precursor to decompilation is control flow analysis, the production of the control flow graph you see in the "before" stages in all of the examples in this post. You can go one step further, on a good day, and make it very difficult (perhaps very very difficult, perhaps impossible) to recover a precise control flow graph for a function.
There are a few different ways to do this, and I like these approaches more than targeting specific heuristics in IDA/Hexrays because, on a good day for the obfuscator, you can make a theoretical statement about the work effort required to un-do the obfuscation. If you can make that work effort large, then you start to have a security guarantee that is a shade of the security guarantee you get in cryptography. The methods outlined in the parent blog post are great because you can start using them today, but if they annoy Ilfak enough, he'll fix them and they'll stop working.
http://www.secdev.org/conf/skype_BHEU06.handout.pdf
The techniques described in that document are different, but in a way they are alike - both use dirty tricks to shield themselves from people trying to reverse engineer them. The only difference is that one takes place "at compile time", in way, while the other works in a dynamic program as it is executing.
Can it make harder for software crackers to achieve their goal ?
upx -d packed.exe
It's been long time since I've hacked anything but first one had fun by storing the decompression code into the interrupt table (yes, there). Or by making sure the decompression routine used accurate timings (so that if you fiddle with step by step execution it fails). Or calculating jump's based on values on the screen (that is, not the values show by your debugger).
That way the decompiler should, at least, be able to execute the code.
But that was before the protected mode in intel. My knowledge stops there.