Microsoft Has Manually Patched Their Equation Editor Executable
0patch.blogspot.com
0patch.blogspot.com
This also compels me to "code-golf" the function even more:
push edi
mov edi, [esp+8]
mov ecx, [esp+12]
jecxz label2
label1:
push ecx
call sub_416352
stosb
pop ecx
test al, al
loopnz label1
jecxz label2
dec edi
salc
stosb
label2:
pop edi
ret
I've done plenty of patching like this, and indeed the relative "sparseness" of compiler output very often allows the more functional version to be smaller than the original. It's amazing how many instructions the original wastes --- notice how none of ebx, esi, or edi are used, yet they get needlessly pushed and popped; and despite saving those registers so they could be used locally, the compiler perplexingly decided to keep all the local variables on the stack instead. The "jump around a jump", with both of them being the "long" form (for destinations greater than 128 bytes away, not the case here) is equally horrible. This may actually be a case where today's compilers will generate smaller code for the same source.
Note that in 32-bit code, memcpy is typically implemented by first copying blocks of 4 bytes using the movsd (move double word) instruction, while any remaining bytes are then copied using movsb (move byte). This is efficient in terms of performance, but whoever was patching this noticed that some space can be freed by only using movsb, and perhaps sacrificing a nanosecond or two.
On older processors this was true, but since Ivy Bridge a REP MOVSB will essentially be as fast but smaller. Look up "enhanced REP MOVSB" for more information.
I think he was talking mostly about size.
Odds are good most programmers tinkering in machine code won't beat the performance of the compiler. That takes experience. It is a good rule of thumb.
I think it is easier to write smaller (size) code than a compiler, but when you measure performance it will beat you often until you get good. Alignment, x86 tricks... It takes a bit of knowledge to do well.
When optimizing for speed you have to consider many factors like the relative speed of each instruction, cache behavior (including size of the cachelines, associativity, number of layers, relative speed of the layers...), pipelining, branch prediction, prefetching, whether moving your data to SIMD registers could be worth it, what to inline and what not to inline, what to unroll and what not to unroll, constraint solving to optimize things that can be computed or asserted statically etc...
The only time ISA really influences optimization is for unique ones like IA-64/Itanium. Otherwise, optimizing for e.g. a modern Xeon vs a POWER8 is not terribly different.
The general trend is that it's been getting harder and harder to do that easily, which means people want to be more focused — something like OpenSSL can still justify hand-tuned assembly for various processor families because it's a widespread hotspot but as compilers continue to improve the number of places where it's worth the maintenance cost is going to keep shrinking.
In the early 2000s, the scientific HPC programmers I worked with were careful to maintain a portable C implementation which they could use as a check both for correctness and for an optimization baseline — it wasn't uncommon for a new compiler and/or processor to substantially close the gap relative to a lot of hard manual work.
Even today compilers tend not to optimise the function preamble/postamble away. I'm only half in agreement with the mantra: you probably can beat the compiler, but is it worth it?
There are a few situations where it's genuinely a good idea to write in assembler to be explicit about predictable behaviour. Short security-critical constant-time functions are a good candidate.
For example, the xchg instruction doesn't have any C equivalent. (although it has a C++ equivalent: std::swap) The programmer may see:
A ^= B; B ^= A; A ^= B
---------------
Most of the low hanging fruit have been taken up for sure. Almost every "memcpy" turns into "rep stos" for example (which is the assembly-language equivalent to memcpy).
A high-level programmer may not know that "memcpy" turns into "rep stos" however, and may emit his own memory copying for-loop.
At very least, a good optimizing C / C++ programmer needs to know about these little things. They'll let the compiler turn "memcpy" into "rep stos" (for -Os) or AVX memory store instructions respectively instead of writing their own less efficient loops on the matter.
Its also a historical artifact from the days when many programmers wrote assembly yet compilers started getting good.
There’s also an element of avoiding premature optimization: don’t assume the compiler will product slower code or that if it does it will matter in your specific application.
At the very least you should give the compiler a chance, profile, then hand-tune after you’ve fixed all the low-hanging fruit.
Word 2000, so a 17+ year old compiler. Compilers have gotten a lot better since then.
Having worked on a compiler team back in the mid 2000s, even then I'd say it was easy for almost anyone to spot areas where a human could optimize more.
Now days, much less so.
I tried equivalent C code and gcc-7.2 gets me 47 bytes, while clang-6.0 only manages 49 bytes (both with -m32 f.c -Os -fomit-frame-pointer).
I have a feeling that size optimization just isn't really important to (at least open source) compiler writers these days. There are more important things, like actual performance, standards compliance and nice diagnostics.
Some software authors do not use memcpy().
https://marc.info/?l=djbdns&m=96477313901746&w=2
#include "byte.h"
void byte_copy(to,n,from)
register char *to;
register unsigned int n;
register char *from;
{
for (;;) {
if (!n) return; *to++ = *from++; --n;
if (!n) return; *to++ = *from++; --n;
if (!n) return; *to++ = *from++; --n;
if (!n) return; *to++ = *from++; --n;
}
}The semi-official Debian server, alioth.debian.org, where a lot of random developer stuff is hosted, is stuck on Debian wheezy for various reasons. Most users, including myself (a Debian Developer) don't have root access to upgrade the server nor install new software.
The version of libapt-inst is too old to support Debian packages with control.tar.xz members (only control.tar.gz members). So we can't upload newer Debian packages to various custom APT repos that we host on that server.
I worked around this by looking at the libapt-inst source code, figuring out how to make it support control.tar.xz instead of control.tar.gz, and binary-patched libapt-inst.so to have this effect instead. It's actually fairly simple
1. there is a check for control.tar.gz, the failure branch prints an error and then returns. I overwrite this with NOP so it goes into the "success" branch.
2. then later it extracts the control.tar.gz member and pipes it through gzip. Luckily, nowhere else in the program uses the exact string "control.tar.gz" or "gzip" so I simply patch that string "control.tar.gz" -> "control.tar.xz" in the binary and also change "gzip" -> "xz\0\0".
(Actually given the change in (2), (1) is not necessary. But without it you get a bunch of spurious error messages.)
Applying this patch makes the resulting .so lose the ability of working with old control.tar.gz members (which is still needed of course). So my workaround does this:
LD_PRELOAD=libapt-inst.so.patched apt-ftparchive [..] && apt-ftparchive [..]
i.e. runs it once with the hack to pick up the new-style debs, and once again without the hack to pick up the old-style debs.
My motto is, "dirty solutions for dirty problems". :D :D :D
Jeez. How is security maintained? That actually scares me a bit.
I would guess the build environment involves lots of dependencies, lots of special config, lots of stuff which has to be the exact correct version, and all that knowledge has been lost as people have left the team and it wasn't properly documented.
Sure, you could spend a couple of weeks setting up a suitable environment again and relearning everything from scratch, but binary patching is probably easier.
They also make other software meant to make math more accessible to people with various disabilities.
The build team was very proud when they announced that the application would finally start being built from the source code in version control.
Stuff happens!
How would you go an entire decade without noticing this?
Wouldn't you have to use that missing source code for something within ten years?
Getting on for a decade ago now I was working at Red Gate when they bought .NET Reflector - a decompiler for .NET code - from Lutz Roeder. After the acquisition we started asking people what they were using it for.
Turns out a significant minority of them were trying to recover lost source code, or source code they never had in the first place (e.g., where a supplier went out of business). I don't remember the exact figure but it might even have run into low double-digit percentages. Bear in mind this is a tool that was being downloaded tens of thousands of times every month by all manner of people working for all kinds of organisations of every size and you can see the scale of the problem.
There were a couple of Reflector add-ins that would allow you to take a .NET binary and generate a C# or VB.NET Visual Studio project with all source code from it. The source code was never perfect and wouldn't likely compile first time, but it was certainly better than starting from scratch. Not surprisingly these add-ins were among the most popular.
Granted, times have changed, and I think source control is probably the default for almost everyone these days - although I would have expected that even in 2008 - but, bottom line: I think this sort of thing happens a lot, for one reason or another.
There is no way I could rely on something like that.
Am I the only one that thinks relying on something that has no source code is just asking for trouble and headaches in the future?
But those DLLs aren't the only, or the biggest, problems with the systems. Hence finding the source and building them from it isn't necessarily the top priority, although we are progressively doing exactly this.
I routinely update single page Powerpoints with more than 4 MB, due to the background image or logos.
The other thing to keep in mind is that Office documents are, in addition to being somewhat bloated internally, far more than just text. Think about all of the people writing things like documentation with hundreds of screenshots (all uncompressed BMPs), audio or video, etc. — and the people with those are far more likely to be at the kind of large corporations which expect value from their support contracts. There were multiple ways to deal with that problem but in general the easiest was switching to a 64-bit address space.
The point was that the code is too big and complicated, even for some of the best engineers in the world.
https://github.com/linker3000/Historic-code-PC-Pascal-and-AS...
If you can't rebuild it, you have to manually edit it, and when you do that you can't really change the addresses, not without a lot of headache.
Disassemblers exist. You can take the binary, generate the assembly code, fix it and then re-compile to find the needed changes. I cracked a few sharewares with OllyDbg this way (just for fun, never distributed), and I'm no "leet coder".
What they said is they found evidence that the binary was modified manually.
This is way more tedious that disassembling and reassembling a binary.
That's not the case.
Tools like IDA Pro, Binary Ninja, and Hopper make this quite easy. A good hex editor and knowing the file offsets is also fine. This is seen as magic because it is a bit of a lost art, but it turns out to be easy to learn.
Check out "crackmes" if anyone has become interested in this topic of mangling binaries by hand. They are fun and you will get results quickly on the easier challenges.
Also check out Microcorruption CTF.
It used to be stuff we did for fun.
Back in the day we might not even load the entire program into memory - I remember manually patching disk sectors on the C64 with tools that'd let me disassemble arbitrary content to see if it happened to match code.
I also spent a couple of years programming assembly directly in a "machine code monitor" - an assembler used to assemble/disassemble memory instruction by instruction rather than from a file.
This was something several members of my primary school class would do for entertainment.
The idea that this is particularly difficult just reflect that fewer developers have spent time getting familiar with assembly these days.
We still do! When I added Retroarch to my HTPC I wanted it to use the "ok" and "power" buttons on my remote instead of "enter" and "escape" which are only found on a keyboard. While I did contribute a patch to the Retroarch project, which I tested using a laptop, binary patching was much easier on the Raspberry Pi ARM binaries than figuring out the build system for LibreELEC (the binary patch drops support for enter/escape, so it's literally changing two bytes for the two keycodes).
It is a matter of hitting F4 in hiew.
We do this as a matter of course all the time. Patching a small handful of instructions is pretty easy. You could learn to do it on a week or less if you are a decent programmer.
Do it well? Do it quickly? Do it idiomatically and in a short amount of time.... Takes real skill.
One fun self-challenge was always to make my modifications as small as possible. e.g. one-byte changes were a lot more impressive than two-byte changes.
It's interesting. I have observed if people learn on an 8 or 16 bit machine, like in Microcorruption, they tend to be able to pick up more complex ISAs much easier. It helps to know the first principles.
Anyway, your points are pretty weak and oh-magic-driven, and I don’t see any reason gp comment to be gray or work to be called stellar. Though of course it was done by asm-skilled person.
It takes a lot of work to get to the point of skill demonstrated in this article... but there's still a lot of skill runway beyond that level of skill, too. It's simultaneously true that this is an impressive amount of talent, and that there are people for whom this would be an entertaining momentary side diversion from their normal job.
Basically, think of it like this: you have an old book, written in some magic runes and is told that a certain (quite short) paragraph is wrong and can be badly misinterpreted due to poor wording. You know, the magic spell goes kaboom.
You have tools that can easily and painlessly: a) translate the runes into a text that mere mortal can reasonably easily read; b) scrub runes off the page and write any new ones over; c) translate your text into runes. There is a simple correspondence between text length and how many runes it would take. Now, all you need is to write the new text that must be no longer than old one was. Not a trivial task, but not something extraordinary. Just rare, because we don't deal with magic runes those days as familiars take orders and handle all the gory details.
Writing assembly requires a skill. So does reading old assembly code of a particular function and figuring out what it did. That's admirable, but not something unbelievable.
It's just rarer to see those days, but not a lost art or anything like that. Crackmes are still alive. On the game cheat forums such patching (albeit, for a different purposes) was the norm, and probably still is for the games that aren't protected. And many embedded developers have their fights for code size quite often.
My patch was to the VC++ compiler nearly 20 years ago. We had source, and my fix was also applied to the source (which I'd imagine is still there today), but a binary patch also made sense in the short term.
The binary that I patched was used to build another important Microsoft product, and this bug was found late in the product cycle where any compiler change was risky.
We weren't 100% confident we had the exact sources used to build that version of the compiler (git would have been handy then), we only knew, plus or minus one day, what the sources where.
After carefully evaluating the binary patch versus the risk of building from uncertain source, the binary patch was taken to reduce risk.
I'm no reverse engineer, but this was a pretty interesting exercise in RE even though I had sources. I had no symbols, and the binary was optimized so that functions were not contiguous, cold paths were moved to the end of the binary. Just finding the code I needed to patch was not easy.
The code review was fun - a dozen or so compiler engineers reviewed the change on paper printouts - the most thorough review I've had in my career, and the only one that used paper.
To the best of my knowledge, this binary was never used to build anything other than that specific version of the product which I won't name - not that it matters really, the product is still in use, but that version is unlikely to be in use anywhere anymore.
I wonder if they patched this way because they wanted to maintain as much binary compatibility as possible, or if they don't have the original source/couldn't reproduce the build process.
Compiling the software with a modern compiler or linking to a modern runtime is very likely to bring obscure bugs in the codebase to the surface. It's pretty hard to replicate the entire build process that produced the original binary, even if they have the source code and everything else on hand.
Horrific, because the average programmer would consider patching the binary a worst-case scenario.
> there is nothing wrong about patching a binary
I would only trust a skilled assembly programmer to do this task without creating other problems, and most businesses don't have those on retainer.
That says more about the average programmer than it does about the reasonableness of binary patches.
I used to table in MASM on the 8088 and have dabbled a little on microcontrollers (6800) as well. But the last time I looked at modern x86, I was pretty lost.
I seem to recall Microsoft rebuilds everything from source daily (or weekly?). This is a common practice in companies with a large code base. "Nightly Builds" even come from Mozilla. The reason is simple - if you wait until it's time to release, you're gonna have problems. Even if a developer tests his changes locally, they need to be integrated in with everything else and retested.
If you're really serious about things you will have your tool chain under revision control. As a result you can reproduce binaries to the byte from source code, with the possible exception of embedded dates pulled in at compile time. This is actually a good fallback when someone finds a critical problem in old code - you start by finding the code and then verify by reproducing the affected binary from source. This is not technically hard, it requires discipline and best practices.
Chances are that Microsoft doesn’t have a license for bug fixes from Design Science (makers of MathType) anymore and isn’t willing to pay for this fix.
Alternatively, Design Science may not be able to deliver a version that, for maximum backwards compatibility, has only this fix (to minimize risks, they would have to have kept an environment around that hosts the compiler used back then)
> Well, have you ever met a C/C++ compiler that would put all functions in a 500+ KB executable on exactly the same address in the module after rebuilding a modified source code, especially when these modifications changed the amount of code in several functions?
It's quite possible they are still contractually obligated to maintain some pretty old systems where changes to the .exe would produce unexpected behaviour. I had Access apps/databases crash on a system if they were built by a different version of Access.
IDA is widely regarded as the best disassembler and debugger out there. It comes with a price to match too though.
Otherwise: https://reverseengineering.stackexchange.com/questions/1817/...
It's also a rite of passage to distribute cracked and boobytrapped copies on filesharing sites...
[0] It was actually quite usable once you got to know its warts.
Keypatch helps you do assembly overwrites in IDA Pro.
Binary Ninja lets you do assembly (and C shellcode!) overwrite patches, and even has undo.
I have my own project [1] for patching ELFs that relies on injecting additional segments and injecting a hook at any address, so as to not require in-place patches. It can also massage GCC/Clang output and inject that reliably into an existing binary.
[1] https://github.com/lunixbochs/patchkit
I have my own story about this as well. A few years ago I released a port of Uplink: Hacker Elite for the OpenPandora handheld with a few game engine patches, and some people were running into a bug: the game would enter the "new game" screen on every launch, even if you already had a save game to load.
I and couldn't find the exact source I'd used to build it and didn't want to spend time making sure I got all of my bugfixes into the vanilla repository, so... I went digging with IDA, found the topmost branch to the "new game" wizard, and patched the address to go to the main menu function instead. At that point you could still click "new game" from the menu and it wouldn't go through the patched address (so "new game" still worked), but you could also load an existing game, thus fixing the bug!
I still have nothing on Notaz, who statically recompiled StarCraft and Diablo for that community :)
That leaves the binary monkey-patching as the only reasonable solution. I'm pretty sure Raymond Chen still works at Microsoft...
From the ANALYZE/IMAGE command…
Patch information --- Indicates whether the image has been patched (changed without having been recompiled or reassembled and relinked). If a patch is present, the actual patch code can be displayed. (VAX and Alpha only.)
Redundant backups are especially important for software companies. It's scary to think how many startups give all cofounders and developers admin access to everything. It helps that git is distributed, but it's not hard to imagine a scenario where a ticked off former employee wipes everyone's laptops and deletes the hosted source code.
Even if you don't update the mirrors regularly, it's good to know that you have some copies of data in BitBucket/GitLab/Heroku/Google Drive.
> While Office has had a new Equation Editor integrated since at least version 2007, Microsoft can't simply remove EQNEDT32.EXE (the old Equation Editor) from Office as there are probably tons of old documents out there containing equations in this old format, which would then become un-editable.
MessageBox.Show("This document contains an old equation and you don't have the editor. Do you want to download the old editor?");
Even better if they could at least render the old equation statically using the new office, but not edit it. Then it would be almost insanely rare that anyone needs the old editor.
Microsoft doesn't preserve backward compatibility because they're stubborn; it's a key part of their value proposition to some of their biggest clients.
This should also be very easy to do e.g. by noticing whether you are in a setting where there is any risk of the scenario you say. If it's a home machine for example, then don't worry about compatibility, focus on security.
That can lead to a very ugly form of bitrot quickly. Do you convert every document you've ever touched every time, even ones you haven't needed in years, just in case? Do you worry that every time you convert a file it might corrupt the file in the process? Do you find some way to keep every version of the program available at all times and play try every version until it opens the file?
Backwards compatibility in general offers much greater means for archival.
In this specific example: losing backwards compatibility for ancient equations directly threatens the archival of math and science documents. That seems like it could have huge repercussions in some fields.
That code is written in some kind of sandboxed VM/bytecode. You freeze the bytecode when you release a version.
When you release version 20 of the app, it has bytecode to convert 1 -> 2, 2->3, 3->4, etc, all the way to version 20. When it finds an old file, it runs all the updates as necessary.
If there is a bug in the updaters, it stays there forever.
If there is a security problem with the updaters, thats whats the sandbox is for.
git rm -r .
git add EQNEDT32.EXE
git commit -m "Fixed CVE-2017-11882"
git push
[0] https://blogs.msdn.microsoft.com/oldnewthing/20121218-00/?p=...