USB Reverse Engineering: Down the Rabbit Hole | Hacker News Reader

USB Reverse Engineering: Down the Rabbit Hole

devalias.net

283 pointsalias147 comments

What a weekend find! This is a nice piece in a puzzle I'm trying to solve related to a consumer audio device with upgradeable firmware.

The platform of the device is used for both a prosumer audio device, a professional device, as well as other consumer devices. On top of that, the prosumer device appears to have different functionality enabled based on the part of the world you buy the device, but my hypothesis is that the hardware is the same, regardless of region, and the firmware just implements a dark pattern to limit functionality (perhaps to avoid additional FCC certifications, support incidents, etc.).

Anyway, I've spent a few nights extracting the firmware from other regions, found a great presentation where the company details the hardware and platform, and discovered that the audio capabilities of the hardware for the prosumer device approach that of the professional device (higher sampling frequencies and resolution during recording).

My hope was to be able to flash the device with a modified file system that enabled the features, but ran into the firmware appearing to be encrypted, but potentially decrypted on the host before being pushed to the device. I've been decompiling the drivers to see if I could dump keys or the decrypted payloads, but also thought about sniffing the USB I/O and dumping that way. It looks like this is a nice intro to help me on my way.

Sounds like a really interesting project! Hopefully you manage to figure it out. Glad I could help start your own rabbithole of a journey! ;p

Sounds like a real classic market segmentation tactic.

I recall reading about a oscilloscope or something that had two models, one monochrome and slightly worse performance, one color and slightly better better.

Turns out that if you flashed the firmware of the latter onto the former, the performance improved to match the latter's.

Basic thing is that by doing this, the marketing people has calculated that they may reach a larger total customer pool than by only offering the more expensive variant.

I can definitely understand the marketing/psychology appeal of it.. but it also feels like a bit of a rip that I either end up with a) a less good product, or b) paid more for the same thing. That is.. until we figure out the ways to DIY upgrade them ;)

And then get slapped with DMCA or something, because "circumvention"?

I'd be really leery about reflashing a $X,000 scope. The NRE on those are huge so just because it's the same hw doesn't mean you aren't still paying for something when you buy the more expensive version.

> b) paid more for the same thing

I find it interesting that, here of all places, people consider two different software applications to be "the same thing" just because they run on the same hardware.

I dare say a significant percentage of the development budget for any smart piece of equipment these days goes into the software, not the hardware.

Putting "#ifdef PROSUMER" or something to this effect into the source code shouldn't be too costly. Disabling existing functionality is easy and it feels like cheating even if it's called "market segmentation".

This is what developers like us tend to think. But consider this: the development cost of the sodtware does not change much in order to create feature reduced versions of most products. But if you are in a small market where the volume of expected sales for the full version of the product can barely lead to proftability, creating a cheaper version for a higher volume market is a sound business strategy. This would not work with a single undifferentiated product.

I'm not saying that it is not a viable strategy. But it needs to be taken into account that tech-savvy users will not see it as a fair play, and probably will not think twice before making the most of a cheaper version.

I think of it as people paying the higher price subsidizing people paying the lower price

You might be thinking of the Rigol DS1054Z digital oscilloscope. It's pretty popular with hobbyists because it's an inexpensive entry-level scope that can easily be "unlocked" to act like a much more expensive model, including better-than-advertised bandwidth.

and a full protocol decoder! For $350!

I heard the difference might be because testing determines lesser quality builds which then are limited in software to meet the respective specs within a broad error margin. The unlocked ones might work and display but you loose all guarantees on precision. This process is called binning, and is done with many electronic parts, most famously cpus that are binned according to energy efficiency, clock frequency, or cache size.

Of course that doesn't excempt them from attempting price differentiation, e.g. under the veil of this process.

Quite a few people have tested the bandwidth of their hacked DS1054Z oscilloscopes and found they exceed the specs of even the better model.

Not saying this isn't happening, but it seems not to be.

Reminds me of some popular headphones (I forget the name) that had two models. One was the standard model and one was the pro model. Someone went to repair their headphones and found that the only difference between the two models was some extra foam that was installed to distort the sound bit in the standard model... I wish I could remember who that was.

It was Sennheiser headphones: https://news.ycombinator.com/item?id=2214158

Thank you!

> I won't get deep on describing all of the facts, since that's what Wikipedia is good at

Sigh. So many links, so much reading, and not one to the USB spec (that I found by skimming and searching). It's not just this article, but seemingly every 'deep dive does it'. Why do people read everything on the Internet, no matter the source and quality, but skip (what is usually) the best, most important source?

I originally wrote this as a way to collate all of the things I was reading as I went from zero knowledge on the subject. I like to document the path I took, and don't really see the point in reimplementing the wheel when someone else has already done a good job of it. That might not align with everyones style.. but can't please everyone.

> can't please everyone

Absolutely; and pleasing an audience of one (me) isn't a great business move. Thanks for your exceptional effort.

> I like to document the path I took

I think people learn more from this kind of inductive approach.

To be fair, specs and standards generally aren't written as introductions, so they're kind of hard to make sense of when starting from zero. USB, with its infernal descriptors and endpoints, benefits mightily from third-party explanations.

> specs and standards generally aren't written as introductions

Sometimes, but I often find them to be pretty good, and if I look at third party sources first, I usually wish I'd come to the spec much sooner in the process; the actual spec tends to make most third party sources redundant or plainly erroneous.

But of course, not all specs are equal; some are unreadable or poor introductions.

Definitely agree here. I tend to like to get the gist of things/cliffnotes from people that have already broken it down, and then deep dive into the spec if I need to know some explicit detail.

Most of the time when I want to hack on something, a super high level abstraction is more than sufficient for my needs (eg. don't need to understand the inner workings of a CPU if all I need is an API). But then there are those times when the spec, in all it's verbose rawness, is the perfect tool for the job.

this is a great writeup, definitely falls into something I've been dreaming of building, but was planning to hack around the usb part of it (using a usb-uart adapter), but now I'm thinking maybe I do it right! should make for a great weekend.

unfortunately, it seems that the pre-req article is down, so here's the cached copy: http://webcache.googleusercontent.com/search?q=cache:https:/...

Glad I inspired you to hack it the right way! Thanks for sharing the cached link :)

I've sunk countless hours looking for a way to emulate USB devices in software under Windows. Two years ago there were many different ways. Today there are zero ways that I know of, apparently all removed within the last two years by an astounding tragic coincidence.

To elaborate on this section:

http://devalias.net/devalias/2018/05/13/usb-reverse-engineer...

- There are two versions of the USB/IP drivers. The initial version has a bug where disconnection of a device causes a blue screen. This version is signed by ReactOS, and so you can install this version's drivers without needing to jump through any hoops under Windows. Then there's a patched version that does not cause the bluescreen. Great! Except that this version is not signed by ReactOS. Okay, maybe we can get ReactOS to sign it -- nope, they discontinued their driver signing program because of new regulations by Microsoft put in place within the last year (https://www.reactos.org/wiki/Driver_Signing). You can also no longer install unsigned drivers on Windows 10 by putting your machine into test mode. This capability was silently removed by Microsoft within the last year. You can still put your device into test mode, but the driver signing checker will still block installation. This is a damn shame because USB/IP is a great project, and it even has a signed driver but the patched version of the driver is not signed. This is infinitely frustrating.

- Microsoft used to have this perfect thing called UDE (https://docs.microsoft.com/en-us/windows-hardware/drivers/us...). They removed it last year.

- Kernel mode drivers for Windows now have to go through this signing process (https://docs.microsoft.com/en-us/windows-hardware/drivers/in...) as of last year. As mentioned before, this is why ReactOS discontinued their driver signing program as it's too difficult to meet these new requirements.

I have read every forum post, stack overflow post, Windows technical docs, everything I could possibly find for a way to do stable, pure software emulation of USB devices on Windows. They all evaporated within the last year or two. So frustrating as I really really want to build out something but there's simply no way to do it.

At this point I have resorted to attempting to hook every Windows system call related to USB device initialization and spoofing them but have been met with no success thus far.

I should note that the reason I want to do this in software is because I want to distribute the end result to a relatively large amount of potential users as GPL'd software. For this reason hardware emulation or patching / downgrading Windows isn't an option sadly.

Does anyone know of something I missed?

Then there's a patched version that does not cause the bluescreen. Great! Except that you can no longer install unsigned drivers on Windows 10 by putting your machine into test mode.

Patching out the signing check itself may not be too difficult... and if the file the signing check is in is also signed and verified by something else, then patch that... it's patches all the way down. I remember doing this with Vista a long time ago. When the signing check fails it fails with a very distinctive error code/message, so I searched for that in all system files and went "down the rabbit hole" of patching out a few checks. I don't remember the details but there was less than a dozen bytes that needed to be changed. Previous cracking experience certainly helps. ;-)

Yeah but I want to distribute the result to users and patching driver signing checks out of their Windows installs is not an option. You are right though.

Instead of patching out one approach could be to add your own certificate to Microsoft's to allow yourself to sign your own drivers. but not nefarious actors.

This might be a dumb question but why not downgrade Windows?

I want to distribute the end result to a large amount of users, and asking them to do things like downgrade Windows or put their device in test mode (which doesn't even work anymore) is too much overhead.

Because then you would be suggesting your customer base to use insecure software. There is a reason IE6 still won't die.

It's hard to know, as you don't mention what physical USB device you're trying to circumvent, but virtualization has come a long way, especially if you take advantage of PCI pass-through. One approach, then, is to run Linux as the host OS, fake the USB device on Linux (hell, the Linux kernel has USB/IP support), and pass that into a virtualized Windows instance, which will just see a regular USB device.

This is an option I hadn't thought of, but the typical end user won't want to do this or know how as it's a gaming peripheral. I can't be more specific as there are some legal barriers I'm sorting out right now.

Myself and a friend needed to do something somewhat similar to get an emulation/mocking framework[1] for Corsair Utility Engine. Our hack was to run the Windows driver in a VM, run the framework in Linux, then pass through the virtual USB device to Windows.

[1]: https://github.com/ckb-next/fakeyboard

If it's an input device for Windows games, without revealing the device, why does it have to go through the USB stack? Basically all games on Windows use DirectX, specifically its DirectInput libraries. (It's been a while, so they've probably been renamed, but the point remains that USB isn't necessary for all input devices, though it is up the application as to what it'll take.) Many emulators are more flexible; might take a look there, and if not, there's also Bluetooth - do the fakery in hardware.

Because the software controller uses Windows system calls (SetupAPI & DeviceIoControl()) and not DirectInput. Can't do it in hardware for the reasons I mentioned in the op.

Ugh, drivers and signing is such a mess right now.

I have before me two tablets. Both running W10 home, both using the same Intel Atom internals. But apparently one is running a the 32-bit version and the other the 64-bit version.

And lo and behold, the 32-bit version has a problem with the latest sd reader driver that MS pushes, so that it fails to see the inserted card. But the very same card, and the very same driver version, and the very same windows updates, works just fine with 64-bit.

At this point in time i wonder if i should either go back to nature or switch every PC-like product in this house to a BSD, because F Apple for their silo, F MS for their back and forth Q&A, and F the Linux "community" with their constant music chair-ing.

Thanks for the links! Definitely interesting, and kind of annoying how hard it seems to be now :(

Would running a VM on top of windows and passing the device through to that be an option given your use-case? Could then run *nix or similar in there, and avoid the windows driver hell.

See my reply to fragmede, I hadn't thought of this but the friction for end users is quite high. Great idea regardless.

I see so much custom hardware. Can somebody tell me why running the drivers of interest in a virtualbox/qemu image, and logging interaction on the host OS, doesn't do a better job than custom hardware?

I'm not sure why this person listed so much hardware, I've done some USB protocol reversing using a TotalPhase Beagle 480. I think I probably could have done something in software, but the nice thing is the TotalPhase software is a lot like Wireshark -- which I was already familiar with. I wasn't totally familiar with hacking around USB so it made learning how USB worked much easier instead of me just pulling data off the wire and trying to start from scratch.

With the software-only approach (e.g. usbmon), the interface is even more like Wireshark :) https://wiki.wireshark.org/CaptureSetup/USB

Is that possible? I thought vbox et al enabled USB communication by allowing the guest OS drivers to hit the hardware directly.

Maybe you need Linux for the host OS, that not everybody already has. It might be like KVM switches, that nobody using Linux would ever feel like they needed.

Depending on your needs, you absolutely can do it in software. The first few steps are about software, and in particular, probably just using Wireshark + USBPcap or usbmon is sufficient for basic cases.

The hardware side of things is the area that interests me more though, if you want to get into higher speeds than can be handled in software only, or if you want to start doing weird/interesting things in the 'emulate' side of it (not just capturing (eg. facedancer). Coming from a security perspective, those sorts of tools are invaluable for getting to some of the depths that I may want to reach.

Reply on news.ycombinator.com