PCILeech
github.com
github.com
Introducing the Memory Process File System for PCILeech http://blog.frizk.net/2018/03/memory-process-file-system.htm...
Using your BMC as a DMA device: Plugging PCILeech to HPE ILO 4 https://www.synacktiv.com/posts/exploit/using-your-bmc-as-a-...
Fuzzing PCI express: security in plaintext (2017) https://cloud.google.com/blog/products/gcp/fuzzing-pci-expre...
(board+CPU for all machines both support it, dmesg confirms it is indeed on)
> Does not work if the OS uses the IOMMU/VT-d. This is the default on macOS (unless disabled in recovery mode). Windows 10 with Virtualization based security features enabled does not work fully - this is however not the default setting in Windows 10 or Linux.
For PCI (shared bus) the device could just spoof packets, but for PCIe, there are dedicated lanes for each device. I wonder if a physical MITM device is practical.
The core problem is that buses should be authenticated, authorized, encrypted, and selective-/least-privileged channels. Exposing a memory and expansion bus to the outside world in the name of "convenience" is insane. A trusted set of components in the OS and in hardware should:
0. Be able to do a HwIDS checksumming of all firmwares to detect tampering.
1. Limit devices' ability to connect unless they are authorized by the user, much like a "hardware firewall" UI.. vaguely similar to say VMware Workstation/Fusion's dialog when plugging in a new USB device mixed with something like Little Snitch'es dialog for a process wanting to connect to a particular port.
2. Authenticate devices with public/private key certs that are burned in, a function where the device can answer challenge requests, and Signal protocol-like construction properly modified for PKI. Then, and only then, can a host talk securely to a device over an encrypted channel.
https://web.archive.org/web/20071011191205/http://md.hudora....
(To be clear, those slides aren't mine, but I can no longer find the firmware we based ours off of, and never did get permission to post source to our mods, which were never distributed.)
I know a couple of our customers took entirely the wrong lesson from our demonstrations and banned mp3 players from their office buildings entirely afterwards :)
Speaking of things that look silly dept.:
Many moons ago around 1998 in uni, I had an HP 48 graphing calculator which was both programmable and had a very powerful IR LED that was usually used for serial transfers between similar devices. It so happened that some enterprising soul made a customizable, preset and code-learning IR remote control app for it that worked, so I put the brand of the lecture hall's 4 TVs in it. From the back of the room, some 15 meters away, I subtly turned on all of the TVs with a phreak'n calculator. Disbelief, confusion and hilarity ensued.
That’s great. I hope I’d have done exactly the same.
Hell no! That's how you get the dystopia of vendor lock-in and absolute control.
If I may be controversial: a little insecurity is a good thing. It prevents such authoritarian monopolies from forming.
Open standards are obvious and essential, and as such a CA for one doesn't have to be owned by any corporation, and are best run by a real non-profit that is independent of governments and bias towards any particular vendor. The hardware manufactures would pay a nominal fee to have a perpetual certificate and millions/billions of people get devices that can't be snooped on from the outside by authoritarian regimes.
Consider a NIC driver where you're mapping an outgoing packet for DMA. What used to essentially be a virtual to physical translation becomes a virt to phys + entering the phys in the iommu + removing the mapping when the transmit is complete. This is expensive for hardware and software reasons. At one point I benchmarked a 100g setup on linux, and with the IOMMU enabled, we lost about 90% of the bandwidth and most of the CPU time was spent in lock contention over the red-black tree that managed the IOMMU tables. This was 5-ish years ago, so perhaps things have gotten better.
So that makes people want to just enable the IOMMU for SR-IOV (and full device) pass-thru to VMs. This is cheaper, since you just set the mapping up when you allocate phys mem for the guest, and tear them down when freeing phys mem.
MacOS used to use a really cool trick where they pre-mapped all mbufs into the IOMMU. That made network traffic transmit and receive comparatively fast. However,it also prevented lots of optimizations that modern operating systems use for zero-copy IO (like attaching pages from sendfile directly to mbufs, similer to skb_frags).
We've done some benchmarks here: https://www.net.in.tum.de/fileadmin/bibtex/publications/pape... (Figure 9 on page 10)
Only a very basic benchmark, working on more...
I think ~100k to 200k TSO "packets" per second should be doable with the IOMMU. But I guess it depends where the data is coming from. Could be one of the odd cases where copying data is faster than doing zero-copy, e.g., just copy everything into the same small set of small-ish buffers to keep the number of pages that need to be present in the IOMMU small?
At most, each kernel driver has to do an extra addition to map its physical I/O offset to the one exposed to the bus by the IOMMU. With huge pages, there’s approximately one offset per driver, so it lives in cache, probably next to other driver state.
Yeah, I think the dTLB is only 64 entries on Intel CPUs as well, but there's a second larger layer behind that, and an even larger third layer. IIRC it's a total of 4096 entries on recent Intel CPUs.
For instance, XHCI (USB3) has an "address" field in its buffer descriptor structures that often it allows drivers to put anything into, and it will copy that into the "transfer finished" message. Some USB3 drivers just put the kernel address space pointer of the bookkeeping record in there; or worse, allocate the bookkeeping records along with the physical buffers themselves, so a malicious device could manipulate those if it knew where to look.
Datacenters have controlled physical access. I think IOMMU is far more important for anything with exposed thunderbolt ports (including the upcoming USB4). So Laptops, smartphones, workstations can still benefit from it even if it's currently not viable for cloud server-class workloads.
Edit: Bleh. Nevermind. I saw this photo:
https://gist.githubusercontent.com/ufrisk/c5ba7b360335a13bba...
with a pcie adapter connected over what looked like USB3 and forgot that it's thunderbolt on the macbook. I was not quite to the middle of my first cup of coffee when I asked that.
Edit: Bleh. Nevermind. I saw this photo:
https://gist.githubusercontent.com/ufrisk/c5ba7b360335a13bba...
with a pcie adapter connected over what looked like USB3 and forgot that it's thunderbolt on the macbook. I was not quite to the middle of my first cup of coffee when I asked that.
https://www.youtube.com/watch?v=MIfY8g73xms&feature=emb_rel_...
A few of the better anti-cheat products have had detection vectors for these for a while now, from simple things like detecting the driver to outright probing the device.