F3 – Fight Flash Fraud | Hacker News Reader

F3 – Fight Flash Fraud

fight-flash-fraud.readthedocs.io

139 pointsCTOSian53 comments

I wrote a Linux/BASH tester [0] for this recently that includes instructions on how to create simulated fake devices:

To create simulated (2GiB) fake devices

    fallocate -l 1G fff_test.flash
    DEV=$(losetup --show --find fff_test.flash); echo $DEV
    DEVNUM=$(stat -c %Hr:%Lr $DEV); echo $DEVNUM

With wrap-around sectors:

    dmsetup create --concise "fff_wrap,,,,0 2097152 linear $DEVNUM 0, 2097152 2097152 linear $DEVNUM 0"

With silently dropped writes:

    dmsetup create --concise "fff_drop,,,,0 2097152 linear $DEVNUM 0, 2097152 2097152 zero"

To test:

    fake_flash_finder.bash /dev/mapper/fff_wrap
    Capacity mismatch at LBA 2097152, data wrapped around to block 0. Size is most likely really 1073741824 bytes

    fake_flash_finder.bash /dev/mapper/fff_drop
    Capacity mismatch at LBA 2097152, data does not match what was written. Size is most likely really 1073741824 bytes

To wipe the device if repeating tests:

    dd if=/dev/zero of=/dev/mapper/fff_wrap bs=64M status=progress conv=fdatasync
    dd if=/dev/zero of=/dev/mapper/fff_drop bs=64M status=progress conv=fdatasync

To remove the device:

    dmsetup remove fff_wrap
    dmsetup remove fff_drop
    losetup --detach "$DEV"
    rm fff_test.flash

[0] https://salsa.debian.org/-/snippets/732

This appears to be a cross-platform version of h2testw, which is widely recommended for finding the true capacity of a storage device. Another common recommendation is ChipGenius and the various manufacturer-specific tools, which can read the real ID of the NAND ICs, that AFAIK is beyond the ability of nearly all the fakers to change:

https://blog.elcomsoft.com/2019/01/identifying-ssd-controlle...

From the research I've done (mainly related to data recovery), the NAND flash industry seems extremely secretive and shady in many ways --- from the near-zero availability of public datasheets, to the many rebrands/"reclaimed"/recycled part sources, to what they're doing to SLC and higher-reliability technologies. There are also ways to determine how worn-out a NAND IC is, but even those may be reversible with the right physical treatments.

I did some digging in the days of yore just because I was curious about how these things worked, and I found absolutely nothing on how people make these things work.

I'm really amazed that unbranded 512GB NVMe drives doesn't randomly eat my data at this point, yet I still can't trust any of these drives w/o file level checksum patrols. So, instead I buy Samsung 9xx drives and use them.

Somebody should make this up as a handheld device. Something that you can use to inspect new items, and retailers can use to inspect what their suppliers are sending them.

Can a Flipper Zero be programmed for this? It connects to Micro SD cards and USB ports.

A retailer wouldn't want to deal with this. They would expect their distributor to weed out the fakes for them. If enough people complain about poor quality drives, you fire them. In my opinion, anyone selling these must be doing it knowingly at this point.

Distributors that care probably use something like this: https://www.ureach-inc.com/

Ah, but we live in the glorious future, where retailers are just fronts for flea-markets filled with third party crap, and claim zero responsibility. You thought you were buying a flash drive from Best Buy? HA, you actually bought it from AHJKHJ Pty Ltd! And no we won't help you!

All while pocketing the listing fee.

Too bad the ureach fails to provide some means to price or even buy one of their products. At least it doesn’t on iOS Safari.

Someone unrmployed should offer to MITM amazon purchases and ensure low priced cards get tested and reshipped or files fraud reports.

This won't work because there are multiple sellers on Amazon. I'd trust the original packaging of the device more than anyone supposedly running these tests for me, especially since they can take hours to complete.

Original packaging won't save you. Moreover, dealing with the manufacturer directly won't save you either in some cases[0].

[0]: https://www.bunniestudios.com/blog/on-microsd-problems/

Put it this way. I would trust the packaging over someone's word that they opened the packaging and tested it. I would trust it 10x more if I bought from the manufacturer's site.

Bunnie bought the cards from Kingston directly, in its original packaging (trays).

Cards were subpar at best, counterfeit at worst. Kingston exchanged them no questions asked after some pressing.

Delidding the cards revealed different components and construction.

So no, buying directly from the manufacturer brought no advantages or guarantees.

Regardless of the packaging and purchasing channel, I’ll only trust my own test. Nothing else.

Despite their good reputation (mostly associated with their RAM), Kingston is just a repackager of third-party parts. They don't actually make any of their products themselves.

Therefore, you indeed can't trust their original packaging if they themselves don't vet their supply chain properly.

This is in stark contrast to manufacturer-vendors like Samsung, Micron (Crucial), and SanDisk (Western Digital) who manufacture either all or at least the core components of their products and have their own manufacturing reputation on the line.

OK well, there may be exceptions. But surely you accept that most manufacturers are more reliable as a source than most resellers, don't you?

If you have to be 100% sure then there is no substitute for doing your own tests. But the context here is trying to avoid that either by paying someone to test for you, or by using a trusted source like some actual manufacturer.

> Can a Flipper Zero be programmed for this? It connects to Micro SD cards and USB ports.

I wrote something barebones for this on my Synology, using just a shell script and (hardware accelerated) openssl, if memory serves; that acceleration was crucial for handling, say, an 8TB hard drive.

Showing my age: based on the title, I assumed it was a ActionScript scanner.

Flash memory was invented by Toshiba in 1980, and marketed starting in 1987.

ActionScript dates back to only 1998.

The practical history of flash memory dates back to 1997 with the introduction of MMC by Sandisk.

I don't blame anyone for not knowing about Toshiba because they actively detested the technology and shunned its inventor and employee Fujio Masuoka.[1][2]

[1]: https://en.wikipedia.org/wiki/Fujio_Masuoka

[2]: "Toshiba gave Masuoka a few hundred dollar bonus for the invention, and later tried to demote him. But it was American company Intel which made billions of dollars in sales on related technology. Toshiba press department told Forbes that it was Intel that invented flash memory."

So this always overwrites the device to test it, and I was wondering if that was necessary (say, you had started using it). Presumably you only need to write at most N+1 blocks, where N is the number of blocks the device actually has.l, in order to detect exaggeration. But at that point the fake device will have overwritten all of your files anyway, even if they were theoretically on different blocks (of the exaggerated inventory). So I guess the minimum harm to test a device is to hash all your files, then write at most N+1 (unused) blocks, stopping after each to check if any of your files got harmed. In theory that risks at most one block...

Of course, it's better to back up the suspect one.

This also has the option of f3probe: https://fight-flash-fraud.readthedocs.io/en/latest/usage.htm...

> hash all your files, then write at most N+1 (unused) blocks, stopping after each to check if any of your files got harmed

That strategy adds O(n^2) reads on top of O(n) writes, though.

Even reads don't come for free on modern multi-level cell NAND (due to read disturb), and for just a thousand blocks, you'd end up reading the first block a million times.

That's to say nothing of the time this would take.

You're right, this is a non-starter. I should stop posting late at night. The owner of a suspect device just needs to bite the bullet and use a destructive method.

For a new device (ie no existing files on it), wouldn't the simplest approach be to full block 1 (whether 512 or 4k bytes) with a series of "1"'s, block 2 with a series of "2"'s, (etc). ie incrementing the number that gets written as the block number being written to is written.

Reading that back (either the full device or a random sample) should pretty quickly identify whether things are still in their expected location.

Well, with the remaining trust available at this point you might just as well use something cryptographically secure, like encrypted ones, twos, or simple HMACs of the block number.

A too-simple scheme is likely to be detected (and bypassed!) by the firmware a nearly no time.

Sure, that's a decent idea too. :)

Simpler: fill the drive with random data, hashing as you go, flush the kernel's buffer cache, hash the entire contents of the drive, and compare.

Conceptually,

  # tee /dev/DEVICE </dev/random | sha256sum
  # echo 1 > /proc/sys/vm/drop_caches
  # sha256sum /dev/DEVICE

though I wouldn't expect this exact command sequence to work unless tee's buffer size divides /dev/DEVICE's capacity and tee errors out writing past the end of /dev/DEVICE before writing to stdout.

I did exactly this earlier last week

the drive size divided by 4MB, so dd with bs=4M and fixed count

(with oflag=direct you don't even need to drop caches)

Oh, that's a clever way of doing things.

The "write the block #'s to the given block" would help identify where a fraudulent device goes wrong.

But for just checking if a device is storing data 100% correctly then your way would probably be more robust. :)

You are supposed to check an empty device. Some of the fake ones have firmware that will silently delete files or else fake writes. If you load it with data before confirming it is legit, you are likely to lose that data.

How long does this tool take to verify a real drive?

"it only writes what’s necessary to test the drive"

How does that actually work, wouldn't that mean the whole stated capacity would have to be written?

write 0001, read and confirm, repeat until drive is full.

For a fake drive, it'll take awhile, because the underlying storage is much, much slower than it should be, often usb2 speeds.

Realistically, this is just a test that satisfies curiosity without opening the drive. It's obvious when you have a fake drive because it won't benchmark anywhere near what it should.

> For a fake drive, it'll take awhile

Or, for any mid to large sized real storage. Writing/reading 64GB takes a while, fake or not

From what I understand, some of the fraudulent drives have firmware that just wrap the writes around to the beginning.

I had or had one in my possession that did exactly this. Another writes above a certain size were simply discarded.

The problem is the fake storage can just overwrite the new data on top existing data. Then you always confirm new data is correct because it broke the old ones.

But something like 2TB micro SD when actually it only has 64GB capacity, that will be very long time waiting 2TB to fully written.

How about write some file, then verify sometimes the new sometimes the old one, repeat until full.

Write(0.h2w) Write(1.h2w) Read(1.h2w) Write(2.h2w) Write(3.h2w) Read(3.h2w) Read(2.h2w) Write(4.h2w) Write(5.h2w) Read(5.h2w) Write(6.h2w) Write(7.h2w) Read(7.h2w) Read(6.h2w) Read(4.h2w) Write(8.h2w) Write(9.h2w) Read(9.h2w) Write(10.h2w) Write(11.h2w) Read(11.h2w) Read(10.h2w) Write(12.h2w) Write(13.h2w) Read(13.h2w) Write(14.h2w) Write(15.h2w) Read(15.h2w) Read(14.h2w) Read(12.h2w) Read(8.h2w) ...Read(0.h2w)

Define hash(x) that takes an integer and returns a sector-sized hash.

Define S = claimed total number of sectors.

  for(i=0; i<S; ++i) {
    write_sector(i, hash(i))
    for(j=0; j<=i; ++j) {
      if(hash(j) != read_sector(j)) {
        return i
      }
    }
  }
  return S

The above pseudo code will return the number of sectors the flash drive actually has.

Write everything before read or you will just get back cached results.

Cache can only mask performance. It can't mask fraudulent reporting of storage capacity. At worst, the test above will return (size of write cache + size of actual storage), but a bargain-bin flash drive is going to have a very small (if any) write cache, so it's not going to overestimate by much.

Why would they only have a small write cache?

You can map a lot of memory from pcie

Sure, so if the drive is genuine, or even one sector short (I recognize it's more typically a much larger fraction for the fraudulent drives), you'd still have to write to nearly the full stated capacity to verify.

This approach though, seems to require reading the first sector many times.

accidentally quadratic.

you just want two non-nested loops.

It's not accidental. It's optimized to minimize the number of writes necessary to determine whether the reported storage capacity is genuine.

If it is genuine your quadratic version will perform the same number of writes as the linear time solution.

If it is non-genuine what do you care how many writes it will perform? In that case the media is trash anyways.

The quadratic test will take an eternity due to the n^2 hashing operations and reads unless it terminates extremely early.

If you really have a need to terminate writing early, you could at least perform only a few reads randomly after each write (which will terminate early with high probably not long after you reached the maximum).

Does anyone know how robust it is? Could sophisticated malicious firmware detect that it's being probed by f3 and fake reads such that f3 thinks everything is OK?

As stated somewhere else, flash fraud has already been committed when you start testing.

Once you grab your dubious device, the seller has already got your bucks in exchange of a fake device.

You've been already and effectively cheated when those flash devices are being tested against cheats.

So what? F3 detects frauds, it doesn't guarantee that you can avoid them.

The argument is against the project name: “fighting” flash fraud suggests this project could somehow disincentivize flash fraud. Like how “fighting scammers” means things like taking down scam call centers.

Maybe some consumer protection agency will use the tool in conjunction with random purchases and some kind of enforcement action.

> Once you grab your dubious device, the seller has already got your bucks in exchange of a fake device.

Unless you're buying your flash device on the street and paying cash, you likely can return it or initiate a chargeback.

And even if you can't undo the purchase, it's better to know whether a device is fraudulent before you start filling it up with real data that you don't want to lose.

There are one or more platforms in the middle. In the US, if you report a fraud your credit card and Amazon will both go out of their way to refund you (Amazon might require a return but will have options to make that free if they do). Other platforms are similar. You, as a consumer, don't have any major problems from the fraud unless you needed a performant device with low delivery times.

Whether the fraudster has somebody's dollars varies, but for that kind of a scheme they're able to just hide in plain sight. If 100 people don't test the device (and it works for months or years) and 1 person does, they have a 5-star rating and can just eat the cost of returns. Even if everyone on Hacker News started testing devices it wouldn't make a dent in fraudulent profits.

> As stated somewhere else

So why state it again? This helps, for instance, people who go on vacation and takes pictures the entire trip, only to come home and realise they have only 16MB of storage, not 16GB, and their pics are gone.

I have bought fraudulent SD cards from Aliexpress at least twice. In both cases, the vendors immediately gave me a full refund when I called them out on the fraud (and let me keep the cards).

Reply on news.ycombinator.com