Testing an SD Card's true capacity | Hacker News Reader

Testing an SD Card's true capacity

blog.nothinguntoward.eu

159 pointsalecjw47 comments

A pretty big chunk of memory cards are fake.

"kind" fakes show write errors when writing data. Evil fakes just return garbage data.

I have even witnessed a few fakes which understand a fat32 filesystem and make sure the filesystem data structures and the first and last few kilobytes of every file get mapped to working flash so that a chkdsk won't pick up issues and browsing the files shows no obvious issue. Many cameras store jpeg thumbnails, and the thumbnails continue to show too.

that's not just bad that's intentionally evil

Could it be that a fake SD card would make all the pictures in it Black or garbled? I had that happen to me, but it was a 2GB SD card from 2007...

Sure. Fakes like that have been around for almost as long as sd cards themselves.

Then there's the fakes that modulo the address. Write beyond the end of the actual flash and you'll write over the start!

Seems like that would be the simplest way to design one. Just leave off some of the high address lines.

I had this a year or so ago from Amazon on a Black Friday deal. A simple I/O test is usually enough to confirm a fake. The sellers are aware of their stock being fake, full refund without return and reported it to Amazon. Ebay have supporting documentation about finding fakes on their platforms and testing authenticity. [0] http://www.ebay.co.uk/gds/Spotting-Counterfeit-fake-SD-Micro...

This is made a lot worse at Amazon because of the commingling of stock from different sellers.

Thats true but Amazon needs to create a more transparent reporting system that is buyer centered such as seller rating rather than just item. This was also highlighted with Apple chargers so Amazon has a problem with no 'solution' to preventing it. (https://www.forbes.com/sites/amitchowdhry/2016/10/23/apple-n...)

And the fact that Amazon will often block reviews that mention receiving a counterfeit or fake product.

There are some good programs already written to test your Sd / flash card / USB / whatever you want:

USB Flash Drive Tester: http://www2.vconsole.com/client/?page=page&id=13

H2testw: https://www.portablefreeware.com/?id=1519

Stock Wordpress takes it hard when HN comes around!

The Internet Archive has it: http://web.archive.org/web/20170711105235/blog.nothinguntowa...

True, thanks for the link. Shame that author uses Wordpress. I'd advice him to use a static page generator for the future.

I use Wordpress, too, because I'm more interested in publishing content than in dinking around endlessly with the latest fad in content management tooling.

To be fair, static blog generators have been around for years, and their use is hardly a fad, considering the number of blogs that people host on github etc.

It hasn't caught on as much among non-tech types, but it's certainly not a fad, and the solutions are mature, usable and open source. I can understand wanting various features that only a dynamic blog offers (like commenting without Javascript), and I run such a blog myself.

There are tools for that as well. Staticman for example.

Wordpress is still essentially the status quo outside of technical circles and for non-technical people I'd recommend it over most other static site generators.

Using Wordpress to generate a static site is a useful thing, too. Improves both the security and the speed.

There's a whole pile of caching plugins for it, too. I can't speak to their relative quality, though; I just stuck Cloudflare in front of my site and called it a day. Perhaps I will come to regret that.

Why buy an SD card of ebay or, alternatively, find a deal which is too good to be true (and is neither good, nor true, of course)?

A branded, well specced SD card is not really that expensive that I would risk - for example - my photos which are certainly much more valuable to me than a few $ saved. Even if it's in the 10s $ range.

I honestly just don't get it.

I don't trust any vendor anymore. I have managed to get fake SD-cards from a local retailer, Amazon, a webshop, and big supermarket chain.

It seems like the distribution channels are poisoned.

I bought a SanDisk SD card from BestBuy (at physical store), trusting that it was genuine, so I never ran any test. Only after a year or so, I think it might possibly be a fake, as the speed was incredibly slow - write speed was slower than my 7 year old SD card.

How you do you purchase cards to guarantee you are not getting a fake? I believe fakes are pretty much everywhere now, in branded stores as well. Maybe buying those from stores that serve people who care helps (e.g., B&H photo -- as you indicate, a pro photographer losing his photos would be furious). They try to act on problems and work with suppliers quickly, but even this is not a guarantee.

A better approach IMO is to test the card first using easily available tools and use backups. This is more work, but can save you a major aggravation. My 2c.

I reply to you, but mean both of you who replied to my comment.

Thanks! I was really not aware that this is a problem and will likely test my cards in trhe future.

This is atrocious.

> e.g., B&H photo

I basically buy all my camera gear from B&H. The prices are competitive, their shipping is prompt and reliable, and I don't have to think about counterfeits - I have a fair amount of trust that they will not devastate their entire professional-oriented brand by allowing fakes into their supply chain. Getting something fake from B&H would probably be more shocking to me that getting it from an Apple store! Amazon is a different story - you can hardly tell if it's the "real" sandisk (or whoever) you're buying from. Hundreds of listings from hundreds of unknown sellers for the same thing, all slightly different, maybe backed by amazon, maybe not? No thanks.

I'd rather have a reasonable amount of trust in the distribution chain and not have to run checks when the thing arrives.

Further to that, I buy as much IT stuff in general as I can get at B&H for those very same reasons. I even got a couple of 52-port PoE switches from them a few years back. Generally the prices are better than/comparable to Amazon and faster shipping (to the D.C. area, to be fair).

B&H has been the only online store (and maybe NewEgg) that I trust for SD/flash memory. Strangely, I bought a SanDisk SD card from Best Buy (in retail store) that went bad after a year of very light usage (write speed becoming slower over time), and although I'm sure it's genuine, it leaves a little bit of a doubt, even when I buy something from trusted retail store.

Thank you for your kind words and for taking the time to post this here. -- Henry Posner / B&H Photo-Video

Our company has a product that is being sold via channels. It uses SD cards for storage. Let me assure you, we have zero idea if our OEM uses good SD cards or the cheapest SD cards on a market that they can buy. We also test thousands of SD Cards to ensure that they work with our product should a user want to replace an SD Card. We acquire said SD Cards from "manufacturers" and all kinds of stores.

There is no rhyme or reason what works and what does not, what is fake and what is not. Apparently, we just had a 256GB no name, found on Alibaba SD card for $44 USD pass all the tests with a flying colors ( and go through about 100 full write/full read cycles ) while newest drop shipped from a "fulfillment" center 128Gb Samsung SD Card ( received via email directly to our Samsung contact ) end up being a 32Gb relabeled card.

[Edit: typos]

If you don't mind me asking... what's the link for that Alibaba 256gb no-name SD card?

The data I was going to put on it weren't that valuable, mostly music and phone photos - I wasn't that distraught when I snapped the old SD card. A quick test that it wasn't completely fake would have satisfied me.

I've been using F3 (http://oss.digirati.com.br/f3/) to test new SD cards. Installable via homebrew and seems to work quite well

Yes. I just run f3probe for a quick capacity & speed test and then run a single-pass, random data badblocks test just to be sure. If there are any 'bad' sectors, I just pass the badblocks output to mkfs. Be careful with your choice of card reader/adapter though--some starts to get too hot and end up breaking the card under heavy testing!

Title should be: Testing for a fake SD card

About the sync: you can bypass the fs cache by specifying iflag=direct/oflag=direct.

In fact, I'm quite surprised that it worked at all without direct. Even if sync writes the cached data to disk, it isn't exactly guaranteed that the following read will read them back instead of just returning a copy from the cache.

Nice, thanks for the tip!

nothing special here, just a 16GB card got marked as 64GB, this kind of scam has been there for ages. You got hit by dishonest marketing BS from big brands all the time as well, buying from reliable brand and its recommended (overpriced) channel doesn't solve the problem, it seem to become the standard way of doing business in storage (what is the likelihood to get a fake RAM/CPU/GPU to this extent?). I think that is actually the most scary part.

I was lately trying to buy a new SSD, surely you want a NVMe SSD, so I picked up the Pro version of this top brand SSD, as the spec is the best of the best you can find on the market. Did a simple test, fsync performance is actually worse than the 2 years old SATA SSD from the same brand and that SATA was a EVO module. Sure, they didn't advertise fsync performance, but do you think it is fair not to mention/disclose/report such very poor fsync performance when it is actually slower than their own 2 years old product?

I didn't waste my time to argue with anyone, got it refunded and bought the EVO version of the NVMe SSD as I thought I just forget about the fsync performance and save that $100 difference. Got the fancy EVO NVMe SSD, did a sequential write test, the throughput performance dropped like 40% after certain amount of writes. Carefully checked the product spec, there is some text on the box written in some size 2 font basically telling people to expect such performance drop as there is some cache style design there.

Well, at least I got this very good random IOPS spec, right? Did some tests myself, guess what? The advertised IOPS performance is only true when running in situations considered as rare or even abnormal for such consumer grade SSD - you need a very long IO queue depth. In plain English, if you just buy this consumer grade SSD and use it in a consumer setting, you are not going to get the advertised performance.

Fake or marketing BS? If the whole idea is to fool me to pay for their over stated stuff, then what is the difference?

The issue is that with these cards they're not just branded wrong, they often present themselves as being bigger than they actually are - so your OS sees it as being 64GB when it's only 16GB, for example.

Were it just a branding thing it wouldn't - from a technical standpoint - be quite so bad, other than you know, the intentional lying...

What is the percentage of SD cards which are not fake ? Are we sure that popular SD cards on Amazon are not fake ?

I do not know about Amazon but I have some interesting experiences:

Over last several years I have ordered a few hundred SD Cards, some from Ebay some from Amazon, some bought at the stores. Store cards were Kingston and PNY branded - bought at Microcenter. Amazon were some unbranded ones and Sandisks.

My method of testing is rather simple - for 64G card generate a file of 63864569856 bytes of random data, dd it onto a card dd result off the card, fingerprint both. If they match the card is probably good.

This is what I found:

* cards with fake sizes will fail to write entire file with IO error

* cards with questionable quality will happily spew the following into kernel log:

[6135586.258783] sd 7:0:0:2: [sdf] tag#0 CDB: Read(10) 28 00 07 6f 4f 80 00 00 08 00 [6135586.258785] blk_update_request: I/O error, dev sdf, sector 124735360 [6135586.654749] sd 7:0:0:2: [sdf] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [6135586.654752] sd 7:0:0:2: [sdf] tag#0 Sense Key : Hardware Error [current] [6135586.654753] sd 7:0:0:2: [sdf] tag#0 Add. Sense: No additional sense information [6135586.654755] sd 7:0:0:2: [sdf] tag#0 CDB: Read(10) 28 00 07 6f 4f 80 00 00 08 00 [6135586.654757] blk_update_request: I/O error, dev sdf, sector 124735360 [6135586.654760] Buffer I/O error on dev sdf, logical block 15591920, async page read

* Sandisk had the lowest number of the failures when bought on ebay - sub 10%, highest rate of failures when bought on Amazon - over 90%.

* Kingstons from Micro Center passed the tests.

* PNYs failed the test.

The only wrinkle that I currently have is that I use Sandisks in a dual card Nikon configured for dual write. So far two of the cards that passed the tests created a "Card error" reported by the camera on separate occasions with separate slots. I do not however know if the issue is caused by the the adapter or by the card. I just ordered a few new adapters to see if I can isolate it.

[Edit: Formatting]

[Late addition: below]

The cards that triggered an error in camera passed the read/write test after the error so I am inclined to believe it could be the adapter.

> My method of testing is rather simple - for 64G card generate a file of 63864569856 bytes of random data, dd it onto a card dd result off the card, fingerprint both. If they match the card is probably good.

One optimization:

  dd bs=1M count=N if=/dev/urandom | tee /dev/mmcblk0 | md5sum
  md5sum /dev/mmcblk0

Ah, I generated a file once and now it is sitting on my disk drive instead of generating it every time. This also allows me to guarantee that I have the data that I have written so I can see if there's a specific set of corruption that happens. That's for example how I found that certain cards do 16GB loop.

I've received some fake ones but they were unbranded... The ones that I ordered online on Amazon with the whole retail packaging have always been good so far.

Sounds like something an evil cloud storage company would do to promote the use of online storage... that and removing SD cards slot from whatever they can.

Hmmm, some start writingGB17 on the position of the GB1, so your 1MB read/test will make all green.

At least for USB flash drives that behave similarly, sometimes there are software tools that let you configure the drive's claimed capacity.

Yes and no. In the sense that many "manufacturer tools" for USB sticks (and controllers) are available (sometimes released by he actual manufacturers, sometimes released by some "OEM" or "Brands" as a recovery or formatting software, the most part AFAICT leaked), while there are very few (if any at all) correspondent tools for SD cards.

Here is a couple known articles by "Bunnie" of BunnieStudios, respectively 2013 and 2009:

https://www.bunniestudios.com/blog/?p=3554

http://www.bunniestudios.com/blog/?page_id=1022

Reply on news.ycombinator.com