Hacker News Reader

Comment by learc83 | Hacker News Reader

This would have been perfect for my last startup. Keeping those damn Pi's running when they were deployed across the country was a nightmare.

The SD cards had around a 20% failure rate per year. Eventually I just started shipping each system with a backup SD card taped to the inside of the case.

View on HN

Were you using decent microSDs? Was each mSD getting QC'ed? We benchmark every single mSD before deploying an SBC with it, and have switched to only using Evo+ 32GB cards as they pass our QC 3/5 of the time, which is much better than the other available brands.

Also, mSD corruption means your using a mSD card with a bad controller. The mSD was bound to corrupt and was a low caliber card to start with, unless you are doing a bunch of writes a mSD should last 2 years minimum, if not longer.

Another tact to take would be to switch to an OrangePi PC+ (8GB eMMC included, $22) or similar, that way you know your getting good quality nand with a decent controller out of the box.

Do you mind sharing your quality control method for this? We are hitting card issues on relatively low device numbers.

Just saw this, TL;DR we transfer a large file onto and off of the mSD card. If its above 10MB/s for both (which is the base rating on a class 10 card), then it is a fine mSD card. I've also noticed that on multi-gig files speeds tend to drop to around 6MB/s, not the end of the world for us, but also not a sign of a bad mSD either.

From my experience the failures were almost always caused by a power loss.

The obvious solution would have been to include a battery backup in the case.

Odd, I've only ever seen this be an issue on the Raspberry Pi platform, outside of that power loss doesn't regularly cause data loss on most Androids or Allwinner boards so long as the mSD isn't of low caliber.

Removable storage is problematic when it comes to reliability. Simply powering the device off while it is writing will cause issues, worse is when you have vibration momentarily overcoming the sprung contacts to the card.

Also mentioned by others are SLC industrial grade cards which have more stable flash memory technology, better controllers, wider operating temperatures and not to mention are much much faster. It all comes down to cost, but trying to build a reliable system with removable cards is not a good idea.

The Pi design team should consider adding some reasonable quality onboard eMMC memory in the future.

Regarding cards, I can vouch for http://www.atpinc.com which we use where I work, expensive but very reliable.

"consider adding some reasonable quality onboard eMMC memory in the future."

The Pi "compute modules" include onboard eMMC and so do these which are based on them.

Have you tried SLC, pSLC or at least, industrial grade MLC cards? They are more expensive, but might be worth it in such applications.

See this whitepaper on why: https://www.cactus-tech.com/files/cactus-tech.com/documents/...

What did your startup do that required that many Raspberry Pis?

The Pis were used in a 24 gym and tanning salon system. They ran a touchscreen for customer interaction, connected to a custom board I built that attached to the proprietary network tanning control systems use, and handled RFID access control.

It worked well, but tanning regulatory changes killed it.

I have a cluster at home, and that's about what I'm seeing. Good wall warts and good power cables reduces the chances of it happening, but doesn't bring it to zero.

At work the SD cards lasted for about 1,5 year and then gave up. I now disabled Ext4 journaling and am considering using the F2FS filesystem which is designed for flash storage.

Reply on news.ycombinator.com