Littlefs – A little fail-safe filesystem designed for microcontrollers | Hacker News Reader

Littlefs – A little fail-safe filesystem designed for microcontrollers

github.com

259 pointsmaydemir65 comments

The problem with this filesystem is that it doesn't actually protect against power fails. Mostly you are OK, but not always.

I run an afl-base fuzzer against it back in 2020, and found a bunch of issues leading to assert failures in the code. I wasn't even checking data integrity. The tickets are still open (and it wasn't obvious to me how to fix them).

Spiffs is another micro controller file system, but it also fails fuzzing tests very rapidly with various failures. Spiffs seems less maintained than littlefs.

Are there any other good, reliable, freely licensed, microcontroller file systems suitable for small flash parts?

I used it at work twice and it was mostly ok with power failures, thing is you have to be careful if you are mounting it very early on boot if your embedded OS can bootloop fast after LFS is already mounted, because if you nuke it with a flurry of power fails it will break. For example if you have a battery you probably want to be sure you have a reliable measurement for SoC and enough juice to stay up before you mount LFS.

> The problem with this filesystem is that it doesn't actually protect against power fails. Mostly you are OK, but not always.

Power failure is a tough problem in embedded file systems (aka--too small for Linux).

Effectively--you need to be "append-only with checksums at checkpoints".

Every one of those words is anathema to making your flash small and cheap.

Just integrated littlefs into my stm32 framework. Author has a helper library for coroutines, which can be used to plug in asynchronous hooks, making the fs non-blocking. My next step is to run SQLite on it.

SQLite on an STM32

STM32 is a very broad range of chips. STM32MP1 runs Linux and I am sure SQLite as well ;)

And even leaving the MP1 aside, the STM32H7 micro controller has a ARM Cortex-M7 core which runs up to 550MHz.

Just wild to me that a 'little' microcontroller for a few bucks is more powerful than any computer I had up to about 1998 or so.

Hurts even more when you were using a 450mhz PIII in 2006...

HA, same here!!

IBM 300PL by any chance?

Mine (256MB, 8GB) ran Windows 98, and (record scratch) a copy of Visual Basic 1.0 that I'd stumbled on an abandonware copy of while attempting to figure the internet out at the local library. lol

Then there was the the 486 with 4MB RAM and 8MB free disk that I was playing with just a year earlier (yep - 2005). And the 800MHz AMD Duron w/ 320MB RAM I used between 2012-2014. And now there's a 2004-era Pentium 4 in the corner here that I'm currently running ddrescue on to try and shoo away ~300KB of unhappy disk sectors. What can I say... work-blocking health issues guarantees life will be InTeReStInG haha

Still playing catchup to equilibrium, and taking longer than I'd like because where it matters I annoyingly go a little slower than everyone else (huff).

I'm almost certain you were probably doing something slightly more interesting on your PIII...? Delphi, maybe, or perhaps a newer (interestinger) version of VB? Or maybe a super-lightweight Linux setup and Vim?

(Oh now that reminds me of the old Toshiba laptop I found on the side of the road with a ~300MHz K6-2 and (IIRC) 32MB RAM in it... hibernation worked so well on that thing it's how I would typically boot into Slackware on it (4 second resume time on a 20 year old laptop?!?! yes please!). Sadly my install CD was a bit scratched so the only things that really worked were bash, grep and sed... which I attempted to write a package manager for Slackware in. I got halfway there too, but had to quit because the 15-line regexes I was writing were giving me a headache. However, to this day I have zero issue with regular expressions up to sed's level of complexity. I do badly need to go back and revisit the subject though.)

Hi! It was just some old no name PC gifted to me with an intel mobo and 128mb of ram. Ran Win98, but I soon discovered Ubuntu 6.10 which ran fine on it.

I got free ram upgrades and used it to play free games (tons of time in Wolf ET) and learn programming.

Tried a bunch of OSes after that. Eventually I got a free 1ghz PIII and mobo which was a nice upgrade that lasted until late 2007 when I built a Core2Duo machine.

Hope you get well soon.

Woops, apologies I lost this tab mid-reply :(

Very cool.

I only knew about Knoppix at that point, which being a LiveCD didn't quite like 128MB RAM (just realized that's how much RAM my PIII had too, huh). Properly switched to Linux (cold-turkey :D) around 2007, and my focus on programming took a giant nosedive as I discovered the internet. Upgraded from Pentiums/Celerons to an i3 a friend gave me in 2012 IIRC. Still figuring out how to juggle the internet and focusing on programming lol

I'm not "sick" per se, it's really just the whole head-exploding dumpster fire that is government support around mental health and "I really cannot jump this curb *points* so what do I do now?" et al. It's just a matter of time, fingers crossed things get figured out before I'm like 35 or 40... hmph

I remember pointing out to someone that they could run ISA-bus peripherals on an AVR-Mega processor. Not that there would be much point, but it would be cool to take a board from an original IBM PC and run it from a $2 microcontroller.

Take solace that your phone would have been a TOP500 supercomputer in the early 90s.

They may be fast but they are still stuck with extremely small amounts of memory.

Even STM32F4 which is sub-10$ can run SQLite decently. 4kb ram for stack is quite enough, and then whatever amount of working memory you can afford on it will make life easier. Using sram extension makes it trivial even.

Difficult but doable. You'd have to shim some syscalls and kludge the build into an embedded toolchain, but there's no technical reason why it can't work.

It's actually not that bad; SQLite has a very abstracted vfs layer that is pretty trivial to port to arbitrary platforms.

There is also FlashDB which is more lightweight and can operate directly on flash

Yes, it only needs like 4 or 5 VFS hooks to be defined. No syscall malarkey

The source files seem to state

    Copyright (c) 2017, Arm Limited. All rights reserved.

so this is from Arm? That is not so clearly communicated, but perhaps given the license (standard BSD 3-Clause) I guess it doesn't really matter. Might be politically sensitive in some organizations, but perhaps I'm just being a cynic.

Looks really nice, and stuff like this is seriously useful and can really make engineering on embedded platforms way easier.

The organization has one person who is also a member of Arm Embed, so it seems to be tied to Arm.

I haven't had chance to use this yet but it's on my list to use with an AVR. Does anyone have any experience using it, how is it? I really like how it's unit tested and seems fairly well documented and even has github actions setup to build and test which seems to be rare for C libraries on github.

It is a nice library. It's backend api is simple to hook into whatever storage scheme you choose to use. It's also nice to be able to disable wear leveling and use it as a fault tolerant private filesystem from an image file.

It's very nice to get your experience feedback. Thank you very much.

I'm writing Embedsys Weekly newsletter and so I have listed littlefs project for the next issue. Do you think you can eventually share to the reader your experience in few words? If you feel good with the idea, my email is embedsys@embedsysweekly.com

It's one of the filesystem options in MicroPython and we've used it extensively for a number of (medical device) products. It's been bullet-proof for us.

Hi Matt,

I just proposed the exact same thing to a sibling commenter. I plan to add Littlefs to the next issue of Embedsys Weekly. If you feel confortable to share your experience with my readers on 3/4 sentences it would be fantastic.

At my work, we are using littlefs on several products, it works quite well. The v1.X have performance issues when you have a lot of small files. But the v2.X solve them.

I am using it in a commercial IoT project where the MCU flash is split into 2 parts: usual program area and a file system to store persistent and temporary data like ota. The main purpose is flash usage leveling which especially critical on MCU-internal flash, and as a bonus, it's just neat to have a real filesystem where you can work with FILE*. In about 2 years in the field - no issues.

Compiling their example with the Arduino toolchain seems to fill up about 90% of an Uno's 32k flash space. I think that rules it out as usable on anything that doesn't have a larger amount than that.

Yes, it seems like that's a good reason to switch to a better board, if this is something you're interested in using.

For example, a Raspberry Pi Pico is $4 and has 2MB. There are many other nice options from Adafruit or Teensy.

I see that the Uno R3 is a popular product on Amazon but I don't really get the appeal.

The Uno has a _giant_ ecosystem behind it going back over a decade and it's more than enough for most beginner mcu projects. That's the appeal. Other than that, it's a pretty weak chip.

Is that an issue with Arduino? This PR [0] suggests a R/W compile is around 13k (or R/O for 5k) on an M0. I don't know if there'd be a big difference compiling for AVR though.

[0] https://github.com/littlefs-project/littlefs/pull/480

The design of littlefs: A fail-safe filesystem designed for microcontrollers - https://news.ycombinator.com/item?id=20845112 - Aug 2019 (72 comments)

A little fail-safe filesystem designed for microcontrollers (2017) - https://news.ycombinator.com/item?id=18989705 - Jan 2019 (30 comments)

This is used in InfiniTime, the PineTime smart watch firmware :-)

See https://github.com/InfiniTimeOrg/InfiniTime/pull/438

Of 545 commits, 193 mention "fix" or "fixed" or "bug". That is 35.4%, quite high, compared to 10 large C projects I analyzed in 2019, which have a median of 25%.

That could mean file systems are hard to create, or that this particular one is buggy.

https://danuker.go.ro/frequency-of-bugfix-commits.html

Edit: looks like it's a problem with other file systems also:

https://github.com/pellepl/spiffs - 40.2%

https://github.com/tytso/e2fsprogs - 36.8%

But a reimplementation of exFAT is not as buggy: https://github.com/relan/exfat - 19.4%

Is it a problem, or is it that filesystems are not something you need to add new features to as often and thus commits are biased towards fixes? Or some other reason for a bias?

Or, maybe, it just indicates they prioritize fixing bugs over adding new buggy features.

In addition, the size of patches can also vary quite a lot on what the author considers a "thing". Some developers fix multiple things at once, while some others think every nut and bolt should be it's own separate commit

Yes I believe in save early save often and don't bother squashing and have multiple commits usually in my branch before a merge request. We have another team member that religiously squashes everything down to a single commit on his feature branch before requesting a pull from his branch. We don't have micromanagers so that's just the way things are.

But a reimplementation of exFAT is not as buggy

That's not surprising, because the FAT family is relatively simple for a filesystem. As a result, you can easily get read-only support done almost right the first time, and writing is only slightly more difficult.

This seems like an awful metric. How many of those projects weren't placed on github until they'd stabilized a bit? Are they file systems? How big were the "bug fixes"? How disastrous/critical were the bug fixes? Do they have a unittest suite? What compilers and linters did they use?

I was a little disconcerted at the size of lfs.c, but then again there are advantages to having it all in a single file when compiling for a microcontroller. Although now I wonder why the crc32 code is the only thing in a separate file?

The latest commit message on that file is "Changed lfs_crc to match more common API (...)" so I would assume that they factored out this function specifically so that it could be easily exchanged with another implementation. There are many CRC32 implementations that may be more suitable for the specific microcontroller you are targetting, or maybe you already have an CRC32 implementation in your codebase that you would like to reuse because space is scarce. For the other functions the reusability isn't really given.

As an example, most if not all STM32's have CRC32 hardware as well, so by pulling it out one can use such a hardware implementation.

They could've put an ifdef/#ifndef around it and not have an extra file.

You could do that but if you want to support a lot of different microcontrollers which may all have different implementations, separating the code into a different file makes more sense. Even if you add ifdef statements for every supported controller in there and the file becomes a mess of conditional code, you still have a clean implementation of the file system.

ive used this in an embedded system that had the power ripped out of it constantly (sorta like automotive battery powered) and it has held up for years in the field. the only failure so far was an sdcard finally going out

don't most embedded devices use emmc now? with the wear leveling under the control of the mmc device rather than raw NAND or NOR?

And there's no specification on how that wear-leveling works (or doesn't work).

That is not true. See https://github.com/littlefs-project/littlefs/blob/master/DES...

Sorry, as a commenter on this pointed out, the comment I was replying to was probably talking about eMMC. For which indeed no wear levelling is specified by the JEDEC standard - it is a standard that defines the communication protocol between a device and a host system. Newer JEDEC standards define the interface to access vendor specific "health reports", the actual method of wear levelling isn't regularly disclosed by vendors.

I think the GP was commenting on the lack of specification of eMMC wear leveling, not Littlefs.

Sandisk has multiple documents on this. "They" is a pretty big paintbrush.

You never know if a marginal brand has licensed the Sandisk patents or implemented them correctly.

I don't know about higher level, more expensive stuff, but many popular microcontrollers (like the ESP8266 and ESP32) have SPI flash onboard, without any built-in wear leveling.

You are thinking Linux class devices, this is about MCUs with a few MiB of external flash

This is pretty cool! It looks like a good option for Flash memory.

If anyone is looking for a FAT file system (ie for an SD card or for making a USB mass storage device) fatfs is still a great option.

Just beware that FATFS is very easy to corrupt in the event of power outages - unless it's read-only.

Personally I would be reluctant to use FATFS as a writable filesystem on a commercial embedded device.

Unfortunately LittleFS is very slow on large (multi-GB) storage like SD cards. Also SD cards are managed flash, so you don't need to do wear leveling in the FS.

Yeah, littlefs wear leveling can be turned off for this case. Protection against corruption, and posix like api at small size is still useful.

Uh I didn’t know that - have you run it on an SD card successfully? What options did you use?

You set block_cycles to -1 to disable wear leveling and have the erase callback be a NOP that just returns success. Other geometry parameters can be whatever suits you.

BSD* 3 clause license, which means Freertos kernel 10.0+ only.

The prime majority of SDKs from chip vendors still use much older Freertos ports which were under GPL

RelianceFS is GPL as well

* Typo fix

MIT license is compatible with GPL.

It looks like BSD-3-Clause to me, not MIT?

What about BSD keeps you from including this in linux? It is compatible with GPL2 & 3. There is some BSD code in the kernel and a ton in Linux utilities.

Reply on news.ycombinator.com