Littlefs – a little fail-safe filesystem designed for microcontrollers | Hacker News Reader

Littlefs – a little fail-safe filesystem designed for microcontrollers

github.com

121 pointspabs321 comments

We use littlefs in Flipper Zero's firmware[1] for storage in leftover flash space after the main firmware image. Flipper implements a virtual FS, where both external SD card and internal storage have own mount points. SD card is used for storing apps and user-created data, and internal littlefs contains persistent data like BLE pairing, system services' configs and such.

LFS has neat features like wear leveling and optimizations for storing tiny files directly in their parent directory's data structures[2].

We never had any issues with littlefs - however, it cannot be easily resized when amount of available leftover space changes with firmware updates. So on installing an update, it gets fully backed up to SD card, reformatted and later restored.

[1] - https://github.com/flipperdevices/flipperzero-firmware/blob/...

[2] - https://github.com/littlefs-project/littlefs/blob/master/DES...

A caveat for anyone wanting to use this - while the library does work and the test suite seems thorough, the implementation does some _very_ sketchy things like forcibly casting pointers to unrelated types so they can be shoved through some multiply-recursive functions and then cast back to the correct type later.

A team I was on a while back ended up abandoning LittleFS as we couldn't fully trust the C implementation and my two separate attempts to port it to Rust both proved futile (_everything_ had to be unsafe).

What do you recommend instead?

We ended up going with a very trivial system based on tar files and bitfield allocators and delegated the more filesystem-y like functionality to a higher level of the stack.

I happen to be in the middle of writing a simple tar implementation, and had in fact just started playing with the idea of just using it as an on-disk filesystem format. The caveats I can see are that 1. it's approximately append-only[0], and 2. you have to replay the whole thing in order to make sure you have the latest version of a file (I'm leaning towards doing so on mount because then I can also build an in-memory filename-to-block-address cache). Have you hit any issues, or is this as surprisingly reasonable as it sounds?

[0] I mean, if the new version of a file fits in the same number of 512-byte blocks as the old version you could update in place, but that's an unreliable condition at best and also guarantees data corruption if you lose power halfway; really going append-only makes the implementation simple and also makes it easy to be really resilient.

I'm not on the team anymore so I don't know if they kept the tar stuff around, but it was straightforward to scan the flash for valid tar header blocks to build the allocation bitmap and then flag header blocks as invalid when files were deleted. The total number of files we needed to keep in the flash was small, so simple brute-force stuff worked fine.

Not so sure about the tar part, but the append-only nature is actually really well-known: look up log-structured filesystems, where all operations append to the log and GC comes to clean up stale data later. I’m in the middle of writing a log-structured disk, and it’s really nice to work with. You get many things like clones, snapshots, etc for free.

The replay problem is usually solved with periodic checkpoints of your map, and a checkpoint on shutdown. This way you only ever replay on failure, at which point performance doesn’t matter as much.

Yeah, knowing there were filesystems designed like that on purpose made me feel better about it:) And once you get over the initial weirdness, the ability to get any version of any file is pretty nice. Though my favorite angle is that if you can live with just burning space[0], it makes the thing so easy to implement:)

[0] To be fair, a huge caveat.

> The caveats I can see are that 1. it's approximately append-only[0], and 2. you have to replay the whole thing in order to make sure you have the latest version of a file

Hmm. I'd go and build a separate metadata/index file that you keep updated on each write(=tar-append) operation. Grow the tar from the beginning of the underlying block device, and the metadata from the rear, and have a pointer in the filesystem header that points to the current metadata address.

That way, everything is atomic, you avoid the RAM penalty from having to maintain the metadata mapping, and the only operation that can result in damage in a power-loss event is when the write of the filesystem header messes up - but that can easily be reconstructed by walking both the tar stream and the metadata stream from the beginning to the last element that has a valid header.

I helped build a tar-backed filesystem for MirageOS[0]. It is definitely easiest to make it append-only. I did add in-place renaming, updates (of file content), pre-allocating a file and removing the last file.

We also scan the filesystem on boot and keep a mapping in-memory of file names to file metadata and block offsets, and update this memory representation when modifying the on-disk tar filesystem. If you have a lot of small files this is maybe not a great idea as you will be storing most in memory, though.

For the purpose we had in mind it worked fine: A content addressable mirror of package archives. The archives exist somewhere on the web and the package file has a link to the archive and a checksum. We can then download the package and store it by the checksum. This gives integrity that the tar filesystem does not offer. Removing the last file works great if you only have one download job. Otherwise if a download fails and it is not the latest file you can rename it to `failed/<hash>-<random>` and do garbage collection in the filesystem at a later point.

[0]: https://github.com/mirage/ocaml-tar/blob/main/mirage/tar_mir...

Update: interestingly, this was motivated after trying to use a (mostly) LittleFS-compatible filesystem which unfortunately didn't work very well for us at the time (bugs, poor performance).

I would recommend thinking long and hard if you really need a file system on a microcontroller. What do you intend to use the file system for? If it's only for read-only assets like images or sounds use a linker script to collect them into a sorted set. If you want to write logs a ring buffer is faster, simpler and more resilient to corruption. Configuration settings are small enough and don't too frequently in most systems so a ring buffer or just a pair of double buffered flash pages with a checksum and version counter can be enough.

LittleFS is tries to hit a sweet spot that only exists on MCUs with NOR flash and relatively much RAM and fast CPU cores. The problem is that it's complex compared to the alternatives. The two most common alternatives I can think of are FAT which will wear out flash given half a chance and SPIFFS which is simpler and slower than LittleFS.

FAT is a terrible file system for writing on a MCU, but because of its history as the file system for floppy disks it's still supported by every desktop operating system. This makes it perfect to allow users to edit configuration files stored on a microSD or replace assets like background images without having to use specialised tools.

The annoying answer: it depends.

Great points. I just want to toss out an example of why you might want an OS to use with flash of decent size to do wear leveling on everything. The classic example I give is a media player subject to power interruption and you don't want to include an internal battery in the design for graceful powerdowns, so the player needs to save a checkpoint of play state every ~5 seconds so the device can resume intuitively. Such as indexing back to your position in a 40 hour audiobook. You're also going to have menu navigation and a combination of changing settings and device local media library changes. And now comes along some Internet angle so there are C libraries that implement browser functions and that means cache and cookie writing. And software updates. Ring buffers are smart but there's a whole lot of writing going on here! It would be great to code everything in high levels so if it emulates properly on a desktop, once it's pushed into the devices (say a generous 128GB flash with 28GB reserved for the system) the devices will have a 20 year lifespan with no batteries to fail or leak.

I can tell you take offense to the questionable casting… but did you discover anywhere it made mistakes and failed - or you just didn’t like it?

Undefined behavior will bite you eventually. Updated the compiler? You have no idea what to expect. Changed lines of code related to that part of code? Again, no idea what to expect.

While I agree that undefined behavior should be avoided, it’s not as rigid as you put it. Just because something is undefined behavior by specification (or lack thereof) doesn’t mean it’s undefined behavior in practice. As a general rule though, I do agree with you.

Casting pointers to the correct integer types and again back isn't UB, but several things you can you can easily do along the way are at the very least implementation specific.

What prevents you from taking the underlying data structures and reimplementing the API?

> ...the implementation does some _very_ sketchy things like forcibly casting pointers to unrelated types so they can be shoved through some multiply-recursive functions and then cast back to the correct type later.

I did not look into the code, however such casts can be a reasonable way to implement polymorphism in C.

Of course, having some common type field would be more robust, instead of blindly casting whatever pointer comes at input.

For example, such type could be defined as first field in the 'base' structure. Thus the cast into, say, `int` should yield a valid/expected type value, which would be checked in the called functions before casting into the expected type.

Pointers correctly round-trip through unrelated pointer types, provided the alignment for both types is compatible.

At least one bit does look a bit questionable: lfs_mlist is treated as the common initial sequence of lfs_dir and lfs_file, even though it isn't, and common initial sequences only apply to union fields anyway. Example cast of struct dir * to struct lfs_mlist * (probably valid of itself, assuming the alignment is compatible): https://github.com/littlefs-project/littlefs/blob/c733d9ec57...; then use struct dir as if it were actually a struct lfs_mlist: https://github.com/littlefs-project/littlefs/blob/c733d9ec57...

(There's other occurrences of the same kind of thing.)

Strictly speaking, I think this might be unfixable without a bunch of work, but so much stuff does this kind of operation that any compiler that doesn't do what you expect will have been fixed by now. (Assuming you're not one of those people who is going to pop up and tell us with a straight face that what we should expect is for the compiler to do absolutely anything - except, perhaps, for having it generate correct code, which would be defective behaviour that should be eliminated.) Maybe improve the odds by using lfs_mlist as the common initial sequence of both structs, and fingers crossed that the compiler considers the union rules to apply to this case too. Or compile with -fno-strict-aliasing.

We used littlefs on an embedded project. Everything worked as advertised! Solid.

We kept up with all their releases until their latest API changes forced us to fork to keep our anti-glitching mitigations.

Very cool. FYI, I had to remove some compiler flags and change the 'section' attributes to get the tests building with Clang, but it seems to be working on my Mac.

I was looking for a benchmark that would give me an idea of the RAM/ROM footprint for the library, but I don't see one.

Reply on news.ycombinator.com