A little fail-safe filesystem designed for microcontrollers (2017) | Hacker News Reader

A little fail-safe filesystem designed for microcontrollers (2017)

github.com

189 pointsopless30 comments

i am very impressed that, although it's designed for the mbed ecosystem, this code is obviously written so that it can be used in any embedded project (especially those that use a single, statically-compiled binary for the entire firmware) with virtually no changes. i could have this working on my project's bespoke hardware in a couple hours.

* zero dependencies, no assumptions about libc, doesn't use timers or heap, fully synchronous, and it's passive when you're not interacting with it.

* variables, constants, and functions are well-prefixed (without being too verbose) to avoid collisions.

* BSD license is friendly for single-binary firmwares.

* the Makefile doesn't assume anything, it just outputs a .a file which you can link however you want.

* the Makefile is simple and doesn't require cmake or autotools. it's easy to point it at your cross-compile toolchain.

it has other hallmarks of a well-planned and well-maintained project too:

* the readme shows exactly why i might want this (or might not), and how to use it. the separate design doc shows exactly how it works.

* functionality is cleanly separated by file.

* consistent style (could specify which style guide they're using, though).

* the tests serve their usual purpose but also serve as examples. they're well encapsulated and the code was obviously written to be tested.

these authors have absolutely nailed everything i strive for at my embedded firmware development job.

I had the same reaction. Really impressive and well documented. I will surely do some tests.

> variables, constants, and functions are well-prefixed (without being too verbose) to avoid collisions.

Unfortunately it has the same name (Little File System, LFS) and the same function prototypes (lfs_*) of a filesystem for NOR/NAND based systems I did like 10 years ago and still in use, so it will be at least confusing.

As a matter of fact, I guess every embedded technology company using NAND/NOR/whatever memories have their own version of a LFS.

You're referring to LFS as in LogFS?

I didn't read all of the design, but it looks like the author had an unusually clear understanding of the tradeoffs involved and how they relate to the specific target environment. There are some clever solutions there, all explained quite well. Definitely worth a look.

I think you can be sure this is not his first attempt at a file system.

Documentation does look really good. I’ll hang in to this.

I am curious - does anyone have some feedback from using the mbed ecosystem? I have generally shied away from using too much vendor middleware but it does look like there are a lot of useful libraries.

I've uzed the original mbed NXP LPC1768 several years ago, and didn't do too much with the vendor ecosystem, but it was very easy to just use the regular ARM-GCC compiler and extract the functionality I needed from their libraries into my own projects.

I would also love to hear replies to this. I used mbed in the past and have been fairly dissatisfied with it for two reasons:

It uses c++ for the sake of using c++ instead of leveraging the extreme bonuses c++ provides over using c for embedded. For example, it tends to use dynamic memory allocation heavily, doesn't make use of unique/shared pointer, doesn't use templates properly (if at all) in places it would be ideal (hence increasing code size and slowing things down).

Last I checked, it doesn't support cmake at all, and instead used an extremely complicated and conveluted build system/process.

Both of those combined made me run away and never look back. Another side thing as of late is it seems to be officially supported by arm, which means it will never run on other cores like risc-v. I am not interested in getting stuck with one platform when riscv (in my opinion) has incredible potential for microcontrollers and is seemingly right around the corner.

Your criticism seems a bit idiosyncratic, to put it kindly. So it doesn't use the flavor of C++ you prefer? Sure, but there's a lot of room for argument about whether shared_ptr or template-instantiation bloat would be good choices for an IoT-level embedded system. Similarly, the fact that it doesn't use cmake seems like more a matter of preference than of actual suitability. As for being supported by ARM, well hey, at least it's open source. It could theoretically be forked for RISC-V. Lacking any mention of alternatives, that criticism also seems rather hollow. Beggars can't be choosers, y'know.

BTW, you maybe should have mentioned that you work on a competing project.

Dynamic memory allocation on an embedded system is pretty bad. Even in desktop applications you should usually try and avoid that.

Over complicating the build system is a reason for dropping almost anything but especially this. The whole point of a framework like that is to make things easier.

> Dynamic memory allocation on an embedded system is pretty bad.

That's the one criticism I found valid. I even called it one of the "Four Horsemen of Poor Performance" in a pretty widely read article I wrote on server performance back in 2002. On the other hand, over-reliance on static allocation means allocating many arrays/pools for the worst case, even when they couldn't possibly all hit worst case at the same time, and that can be a pretty bad choice on a highly memory-constrained device. (Doing it for the sake of real-time predictability is actually a different issue.)

OTOH complaining about not using templates because of performance seems exactly backwards, and complaining that it's not likely the the project sponsor will port already-open-source code to a competing architecture themselves seems a bit too entitled. When I see a list of negatives that are mostly bogus, and no mention of positives at all, it's a strong indicator of NIH syndrome.

(replying to parent as well).

I have not used it but zephyr (https://www.zephyrproject.org) is a cross platform (including riscv) os/platform that looks interesting for embedded development.

The EULA for mbed says "ensure that they are licensed for use only as part of Software Applications and only on microprocessors manufactured or simulated under licence from Arm" https://os.mbed.com/eula/

It is confusing though as the code mostly appears to be Apache, not sure if it is just parts that are constrained by the EULA, but it is clearly not intended to be ported to RISC-V or MIPS or whatever.

That EULA is specifically for the ARM RealView compilation tools, which are proprietary AFAIK.

You don't need any EULA to use any of Mbed's stuff, it's all Apache licensed and can use any compiler.

There is a difference between mbedOS and the support libraries which can all be used independently of their monolithic system. Modern C++ is not supported by some commercial compilers so you get lowest common denominator in that case but that is irrelevant to the core libs all implemented in C.

I'm a huge fan of mbed OS. The build system adapts to any environment, and you can "eject" if you want to tweak a specific build style. I personally use VS Code with the "mbed compile" workflow.

The official libraries are well-documented and easy to use. I've had to fight a couple vendor-specific oddities when adapting third-party components from their online community, but everything else seem to work as expected.

I worked with mbedTLS on a desktop/mobile project a couple of years ago. I'd previously used OpenSSL, and I wasn't very impressed, but mbedTLS seemed OK: easy to get it building with cmake, easy to get it building on Windows/OS X/iOS/Linux, no actual I/O in the library.

I remember finding integration with libuv a mite annoying, as mbedTLS is rather pull-minded, and I didn't find the documentation for the I/O callbacks super-clear. If your I/O callbacks are making blocking calls of a pipe or socket, though, I'm sure it's much simpler.

Would use again.

(Would not use OpenSSL again.)

> (Would not use OpenSSL again.)

Reminds me of someone using this gif to describe working with OpenSSL.

https://imgur.com/gallery/ikBZF82

mbedTLS is mostly nice but it does suffer from some poor coding practices like large buffers allocated on the stack.

I would warn anyone to stay away from Cypress/Broadcom's WICED OS which has a modified version of mbedTLS with foolish changes to the API that makes porting code extremely challenging.

I have a tiny bit of experience with it. I found it hard to keep track of which libraries worked with each other or with my particular microcontroller (particularly anything involving networking), but it was probably just my own ignorance.

My experience with it was all the packages for the platform I wanted to use were broken.

mbed is great for prototyping. It's basically the Arduino system you would have gotten if Arduino was developed by people who mostly already know what they are doing instead of learning as they go.

Lots of libraries for various sensors and other applications, so if you do early stage development on hardware you can quickly throw prototypes together and not need to deal with sorting through a lot of mediocre Arduino stuff.

YMMV on keeping it for production use. Some parts of it should be fine for some applications. Other things not. I've run into issues with things like threads and timers not always working right for every library, but really that's par for the course on a system of libraries that diverse, and supporting so many different chips.

yes. i found a bug in the gcc export along with a dozen others. mbed devs hemmed and hawed about it for 6 months until they convinced themselves it's not a bug. I dropped mbed after that, switched that project over to Arduino.

Nice, perhaps a more robust fatfs replacement?

If you've used this (or are the author) how is the "storage on disk ... always kept in a valid state" feature implemented? Does it write two copies in case a write's interrupted so one can always roll back to the earlier valid copy?

https://github.com/ARMmbed/littlefs/blob/master/DESIGN.md answers those questions in admirable detail.

Thank you. Makes me wish everything else in the world was so thoughtfully documented.

One problem I see is when they write the pairs of metadata blocks for redundancy, in many FTL implementations they might fall on the same physical page of flash so there is not much redundancy. When FTL write metadata for their own needs, they are congnizant of the flash layout and make sure to write metadata to separate die/block/page.

Seems well thought through, but the concept of putting a general-purpose file system on an embedded microcontroller is a common mistake.

These are single-purpose systems, you don't need to use filenames and have POSIX semantics, you typically just need to put your data into a circular buffer on FLASH. The circular buffer (with checksummed elements) will take care of bad-block management and wear levelling for you.

I've seen this far too often, filesystems are not ideal for this class of system.

i have the exact opposite opinion. such circular buffers only suit limited file system activities, such as logs. if you want editable config files of varying lengths, you need a real fs. i'll concede that features like directory structures and timestamps can be omitted though.

No, you can do log-structured BLOBs which essentially given you version control in a simple manner with circular buffers.

As for variable-length configs, yes, but you should always be allocating the worst-case (i.e. defined maximum), hence no need for variable length.

Reply on news.ycombinator.com