LittleFS - A high-integrity embedded file system

This post was originally published on Mbed Developer Blog.

Having a small and resilient file system is crucial for many IoT devices. But utilizing the file system and pairing it with the correct storage technology such as external flash or SD cards can be difficult. Mbed OS is making it easy to add file system support by providing a wide portfolio of file systems. Mbed OS 5.7 supports both a FAT file system and introduces a new high-integrity embedded file system. This high-integrity file system is small, power-cut resilient and has wear-leveling support for flash chips that do not have their own wear levelling controller.

Storing data on an embedded device is useful: whether it’s configuration files, batches of sensor information or a new firmware update. You can grab some non-volatile memory (such as EEPROM or an SD card) and write this data to random flash pages, but this is error-prone. There’s no overview of the data on flash, no guarantee that you don’t overwrite pages that other data use, and writing to the same flash page over and over again is bad for durability. So, you want a file system (around since 1965) that manages this all for you.

File systems for embedded systems and IoT devices have some additional requirements:

• Power-cut resilience - it requires strong guarantees that the file system remains consistent, and data is flushed to the underlying storage.
• Wear-leveling - typically storage supports a limited number of erases per block, so making use of the entire storage device is important for reliability.
• Tiny footprint - IoT devices are constrained by ROM and RAM. A small footprint saves money.

There are a number of commercial and open-source embedded file systems in the market but none that quite met our design code size, features or reliability requirements that are crucial for successful IoT device deployments. We are releasing High-Integrity Embedded File System, a little fail-safe file system designed for embedded systems. It’s available in early-release form as part of Mbed OS 5.7 and as a standalone C library for non-Mbed systems. It’s licensed under the permissive Apache 2.0 license and available from GitHub here.

High-integrity embedded file system vs. FAT file system

Mbed OS has long supported a FAT file system backed by either an secure digital card or NOR Flash memory. The FAT file system was first introduced in 2010, as an external library and then integrated as part of the core operating system in Mbed OS 5.5. FAT file systems remain an important feature due to its wide support and compatibility with other operating systems ranging from DOS 6 to Mac OS 10.13. In many IoT use cases, there is a need for power loss resilient, data integrity and higher memory lifetime. For many IoT devices, High-Integrity Embedded File System is a better choice compared to traditional FAT file system.

RAM/ROM size

Littlefs, the high-integrity embedded file system in Mbed OS is optimized to work with a limited amount of RAM and ROM. It avoids recursion, limits dynamic memory to configurable buffers and at no point stores an entire storage block in RAM. By focusing on a small set of multipurpose data structures, this high-integrity embedded file system uses 13K less ROM than FAT and 4K less RAM.

Power-loss resilience

We designed this file system for systems that may have random power failures. It has strong copy-on-write guarantees, and storage on disk is always kept in a valid state. In the gif above you can see how quickly FAT file systems get corrupted and how much more resilient this file system is. For more details, please see the program we used for testing.

Wear-leveling

Most storage chips that embedded devices use support a limited set of erases per sector. If you do not have a storage controller with support for wear leveling, the longevity of your embedded device could be compromised. The embedded file system provides dynamic wear leveling to spread data across sectors throughout the full size of the flash. This prevents applications from writing the same sectors too often.The failure mode is a combination of file system use and the amount of data stored in the file system. The wear-leveling algorithm is not perfect, but it has the nice property that expanding the storage size of the flash increases the lifetime of the device.

The gif shows a comparison between the high-integrity embedded file system and FAT file system wearing down a block device with 3K of static data and 3K of dynamic data. To see wear level behavior for different configurations, please see this interactive simulation.

How to start using the High-Integrity Embedded File System?

Any device that implements the BlockDevice interface can host a file system in Mbed OS. You can change the BlockDevice based storage drivers without changing your application or library code.

To start:

1. Choose a driver that implements the BlockDevice API for your flash storage: DataFlash, SD card, SFDP SPI Flash or implement your own.
2. Initialize and mount the file system:

BlockDevice bd = /* get a block device */;

// first argument is the mount point, for example files will be available under /fs/, second a pointer to the block device
LittleFileSystem fs("fs", &bd);

1. Use POSIX file system calls to read and write files (such as fread, fopen and so on).

You can find an example program here. If you don’t have any storage on your development board you can test this high-integrity embedded file system with the HeapBlockDevice, which stores the data in memory (but does not persist it).

Tests

Because a file system is a critical part of a device’s firmware, this high-integrity embedded file system comes with an exhaustive set of unit and integration tests, which the Mbed OS test farm runs daily. To run these test you’ll need mbed CLI:

• Functional tests - mbed test -n 'features-filesystem-littlefs-tests-filesystem-*'.
• Retarget tests - mbed test -n 'features-filesystem-littlefs-tests-filesystem_retarget-*'.
• Wear leveling tests - mbed test -n 'features-filesystem-littlefs-tests-filesystem_recovery-wear_leveling'.
• Simulated power resilience tests - mbed test -n 'features-filesystem-littlefs-tests-filesystem_recovery-resilience'.
• Hardware power resilience tests - mbed test -n 'features-filesystem-littlefs-tests-filesystem_recovery-resilience_functional'.

In addition, we have set up soak tests to verify that the file system properly handles flash exhaustion failure on real hardware.

-

Jan Jongboom is a developer evangelist in IoT at Arm and has seen too many corrupted SD cards. Chris Haster is a software engineer at Arm.