Hacker News Reader

Comment by codys | Hacker News Reader

My experience with btrfs is from a few years ago, so it's possible they've changed the semantics here since then, but this might be applicable:

Btrfs requires a regularly run administrative task via "btrfs balance". Balancing consolidates data between partially filled block groups, which will restore consumed space even if a block group is not entirely empty. Typically, one runs balance with a set of gradually increasing parameters to control the records affected (ie: bash script that starts with block groups that are more empty and proceeds towards ones with a higher percentage utilization).

In my experience with btrfs a few years ago, running a balance in some situations would result is incredibly high system wide latency.

The man page [1] does not appear to reference the free space issue directly, so I'm not sure if they've removed this need.

1: https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-balanc...

View on HN

What is the source of "Btrfs requires a regularly run administrative task...".

No file system should require this kind of regular hand holding. If it does, it's a bug, and needs to be reported and fixed, not papered over with user initiated maintenance tasks. It is acceptable to have utilities for optimizing the layout from time to time, but they should be considered optimizations, not requirements. If it becomes required to use them as a work around, that's both suboptimal and neat. But there's still a bug that needs a proper fix.

In the case a workaround is needed, it's recommended to do targeted filtered balances, not a full one. A full balance isn't going to solve a problem that a proper filtered balance can't.

There are whole repos[1] dedicated to scripts to perform the delicate work of running `btrfs balance` in various ways to ensure space is recovered.

openSuse (a distro which notably defaults to btrfs) packages these btrfsmaintenance scripts [2], and it appears they may have included it in their default install (I can't find a list of packages). Their wiki page on disabling btrfsmaintenance [5] implies that if disabled, manual maintenance (presumably via running, among other commands, some form of balance) is needed.

There's also an entry in the btrfs wiki [3] that indicates running `btrfs balance` will recover unused space in some cases. It helpfully notes that prior to "at least 3.14", balance was "sometimes" needed to recover free space in a file-system full state. (The lack of precision here doesn't inspire confidence)

Another btrfs wiki page [4] indicates that running balance may be needed to recover space "after removing lots of files or deleting snapshots".

1: https://github.com/kdave/btrfsmaintenance 2: https://software.opensuse.org/package/btrfsmaintenance 3: https://btrfs.wiki.kernel.org/index.php/FAQ#What_does_.22bal... 4: https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-balanc... 5: https://en.opensuse.org/SDB:Disable_btrfsmaintenance

openSUSE has automatic snapshots with a fairly extensive retention policy.

Kernel 3.14 is ancient. I can't bring myself to worry about it.

I think your fourth paragraph cherry pick is disingenuous. A more complete excerpt, "There’s a special case when the block groups are completely unused, possibly left after removing lots of files or deleting snapshots. Removing empty block groups is automatic since 3.18."

Fedora 33 users will be getting kernel 5.8 from day one, and 5.9 soon after release.

I doubt my case is atypical. I haven't balanced my years old non-test real world used Btrfs file systems.

I think you could switch your concern and criticism to the fact wikis get stale.

> openSUSE has automatic snapshots with a fairly extensive retention policy.

Ah, so Fedora is limiting it's use of snapshots to avoid the need to have balances occur? Do you have some info on what level snapshot usage has to rise to before balances are needed on a regular basis? Is Fedora using snapshots at all?

There is no automatic snapshotting regime on Fedora.

There's no direct correlation between having many snapshots, and needing to balance. The once per month balance used in openSUSE is to preempt or reduce the chances of out of space error in one type of chunk (block group of extents), while there remains significant free space in another type of chunk.

Chunks are created dynamically, and are either type metadata or data (also system but it can be ignored). Different workloads have different data/metadata ratio demands, hence dynamic allocation. Snapshots are almost entirely metadata. More snapshotting means more usage of metadata.

If the pattern dramatically changes, the ratio also changes, and the dynamic allocation can alter course. Except when the disk is fully allocated. In that case, heavy metadata writes will completely fill metadata chunks, and ENOSPC even though there's still unused space in data chunks.

A filtered balance can move extents from one chunk to another, and once a chunk is empty, it can be deallocated. That unallocated space can now be allocated into a different type of chunk, thus avoiding ENOSPC. Or at least when ENOSPC happens, there's essentially no free space in either chunk type, at the same time. A "true" ENOSPC.

There are all kinds of mitigations for the problem in newer kernels. And in my opinion it's better to not paper over problems, but fixing the remaining edge cases.

> Btrfs requires a regularly run administrative task via "btrfs balance"

This hasnt been my experience within the past few years, though I do remember it being somewhat necessary in the past. Both systems I'm running btrfs on have their free space within a few percent of the unallocated space (indicating most used blocks are fairly full and little space is wasted).

Are you using snapshots in btrfs? Perhaps trying to use this feature was what caused it to be needed for my use case.

Interesting.

I would assume that this is not a factor with RAID-1, where all system, metadata, and data is duplicated. I could see this as being very important for the higher RAID levels.

We have since stopped deploying any sort of btrfs RAID (even RAID-1), and have gone back to using Linux MD.

Free space and the space taken up by metadata and snapshots can take a little attention to figure out

https://ohthehugemanatee.org/blog/2019/02/11/btrfs-out-of-sp...

Now that is interesting.

If I could mount the failed btrfs filesystem read-write for an appreciable length of time, I could try the balance operation.

I'll need to set up a cron job on the other systems to run balance on a regular basis alongside scrub (obviously not at the exact same time).

To be clear: I was running a non-raid (single disk) setup where I found `btrfs balance` to be needed.

Reply on news.ycombinator.com