Comment by codys | Hacker News Reader

Comment by codys | Hacker News Reader

Parent Full thread

There are whole repos[1] dedicated to scripts to perform the delicate work of running `btrfs balance` in various ways to ensure space is recovered.

openSuse (a distro which notably defaults to btrfs) packages these btrfsmaintenance scripts [2], and it appears they may have included it in their default install (I can't find a list of packages). Their wiki page on disabling btrfsmaintenance [5] implies that if disabled, manual maintenance (presumably via running, among other commands, some form of balance) is needed.

There's also an entry in the btrfs wiki [3] that indicates running `btrfs balance` will recover unused space in some cases. It helpfully notes that prior to "at least 3.14", balance was "sometimes" needed to recover free space in a file-system full state. (The lack of precision here doesn't inspire confidence)

Another btrfs wiki page [4] indicates that running balance may be needed to recover space "after removing lots of files or deleting snapshots".

1: https://github.com/kdave/btrfsmaintenance 2: https://software.opensuse.org/package/btrfsmaintenance 3: https://btrfs.wiki.kernel.org/index.php/FAQ#What_does_.22bal... 4: https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-balanc... 5: https://en.opensuse.org/SDB:Disable_btrfsmaintenance

openSUSE has automatic snapshots with a fairly extensive retention policy.

Kernel 3.14 is ancient. I can't bring myself to worry about it.

I think your fourth paragraph cherry pick is disingenuous. A more complete excerpt, "There’s a special case when the block groups are completely unused, possibly left after removing lots of files or deleting snapshots. Removing empty block groups is automatic since 3.18."

Fedora 33 users will be getting kernel 5.8 from day one, and 5.9 soon after release.

I doubt my case is atypical. I haven't balanced my years old non-test real world used Btrfs file systems.

I think you could switch your concern and criticism to the fact wikis get stale.

> openSUSE has automatic snapshots with a fairly extensive retention policy.

Ah, so Fedora is limiting it's use of snapshots to avoid the need to have balances occur? Do you have some info on what level snapshot usage has to rise to before balances are needed on a regular basis? Is Fedora using snapshots at all?

There is no automatic snapshotting regime on Fedora.

There's no direct correlation between having many snapshots, and needing to balance. The once per month balance used in openSUSE is to preempt or reduce the chances of out of space error in one type of chunk (block group of extents), while there remains significant free space in another type of chunk.

Chunks are created dynamically, and are either type metadata or data (also system but it can be ignored). Different workloads have different data/metadata ratio demands, hence dynamic allocation. Snapshots are almost entirely metadata. More snapshotting means more usage of metadata.

If the pattern dramatically changes, the ratio also changes, and the dynamic allocation can alter course. Except when the disk is fully allocated. In that case, heavy metadata writes will completely fill metadata chunks, and ENOSPC even though there's still unused space in data chunks.

A filtered balance can move extents from one chunk to another, and once a chunk is empty, it can be deallocated. That unallocated space can now be allocated into a different type of chunk, thus avoiding ENOSPC. Or at least when ENOSPC happens, there's essentially no free space in either chunk type, at the same time. A "true" ENOSPC.

There are all kinds of mitigations for the problem in newer kernels. And in my opinion it's better to not paper over problems, but fixing the remaining edge cases.

Reply on news.ycombinator.com