Clean mount lists in Linux | Hacker News Reader

Clean mount lists in Linux

dbohdan.com

189 pointsnetworked69 comments

Somewhat related - `duf` is "a better `df` alternative":

And `dust` as a `du` replacement.

Also `dua`[0] is a great `du` replacement which is must faster on modern NVMe drives. It includes an interactive mode `dua i` which I'd frame as a `ncdu` replacement.

[0] https://github.com/Byron/dua-cli

Started using dust yesterday and like it so far. Also a big fan of diskus by sharkdp as a blazingly fast du -hs alternative

I also like https://github.com/Byron/dua-cli, does disk usage like du but also has an interactive mode which makes finding and freeing up space a breeze.

Even though it's not supposed to be my day job, I've ran `df |grep -v <super long list` hundreds of times in the past year, yet it never occurred to me that I could try and find a better tool.

Preach. This happens to me all the time and keeps happening.

There's a lot of Unix/Linux stuff that I wish I could UNlearn, I'm aware of other tools but muscle memory keeps me using worse things.

(honestly, I also feel this way about vim as well as 'writing in cursive')

I feel you.

I still `cat foo | grep bar` instead of `grep foo bar`

The reason I stick to `cat foo | grep bar` is that I am never quite sure if `grep bar foo` would be correct instead (it is).

The file argument is usually the last so that you can do eg `grep foo -- *`

I remember it as "grep for a needle in a haystack"

But to me -- this is different?

I believe "useless use of cat" is FAR MORE useful than what the ultra-nerds say; I could care less about "saving a few characters."

The mental model of the pipeline --in one end and out the other -- is entirely too usefully intuitive and way better than some imaginary geek-cred.

(I was more thinking about the new rust tools, e.g. rg instead of grep)

Plus sometimes you want to throw something like `pv` in.

Plus the regex is more likely incorrect in the first attempt and needs to be fixed.

The former has one big advantage: I can easily change "needle" without having to skip all over the cmdline...

Pretty usefull when debugging something interactively.

Might help to replace

    cat foo | $command

with

    <foo $command

It's the same order, but I think $command can sniff what kind of file stdin is, and optimize.

Actually instead of `grep bar foo`.

Installing a new tool just to exclude certain parts of the list has much more friction involved than just making a shell alias/function/script though. It's also much more portable.

If it's possible to include (rather than exclude), you can:

    $ df /dev/sda1 /dev/nvme0n1p1

And get only those file systems.

And more importantly, you can specify fs locations, so `df / /home` for example

`df|grep` might be “the better tool” because the cognitive overhead of knowing an extra program’s interface.

You’re just saying “show me the subset of this other list” and you automatically select your go-to solution for that: `|grep`

The reason for seeking a better tool is when too much excessive work (== time) is involved, or list generation has side effects. But in this case you don’t have two million mount points, so no need to worry.

replacement?

why not make df better...

And miss the chance to rewrite it in Rust or Go? :)

Jest aside, it's much easier to write a small tool from scratch, than to contribute to coreutils, where this kind of improvement is likely to be rejected, for various arguably valid reasons.

Besides, having alternatives is healthy for the Linux ecosystem, and writing something from scratch is a good learning experience.

Making a new tool means you can break backwards compatibility.

Better is also subjective.

I've always used ncdu for this. It also allows deleting files/directories for when you're trying to figure out how to free up space.

https://en.wikipedia.org/wiki/Ncdu

For anyone that likes ncdu I would highly recommend gdu. https://github.com/dundee/gdu

Sounds like a Snap/Ubuntu thing. On my Arch system:

  $ df -h | wc -l
  7
  $ lsblk | wc -l
  6
  $ mount | wc -l
  26

(in the article: 25, 56, 77)

Yes, rather than this, I moved to Mint.

I have been using `duf` [1] for a while, as a "cleaner" alternative to `df`.

[1] https://github.com/muesli/duf

something I came to use later in my cli life - using `df .` as in most cases the working directory I'm in is the filesystem I want to get the info on

Sort of related, 'du | sort -n' to figure out what's using the space.

Even better 'du -h | sort -h' for human readable sizes.

Why not just use the designated `findmnt` command from util-linux for the majority of this?

Yea, seems like:

findmnt --options rw --real

... gets you most of the way there.

Thanks. I have added a section about `findmnt` to the page. I'll expand the section later.

Edit: It seems an adequate replacement for `df` and `mount`. Can it print a block device tree organized by device name like `lsblk`? I have skimmed the man page and not found options for it.

This article covers three commands, and the two I use the most--df and lsblk--don't seem easily covered by "findmnt"?

> df -x tmpfs

findmnt -Dt notmpfs

> lsblk -e 7

findmnt -iS 7:0 (excludes mounted loop devices, not functionally equivalent to lsblk)

> mount -l -t btrfs,fat,exfat,ext2,ext4,iso9660,ntfs3,ufs,vfat,xfs,zfs

findmnt -lt btrfs,fat,exfat,ext2,ext4,iso9660,ntfs3,ufs,vfat,xfs,zfs

findmnt -O rw --real

findmnt -lnt noautofs,nobinfmt_misc,nobpf,nocgroup2,noconfigfs,nodebugfs,nodevpts,nodevtmpfs,nofuse,nohugetlbfs,nomqueue,nonfsd,nonsfs,noproc,nopstore,noramfs,norpc_pipefs,nosecurityfs,nosquashfs,nosysfs,notmpfs,notracefs

Or simply invert the target filesystems with --invert instead of prefixing "no".

That counts as the majority for me.

Ok, the df one works, so--assuming the mount ones work (I haven't checked)--I guess you are up to 2/3 and so "the majority": touché ;P. The lsblk one though isn't even close... I'm not sure whether you are trying to cover this with your parenthetical that the functionality isn't the same, but then I don't get why you are trying to slip it in that list.

The basic/key functionality of lsblk is that it shows me all the things that look like disks; which, to me, shouldn't include all of these squashfs mounts, but--very very critically--includes disks that haven't been mounted, as the whole point is that we are looking at block devices, not mount points, or we would be using mount instead of lsblk.

Another important feature of lsblk is how it then takes those block devices and builds them into a tree, showing me how the various layers of--again: potentially as yet unmounted--block devices are "built": I can see that this device isn't in a filesystem, but is being used by an md array which is then used by a bcache or (conversely) that this filesystem isn't just some opaque bcache but is a cache of an md array of some set of specific disks.

The findmnt command you provided thereby simply doesn't seem to do anything related to lsblk; again: you said it wasn't functionally the same... but then what are you trying to show me? I will note your findmnt version also doesn't even succeed in filtering all of the loopback disks, much less all of the random mount trash... 7:0 is just the first /dev/loop, but I have a giant pile of them (due to the stupidly slow and wasteful way Ubuntu insists on using squashfs). I don't think -S even supports something akin to filtering '7:*'?

(FWIW, I do thank you for at least causing me to think about my lsblk usage, as I usually hated how the tree was to-me inverted and I was going to say I'd love to have a tool which didn't do that... but after decades of using lsblk I finally looked at --help and it turns out I can use -s to get the output I actually want, so I am amazingly happy! findmnt definitely isn't a useful alternative, enough.)

The block post is titled "Clean mount lists in Linux" not "Clean block device lists in Linux". I was demonstrating that findmnt can filter the source based on MAJOR:MINOR as a tangential, there was no attempt to "slip" anything in, hence the disclaimer.

You are correctly pointing out that it filters only the first loop device and doesn't support globbing. lsblk is clearly the better suited tool for the job you are describing and the reason I said "majority" in my first comment.

That's the article's title, but the actual content of the article is about: 1) df, 2) lsblk, and 3) mount. And yes: I clearly caught that you said "majority", as my first comment was about how you only seemed to have 1/3... but, now, showing -D for the df functionality (which was you showing me wrong, absolutely), I did agree that you have hit the majority (2/3). Adding the lsblk one continues to confuse me as to the intent.

(FWIW, I frankly just didn't like how you flippantly came in and off-handedly dismissed the article without much explanation and by redirecting people to a Linux-specific tool that has a million options that you apparently still have to configure. I found the article quite valuable as it somehow never occurred to me to try to get these tools to filter these crazy Ubuntu devices, and it felt as if--honestly still kind of does--you were viewing the article extremely narrowly, maybe based on just the title.)

Off topic but the favicon for this website is giving me big Windows 3.1 File Manager nostalgia. Whenever I see a file icon from that era it takes me back. </getting-old>

I am glad someone noticed. :-) It is the File Manager icon. I have always liked this icon and the aesthetic of graphics made for Windows 3.x, even though my first Windows was 95 OSR2. Since File Manager and the icon are now free (https://github.com/microsoft/winfile), there is no copyright reason not to use it verbatim.

If you liked it, you may enjoy my page https://dbohdan.com/gui-games.

Great post, thanks, also 'duf' was new to me, like it.

One question: I have 2 ntfs drives and they show as type "fuseblk". They don't show now with the new "mount" alias. I tried removing "fuse" from the list but no change. How can I bring these back to displaying when I enter "mount"?

dfc may be of interest to some:

https://github.com/rolinh/dfc

https://www.linux-magazine.com/Online/Blogs/Productivity-Sau...

Here we are, one of the primary reasons why I adore the BSDs. Linux seems to be just neverending layers of indirection, abstraction, and a kitchen sink of features that less than 1% of its users actually need.

The full command to get a "clean" mount list on OpenBSD, including disk usage with human-friendly units, is:

    $ df -h

https://man.openbsd.org/df

Well, if you're seasoned enough, you start to read df output from right to left.

Find the mountpoint you're looking for, read to left to see the stats. When you're managing a storage box, no amount of tmpfs entries can create the noise stated in the blog post.

P.S.: If you know the device names like your name, you can quickly skim that list, too.

I prefer, in this order:

1. For the computing device to do the more reasonable thing;

2. For the computing device to adapt to and serve my needs, not the other way around.

You want the mount points first? In school we were taught to think in what then where order. That's how the df -h command works on Linux, the way most people think. When you want to eat, most of us think what do I want to eat, then where.

I think Linux is doing the more reasonable thing. The features are implemented with the existing facilities and extended to the bare minimum as required.

On top of that the commands are adaptable, and you can always write the tools you need. There are no hidden/forbidden interfaces.

I commend BSDs for what they are, but blindly praising any OS is wrong IMHO. Is Linux perfect? Heck no. Is BSDs are squarely better at everything, again no.

These are different OSes shine at different things. We use all of them, but they are not that interchangeable.

My praise is far from blind; OpenBSD lacks a modern filesystem, FreeBSD still uses /usr/local/etc for some configuration, and all of them suffer to a various degree from simply not being x86 Linux with glibc and systemd. What you do get in return though is some sanity. When I type "df -h", what I mean is: "show me the remaining free space on the internal disks and attached media". It's not a big ask for that information to not be littered with half a dozen lines of useless noise?

You have a point, but I don't think you have the point you think you have. It seems like df(1) as a tool should be about showing you "free space, on things where the amount of free space might ever be a concern." (Otherwise you'd just use mount(1)!) So df(1) itself, should be filtering the mounts list for only devices capable of having free space.

That is exactly my point. But my other point is that keeping things simple (what is /dev/shm? I thought /dev is already a synthetic FS?) helps avoid the problem in the first place.

> what is /dev/shm? I thought /dev is already a synthetic FS?

/dev/shm is a tmpfs mount. It's not synthetic. It's basically "/tmp but it uses your RAM so it's not essentially-unbounded, so we didn't put it on /tmp because then stupid legacy programs might OOM the kernel."

It's mounted under /dev presumably because you can think of "a tmpfs mount" as a system facility for "named shared-memory allocations" — in the same way that e.g. /dev/audio is a system facility, or /dev/random is a system facility. It's a facility that presents itself as a virtual filesystem, but it's still fundamentally a "low level system API" meant for consumption by the systems programmers writing e.g. glibc, rather than something the user is meant to see and browse into as part of the filesystem abstraction. (I believe that /dev/mapper is another thing that ended up under /dev for this reason.)

Also, /dev was a synthetic FS for a little while — devfs — but this long post-dates the creation and conventional-ization of these other mountpoints inside /dev. Until 2009, /dev was just a directory in the rootfs, with special-device inodes created manually during rootfs creation with mknod(8).

These days, under udev, /dev is once again "just a directory" — just one with a daemon that automatically creates and destroys the special-device inodes in it to match what's actually available on the system.

Thank you for the explanation. The implementation details and the history are both interesting, but again - completely irrelevant to the question of "am I running out of disk space".

Having thought of your suggestion again (df should hide irrelevant mounts), I think it would be unfortunate. I might have legitimate reasons to check on my tmpfs mounts (e.g. as you suggested, a stupid program dumping cruft into /tmp). Changing df also doesn't help clean up my Grafana dashboard.

Again, I think the root cause of the problem lies somewhere between exposing implementation details to the user, and layering on more uninvited complexity. Between a bunch of Linux boxes I have at hand, /run, /run/lock, /run/wrappers, /run/user/1000 (and /run/user/1001, 1002...), are all separate mount points. If /run is already a tmpfs mount point, why does /run/lock have to be one as well? The mount flags are pretty much identical. If uid 1000 shouldn't be able to deny uid 1001 the ability to write to their XDG_RUNTIME_DIR, why not implement some form of quotas?

I'm sure there are good answers to all of these questions, but my main point still stands - even when this information is relevant, it's being drowned in the noise.

> I might have legitimate reasons to check on my tmpfs mounts

Yeah, but the output of df(8) re: tmpfs is meaningless, as 1. every tmpfs mount is considered to each have as much free space as there is RAM in the system (total RAM, not free RAM!); and 2. the "used space" stat will track unlinked-but-open files, which for a tmpfs shouldn't be thought of as even being "in" the tmpfs at that point, but rather just being reserved IPC memory held by one or between several processes.

If you want to know the disk usage of a tmpfs dir, use du(1). That's what it's for: seeing what the on-disk space taken by the tree of inodes in a directory or mountpoint aggregates up to.

df(1) is an abbreviation for "disk free" for a reason — the output of df(1) only makes sense insofar as there's a bounded, independent, reserved pool of "free space" of a resource that you want to measure and manage.

Since df(1) used to exist in an environment where the only mount points were of such resources, df(1) never previously had to do any filtering of the mounts table to achieve its stated job. But the "spirit of its semantics" would today imply filtering.

> If /run is already a tmpfs mount point, why does /run/lock have to be one as well?

I believe this is a long-term deprecation caught in mid-transition — in theory, /run itself doesn't actually need to be a tmpfs any more; according to the FHS and the newer versions of XDG, all the tmpfs-es should be subdirectories of /run. Once that actually happens, /run would just become a regular directory, like /dev is.

> If uid 1000 shouldn't be able to deny uid 1001 the ability to write to their XDG_RUNTIME_DIR, why not implement some form of quotas?

I believe these dirs are actually done this way for efficiency, not security: one magical thing about a tmpfs mount is that it somewhat acts like a memory arena for the allocations within it; so unmounting it will batch free all those memory allocations in a very efficient manner. (Think: the time it takes to `rm -rf` a million tiny files in a scratch partition, vs. just reformatting said partition.) /run/user/... is created at user login; users expect logout [i.e. session refcount dropping to 0] — and/or graceful shutdown — to not hang on unlinking a million tiny files; and having /run/user/... be a separate tmpfs that gets unmounted on logout, helps with that.

Separately, I believe "application users" for containerized services get /run/user/... mounts created for them, within their separate mount namespace (and this is the only way that that can work); having non-containerized users also use mountpoints here allows code reuse, rather than two mostly-redundant and potentially-buggy codepaths.

> even when this information is relevant, it's being drowned in the noise

Yes, I agree. My point is more that only certain filesystem mounts are actually mounts of bounded-size writable disks. And that it's only mounts of bounded-size writable† disks that a system administrator would be concerned about "managing" by asking the question "how much of this disk is free?"

(† One example you didn't mention: read-only squashfs images, used for e.g. Ubuntu snaps — which show up under df(1) as always-100%-utilized mounts.)

> 1. every tmpfs mount is considered to each have as much free space as there is RAM in the system (total RAM, not free RAM!)

This is wrong as can be seen from the output of `df -h -t tmpfs` on a Linux system:

    df -t tmpfs -h
    Filesystem      Size  Used Avail Use% Mounted on
    tmpfs            22G  168K   22G   1% /dev/shm
    tmpfs           8,6G  2,2M  8,6G   1% /run
    tmpfs           5,0M     0  5,0M   0% /run/lock
    tmpfs           4,3G  464K  4,3G   1% /run/user/1000

Every tmpfs has a different size / available space.

> 2. the "used space" stat will track unlinked-but-open files, which for a tmpfs shouldn't be thought of as even being "in" the tmpfs at that point

This is also wrong as they still count against the space the given tmpfs might use. That is relevant information when something reports ENOSPACE.

> If you want to know the disk usage of a tmpfs dir, use du(1).

This number is usually irrelevant.

> > If /run is already a tmpfs mount point, why does /run/lock have to be one as well? > > I believe this is a long-term deprecation caught in mid-transition

This is wrong too. /run/lock is a separate tmpfs as /run/lock is world-writable, but /run is not. It's basically to prevent denial of service by using the world-writable /run/lock to fill the /run tmpfs. Which separate tmpfs for /run and /run/lock this is no longer possible for all users.

This is also why /run/user/X is a separate tmpfs.

>/dev/shm is a tmpfs mount. It's not synthetic. It's basically "/tmp but it uses your RAM so it's not essentially-unbounded

/dev/shm is for the shared-memory facility. It's not used as a temporary filesystem by anything (possibly excepting the user). FreeBSD's shm implementation actually does support the read/write interface, but it doesn't expose a filesystem.

>, so we didn't put it on /tmp because then stupid legacy programs might OOM the kernel."

>Yeah, but the output of df(8) re: tmpfs is meaningless,

/tmp is tmpfs on Fedora and other distros. The size parameter on mount can limit available 'storage' so the free 'space' df output can be useful.

That command works fine on Linux, what am I missing?

The fine article.

The article is useless. What am I missing, I'll ask again.

The article describes the steps necessary to get the short and relevant answer: how much free space is there left on my physically attached disk devices? On Linux, the answer you're actually looking for is hidden among half a dozen tmpfs/devtmpfs/udevfs/whateverfs "synthetic filesystem" mounts; the situation is even more dire when you type "mount" which will list another couple dozen, with the relevant information (such as, which external media is mounted?) hidden in that noise.

The BSDs, rather than drowning the user in noise, and creating the problem that the article tries to address, provide just the relevant information.

On FreeBSD `du` also doesn't necessarily go e the right info due to us, where having multiple filesystems on the same pool is easy and simple. This leads to many filesystems reporting same free space. Right way there is to check the zpool directly ...

That's 100% a ZFS problem, not a FreeBSD problem ;) I have the same issue on my Ubuntu and NixOS systems. Well that's the price you have to pay for ZFS, it doesn't exactly fit in a traditional model - but the space reported by "du" is still accurate e.g. wrt the remaining quota.

True, but given the role of ZFS in FreeBSD this hits most FreeBSD installs, thus the claim of any BSD solving that isn't true.

OpenBSD solves that by limiting their features.

Valid content in your criticism is missing.

In Linux is no longer:

    df -h

but

    df -h | egrep SOMETHING

or maybe

    df -h | less

and yes I still don't understand why dozens of mount points reported.

To be fair, FreeBSD with ZFS (especially with the poudriere pkg build system, and 2 different linux compat environs) has quite a few (40+) filesystems mounted. See my workstation: https://dpaste.org/bnwdt

It's down to the distribution. If you roll without all the systemd and canonical nonsense, then you get a pretty clean system that's on par with the BSDs.

I'm OK with how the disk tools show stats on Linux, thank you.

Reply on news.ycombinator.com