Ncdu – NCurses Disk Usage
dev.yorhel.nl
dev.yorhel.nl
`rmlint` and `jdupes` are also seeing a lot of use here lately, reclaiming terabytes of space from years of sloppy media organization (or the lack thereof!)
Why can’t either of these systems do what the Mac has been able to do since the 90s, and display the recursive size of a directory in bytes in the file manager, allowing one to sort directories by recursive size?
I am not exaggerating to say this is the single biggest roadblock to my permanent migration to Linux!
(I would love nothing more than to hear I’m wrong and “you fool, Dolphin can do that with flag: foo”!)
use a cronjob for `duc index`, then you can use `duc ui` to see the index. it doesn’t immediately update on change so it’s not quite what you’re looking for, but it might be the closest thing.
I made jsdu to get progressive (and recursive) size.
I mostly only use jsdu on a few top levels directories, and use ncdu for the rest or after the stats is cached by jsdu.
You can install jsdu with "sudo npm i -g jsdu" or run it without install with "npx jsdu"
Windows will tell you the size of a dir in the right click -> properties menu, but it takes a while to calculate for large/complicated directories.
Caja (and probably Nautilus/other-Nautilus-based managers) does that as well. But although can show it in properties arranging by size doesn't take it in consideration. (Rather it just sorts them by number of items inside.)
This broke some time in the past (KDE really jumped the shark) and is now available as stand-alone applications only: k4dirstat and filelight. The MIME type inode/directory is already associated with those, so you can run them from the context menu of a directory anywhere, including file managers.
Many file managers can do that, although for obvious reasons it's rather built as a contextual action on a single directory than an always on feature than would slow down the filesystem horribly by accessing it recursively on many levels. On Thunar (XFCE's file manager) for example it's accessible from the contextual menu opened using the right mouse button on a directory name; other file managers would work in a similar way.
I'm sure filesystems could be modified so that any write would automatically update a field referred by the containing directory, so it would quickly propagate to the upper level, but that would imply many more write accesses which for example on SSD media would do more harm than good.
I would like something that starts estimating sizes immediately, and then refines those estimations as it is able to spend more time. I tried writing it myself, but I ended up not quite knowing how to go about it, because just getting the count of files in a directory takes about as long as getting the total size of said files...
(Another problem is that file sizes are notoriously fat tailed, so any estimation based on a subset of files is likely to underestimate the true size. Maybe by looking at how the estimation grows with more data one can infer something about the tail exponent and use that to de-bias the estimation?)
Seems to be specific for btrfs
Painpoints are that there seems to be no progress bar in interactive mode, the ui is (imho) ugly/unintuitive (for instance the usage bar seems to be relative? and the shortcuts look like glyphs) and there are functions missing (like exclude patterns, you can exclude dirs though!).
So it won't replace ncdu, but if it get a interactive progressbar maybe it will be on all my machines (with arch)
> btdu is a sampling disk usage profiler […] Pick a random point on the disk then find what is located at that point […] btdu starts showing results instantly. Though wildly inaccurate at first, they become progressively more accurate the longer btdu is allowed to run.
There are so many now that are better than the old stuff; I almost feel like a unified round up of these, maybe even in a distro form, might be good for linux enthusiasts, newcomers, old-timers, etc.
> A collection of modern/faster/saner alternatives to common unix commands.
Xonsh instead of bash (because you already know Python, why learn a new horrible language?)
bat instead of cat (syntax highlights and other nice things)
exa instead of ls (just nicer)
neovim instead of vim (just better)
helix instead of neovim (just tested it, seems promising though)
nix instead of your normal package manager (it works on Mac, and essentially every Linux dist. And it's got superpowers with devshells and home-manager to bring your configuration with you everywhere)
rmtrash instead of rm (because you haven't configured btrfs snapshots yet)
starship instead of your current prompt (is fast and displays a lot of useful information in a compact way, very customizable)
mcfly instead of your current ctrl+r (search history in a nice ncurses tui)
dogdns instead of dig (nicer colors, doesn't display useless information)
amp, kakoune (more alternative text editors)
ripgrep instead of grep (it's just better yo)
htop instead of top (displays stuff nicer)
gitui/lazygit instead of git cli (at least for staging, nice with file, hunk and line staging when you have ADHD)
gron + ripgrep instead of jq when searching through JSON in the shell (so much easier)
keychain instead of ssh-agent (better cli imo)
Wrote this on the train with my phone by checking https://github.com/Lillecarl/nixos/blob/master/common/defaul... for which packages I have installed myself :)
lsd instead of exa (better formatting, icons)
mosh instead of ssh for interactive sessions (maintains session even with bad connectivity)
hyprland instead of sway instead of i3 instead of XMonad
The first installation method it shows for Linux systems is download a statically compiled binary and it already exists on the repos of every major distro. Where the only uses snap comes from?
Btop seems to only support Macos, Linux and FreeBSD.
Exactly, one horrible language is enough!
There's also more detailed CPU usage you can turn on, including I/O wait.
Use it, it's great.
The old tools have been there forever and used everywhere. My assumption would be these are safe and don't change often. For better or for worse, I would be concerned about using the newer tools unless they are backed and/or approved by a large open source org.
Basically, I don't do "mission critical" Linux things. I teach IT and I hack around on my own boxes with scripts and stuff because it's fun and useful to me. I'm always on the lookout for the hooks and such that can get more and different people into Linux.
I use 1 daily driver for everything, including my finances and crypto. So trust me, I'm bummed out about this. I'll still check some of these out. As long as they seem safe, I will install a few.
Go take a trip through the GNU userland tools and you'll find a lot of dodgy code that hasn't been touched in 30+ years.
find . -type f -print0 | xargs -0 ls -s | sort -n
If your find/xargs don't support null delimiters, then the guy who wrote musl has some tricks:Not exactly what you had in mind but might still be interesting.
[0] https://jvns.ca/blog/2022/04/12/a-list-of-new-ish--command-l...
$ sudo apt install ncdu
> Not enough free space
Fuck!Actually, one trick I learned was to create a dummy file full of zeroes or rand, maybe 200MB or so, and place it in an obvious location (e.g /) for easy deleting in such a crisis.
[0] https://docs.cloudera.com/cloudera-manager/7.4.2/managing-cl...
Doesn't help if the disk is filled by processes running as root, of course.
Then you can delete that file and ponder your life choices that led to that situation.
Just remember to create it again :)
I always remember that VMware vCenter appliance didn't come with a working logrotate config for years and feel better.
Using GrandPerspective on macOS was a similar revelation, but ncdu being keyboard driven and allowing you to quickly launch a shell inside each folder and then apply find -exec there quickly is a productivity boost on yet another level.
On a Mac, I've been using OmniDiskSweeper for years, but this can be run in a single directory and on my Linux machines as well. Fantastic!
And look, it didn't ask me to sign up for an account, and it didn't require me to consent to usage data collection with a huge Privacy Policy attached! How is that possible? (still dealing with this morning's scars from trying, and failing to run the warp terminal)
(I should probably look into this...)
Be weary on machines that have not been restarted I'm a while. The free space reported on the file system layer may not be the same as the amount of space taken up by files.
i.e. It's possible that delete files may still be referenced by processes still running since the space will not be reclaimed until they are killed or close the fd. This commonly happens with GB of log files you deleted but at still `open()`.
That seems to be the greatest difference between Windows and Linux from an OS & file system perspective. - Windows treats files similar to a row in adl database with locking behavior: "This file entry must be deleted from the system. Tell me if I'd be locked out by another process". - Linux treats files as reference counted resources: "I no longer need this file in this directory. The underlying data can still be referenced. You can unlink it."
ncdu -x to instruct it not to traverse foreign filesystems
I had to delete a lot of stuff so I turned off the 'Are you sure?' prompt.
Sure enough, 4 minutes later I fat fingered some wanted files into oblivion.
I’ve also used windirstat (I think that’s what it was called) years ago on Windows and Disk Inventory X on MacOS graphically.
This seems like a nice in between that could replace both methods for me. I’ll have to try.
The GUI bunch of squares is really only useful when you can hover with a mouse and would be clunky in a TUI even if you could render it. And the FS structure isn’t immediately obvious, so I find myself wasting time wandering hovering over big files and globs of little ones.
I really like to avoid a mouse when I can unless it’s really useful.
However I use ncdu all the time.
On Windows, the new hotness is WizTree, which rather than recursively calling directory listing functions, it directly reads and parses the file tables itself. This makes it orders of magnitude faster. I have a 2 TB hard drive full of a million files, and WizTree reads and parses it all in under a minute, whereas I can expect WinDirStat to take half an hour.
My windows experience is really out of date. I knew NT4 and 2000 the best. Then I picked up 2008 for a while supporting small businesses. I don't hate it or anything, but am definitely deeper on Unixes and about equal on VMS that I supported as my NT4/2000 time. I'll work on whatever pays :).
The only thing I wish it had was a squarified treemap view of disk space. There was an old graphical tool for Windows called SequoiaView that I used to use years ago, and I've never found a worthy replacement for it on Linux or MacOS.
On an SSD, WizTree only takes a couple seconds.
On an SSD, WizTree only takes a couple seconds.
I type `du -sk *|sort -n` a lot less frequently now.
Edit, to spell it out: op was surely joking
Another tool I've used is jdiskreport[1]. It's a java app with a straightforward GUI. File system scanning is multi-threaded and quite efficient.
xdiskusage -qa /
I've used it on everything from Windows 10 to my Win98SE oldschool gaming box.
It's abandoned but it still works, and I like it because you can pipe the output of du into it, which is useful for visualizing remote systems.
Another disk scanner worth plugging that I came across for some use cases where I needed to generate single-view reports is pdu - it has the same concurrency implementation that other ncdu alternatives use so the performance is much better too.
find / -type f -mtime +200 -exec rm -f {} \; du -hs * | sort -hdu will have to rescan as you traverse down the tree
du -ax / | xdu -n -c 9