A minimum complete tutorial of Linux ext4 file system (2017) | Hacker News Reader

A minimum complete tutorial of Linux ext4 file system (2017)

metebalci.com

169 pointsgiis29 comments

Open articleView on HN

Awhile back I put together a visualization of ext4.

https://buredoranna.github.io/linux/ext4/2020/01/09/ext4-viz...

This is fantastic.

Thanks for saying so :)

Agree, it's awesome.

It would be good to see example animated GIFs of current Windows-11-formatted FAT32 and NTFS volumes on the same-sized partitions.

Maybe even comparison to recent but unsupported OS like Windows 10 version from 2015, Windows 8, or Windows 7.

And see how closely the current or former mkfs in Linux compares when it is employed to create FAT32 or NTFS volumes on the same size partitions.

thats very cool. i feel like a long time ago we used to be able to cat /dev/hda1 > /dev/video ? but maybe im imagining things. i know you could do it with audio.

Yeah, it would work with /dev/dsp, the primary interface for audio IO in Open Sound System (OSS) -- the first media API for nx operating systems. Around the turn of the century it was replaced in the Linux kernel by the Advanced Linux Sound Architecture (ALSA). Which, to this day, is still the "lowest level" high performance way to do audio w/ linux.

As for _visualizing_ the audio as video, no, that was never possible w/o writing code. =) And to do it _well_, you need to do some math as well. I say this because I wrote that code in 2001, 2003, then again in 2008, then again later... it's a ... hobby of mine :)

/dev/fb0, yes you could do that if you configured your system to use framebuffers.

Thanks. I really do miss the disk visualizations of the DOS and Norton Disk Doctor diagnostics and defragmenters from the old days. The one for the original Mac was pretty incredible due to high res graphics, even had color coded filetypes, etc if memory serves.

Linux never had a good one to my knowledge. Improved filesystems reduced the need and then SSDs delivered the deathblow.

But I feel like the visualization was useful in itself to see what was happening on the storage device and unfortunately forgotten due to those improvements.

>really do miss the disk visualizations of the DOS

Even Windows XP had a graphical representation of the disk contents, and it animated during defrag.

#keeper

Does anyone know if there exists a tool that can convert tarballs to filesystems and back. I know you can make a loopback device, but it can be pretty hard/impossible to do inside a container, and often requires special flags and privileges.

For squashfs there is `tar2sqfs`[1] and `sqfs2tar`[2] (and now also `sqfstar`[3]) that can go from tarball to filesystem and back again.

For ext4, I recently saw a patch set on the mailing list[4].

Not sure about other filesystems tough.

[1] https://manpages.debian.org/unstable/squashfs-tools-ng/tar2s...

[2] https://manpages.debian.org/unstable/squashfs-tools-ng/sqfs2...

[3] https://manpages.debian.org/unstable/squashfs-tools/sqfstar....

[4] https://lore.kernel.org/linux-ext4/20230812150204.462962-2-j...

guestfish and libarchive are sort of in this space. Neither is a canned tool for exactly this.

You can extract the tarball to a directory, then run mkfs.ext4 -d DIRECTORY FILENAME BLOCKS to create a filesystem[0]. You'll need to know how many blocks your filesystem needs to be in advance.

Unfortunately, mkfs.ext4 only works on Linux. There is no port for other operating systems.

[0] https://github.com/cedws/concrete-ubuntu/blob/0ae3f076c5a20d...

If you want to create read-only, use ext2. No point in using ext4 if you don't want journaling.

Extents are a useful feature even for readonly although the gain is smaller.

You can also use ISO9660.

7z can list and extract from ext4 images. Respectively:

$ 7z l <image>

$ 7z x <image>

You can use fuse2fs to mount any ext[24] file system in user space. You can even use "-o fakeroot" to get complete control over UIDs and modes.

I've been doing something similar recently, though it doesn't directly convert a tar to an ext4 FS. But maybe this can help you get to where you want to be.

On Alpine Linux:

```

apk add --no-cache coreutils e2fsprogs

```

#!/bin/sh

# Untar the tar file

mkdir -p /tmp/my_untarred_files_dir

tar -xvf my_tar_file.tar -C /tmp/my_untarred_files_dir

# Make an empty image file.

dd if=/dev/zero of="fs.raw" bs=1M count=1024

# Format the file as ext4 (with journaling) and copy untarred files into it

mke2fs -t ext4 -j -d "/tmp/my_untarred_files_dir" "fs.raw"

```

If you want to make a qcow2 image, you can then do this:

```

apk add --no-cache qemu-img

qemu-img convert -O qcow2 "fs.raw" "fs.qcow2"

```

Maybe you are looking for something like fsarchiver?

https://www.fsarchiver.org

It's absolutely fantastic, how gracefully ext2/3/4 stood the test of time, for something designed in early 90s by people who hadn't had 30+ years of kernel hacking in their pockets (yet). While I generally prefer ZFS, ext4 remains a rock solid choice - I rarely have a reason to pick anything else (on Linux, at least).

Uses the ext4 file system --> "LUKS encryption and decryption: In the cryptsetup-laboratory with Termux (running under the Android 11 operating system), "cryptsetup reencrypt --disable-locks --type luks2", no root access, no loop device, and an unusable "mount" command.": https://old.reddit.com/r/termux/comments/18am78j/luks_encryp... (old.reddit.com/r/termux/comments/18am78j/luks_encryption_and_decryption_in_the/)

i am baffled this article used a usb stick instead of a file where it could have saved a lot of unnecessary i/o, but a good writeup. i had to implement ext4 support in something back in like 2017 and wish something like this existed

Maybe they wanted to avoid having to deal with setting up the loopback device?

You mean the one losetup command?

Definitely that would be less portable syntax.

what definition of i/o are you using, does file i/o not count?

i apologize, i meant external (and prob slower)

Reply on news.ycombinator.com