Getting better at Linux with mini-projects
carltheperson.com
carltheperson.com
While the technical accomplishments and understanding are commendable, I think that the main takeaway for me is the excellent approach to learning. Keeping it simple and taking an open-minded approach to learning (as exemplified by his attitude toward systemd) is an approach that more people could stand to take. So often we complicate things in our heads to the point where they seem unknowable and so we assume that they are too difficult and we don't try. I have seen this in myself at times and I see it others as well.
I'll be sharing this post at work so that the people who I work with who don't really understand Linux so well can appreciate the approach that was taken and hopefully also glean something from the excellent write-up itself.
Good skills Sir. At your age I couldn't find my arse with both hands 8)
As in the well-known dictum by Richard Feynman, “What I cannot create, I do not understand,” successful design is also a powerful way to show that a design principle has been understood.
sync
sync
halt
$ sync && sync && sync && halt
Disclaimer: had that muscle memory burned into me for decades, I still do it ...
Qudos to the young'n' for doing what I'm talking at! That is a person with the ability needed to go far.
We need more people like you. Kudos for having the attention span to do this. Enjoy your HN top spot, well deserved.
By contrast, Unix, in its BSD and GNU variants, was vendow and hardware independent, and by the mid 1990s, Linux was pretty clearly its successor.
Editor / word-processing tools have been especially telling. I've used and often been expert at: DOS Edit, EDT and EVE (both VMS), MacWrite, Wordperfect, Wordstar, MS Word, AmiPro, Notepad, MS Write, Aplixware (an early Linux office suite), and multiple variants of Libreoffice (StarOffice, OpenOffice, NeoOffice, ...).
And then there's vi/vim, which I first cut my teeth on in the mid-1980s, and which I still use daily and learn new features and uses of all the time (some themselves new, often long-time capabilities). That's been an extraordinarily durable learning investment. (Emacs similarly so.)
Awk's another basic tool that's quite useful and ubiquitously available.
There have been hitches and inconsistencies: hardware changes invalidate old concepts (shutdown sequences as noted here), firewall, audio, and other subsystems have changed dramatically and multiple times. Physical vs. cloud server management is a very differrent head. Systemd is a major (and often disturbing) development.
Generally, though, the further you stay from GUI and vendor-specific tools, the more durable the knowledge. My preferred desktop environment is based on the 1980s NeXT and has remained almost wholly unchanged since the mid-1990s whilst GNOME and KDE underwent multiple revolutions and Xfce4 came into being. Windowmaker still does things none of the other three can touch, and my muscle-memory is well into its third decade.
As you get older, learning slows, but it's forgetting which becomes especially hard. Durable, incrementally-evolving tools minimise that pain remarkably.
For example CMU also makes you write your own shell. The main focus is implementing job control like: jobs, fg, bg, and various signal handlers in order to understand how forking child processes work.
I see that your shell doesn't handle much other than cd so this might be a nice next step if you want to continue working on it. (I can't find a good public link for it but you can probably follow along using: http://www.cs.cmu.edu/afs/cs/academic/class/15213-s02/www/ap...)
This makes you very aware of the way login works in linux and teaches you how to compile your library as a shared object conforming to an interface. Of course, you should be aware that you just baked your own security mechanism, so use it that way and better avoid using it for anything sensitive :)
There is a video presentation, and a set of "challenges" you can use to incrementally get more complex with awk, starting from super simple.
The challenges repo is on github here: https://github.com/FreedomBen/awk-hack-the-planet
If you want to watch the videos, there are links in the github repo but for convenience:
Presentation video: https://youtu.be/43BNFcOdBlY
My solutions to exercises video: https://youtu.be/4UGLsRYDfo8
Curious if you compared the end result with "tac".
Source: https://github.com/coreutils/coreutils/blob/master/src/tac.c
Walkthrough of how it works: http://www.maizure.org/projects/decoded-gnu-coreutils/tac.ht...
tac | revIt's not exactly the same as "tac | rev" though. For a typical text file with each line ending in a newline...this recat tool outputs a newline first, and the last line is missing a newline. So it's a true reverse printing of last byte to first.
For bash scripting I found Julia Evans' zine bite size bash very interesting. Even though I had been scripting for quite some time I found hidden gems in there.
purely judging from your profile picture it seems that you're about half my age :) and I think it's excellent that you're doing a bunch of things - personal blog, linux, tutorial post... Just remember - most of the guys around here tend to do it for marketing reasons (i.e. we want to find a job and we're advertising our capabilities), and those projects might not be the most interesting things out there.
Bottom line is - just because everybody else is doing it, might be because they have different motives than you, and I hope you'll do/build firstly what's interesting and fun, and secondly what's financially viable.
Good luck, have fun!
Thank you so much for the heads up. I'm only doing this because it's fun! I love using Linux and I feel like I learned a ton from this.
> :)
I have full source of ALL of that!
I also discovered linux in that time, and one post mentioned running Mandrake 7.2--which I doubt was the first version I ran.
Those skills eventually turned into jobs. And some of my peers who were doing the same thing turned into lifelong, important friendships.
Anyway, just reminiscing. I had more fun computing in those few years than I've had in the whole time since. And even though it was for fun, it was easily some of the best-spent time in my life for $$$ in the future.
I was so put off by their syntax that I didn't even bother to try to learn them when I was a newbie. If I had a time machine, I would certainly go back and tell myself to learn these tools and save plenty of time instead of always using vim/perl for every text processing problem. I've now written my own books (https://github.com/learnbyexample/scripting_course#ebooks) - might help you.
For `bash`, I'd highly recommend these resources:
* https://mywiki.wooledge.org/BashGuide, https://mywiki.wooledge.org/BashFAQ, https://mywiki.wooledge.org/BashPitfalls, etc
* https://devmanual.gentoo.org/tools-reference/bash/index.html
Also, https://linuxjourney.com/ is nice for many topics.
It is a simple page with links to the corresponding parts of the POSIX standard which makes it a lot easier to find the correct documentation when you need it.
For those who wonder what POSIX is: It is a standard for operating systems and one part is about shells. So if a shell whats to be POSIX compliant it has to support certain commands. And if you write POSIX commpliant scripts, they won't just run on Linux, but also on MacOS, other BSDs and other POSIX compliant operating systems. Bash supports POSIX, but if you use Bash extensions that are not part of POSIX, you might run into trouble with other shells (e.g. dash on Debian).
I've always wanted to make a unix sandbox environment, using FreeBSD jails provisioned from a webapp, that has little challenges and games to teach the basics of the command line all the way up to hosting a basic HTML website with Apache.
Is this something people would find useful? I've been thinking about implementations but I don't want to jump too far into it without validation that this would be something people would find helpful and interesting.
[1] https://vim-adventures.com/ [2] https://gitlab.com/slackermedia/bashcrawl
If you implemented your game, I would definitely give it a try. More for fun than for the learning experience, but if I happen to learn something new along the way, I certainly would not complain. ;-)
Great article, and great approach to learning the inner workings of the standard Linux suite!
Actually I recommend reading the entire man page for bash.
Edit:
I think maybe you are referring to those numbers people used to quote after mentioning a manpage (1) (5) and so on. I totally missed the boat on that one, and never learnt what these designations meant.
"Info" is a different documentation format, most commonly used for GNU tools (including bash). The section numbers don't generally apply to info documents.
For some tools, the man and info documents have the same information. For some, the man page is a brief summary that directs you to the info documentation. For some, there's only a man page. (And for some there isn't even that.)
IIRC for a while they used to have man pages that basically said “just read the Info pages instead”. This is not the case anymore, in Linux distros these days even the GNU tools have decent man pages.
Anyway, the Bash manual is actually written in the Texinfo format. It’s not so bad, to be honest :)
A large portion of them are just auto-generated from the `--help` output (and --help has always been fairly high quality for GNU tools). Perhaps with a few extra paragraphs sprinkled in.
I've found that GNU man pages are often an unhappy medium between the terseness of `--help` and the verbose explanations in the info manuals; and so the GNU man pages are almost never what I want. If I want to just quickly look something up, the man pages are too big and I should have used --help, and if I want to understand something they're too brief and I should have used info.
That said, the Bash man page is great (as is the info page). It's always amazed me that the Bash maintainer (Chet Ramey) goes through the trouble of maintaining two entirely separate documents with such detail and quality.
It is so long, I never find anything there
And if I search for some command, like /if or /fi, there are far too many ifs in the text to find a useful match
As for what those sections are, `man man` tells me (my system gets this man page from the "man-db" project https://www.nongnu.org/man-db/ ; your system may use a man page of different authorship, but it should tell you roughly the same thing):
The table below shows the section numbers of the manual followed
by the types of pages they contain.
1 Executable programs or shell commands
2 System calls (functions provided by the kernel)
3 Library calls (functions within program libraries)
4 Special files (usually found in /dev)
5 File formats and conventions, e.g. /etc/passwd
6 Games
7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
8 System administration commands (usually only for root)
9 Kernel routines [Non standard]
You can tell `man` which section you're looking in like `man 5 passwd`. This is useful for looking up passwd(5) which describes the /etc/passwd file, rather than passwd(1) which describes the `passwd` command to change your password. Or `man 2 signal` to describe the "signal" system-call instead of the signal(3p) libc function, or `man 7 signal` to get an overview and listing of signals.
Weirdly, they don't share much language. Seems like a waste of effort to write two distinct high-quality manuals for the same piece of software.
It took a while to get Disks burned to install. I had struggles moving back and forth between two machines across rooms to look up "help" documentation from the internet. When it was all installed, nothing worked. Network cards, sound, etc. When my network card worked, it felt like heaven. When my screen resolution was fixed (x11 settings), felt like heaven. Repeat.
He had asked, do you really want to learn how it works? I said, yes, let's start there. I was very confused with all the Linux variants so he recommended one. Now I look back and unsure that is where I should have started. L.O.L.
Nothing like building something useful to learn a new tool or technology.
MESA appears to be All about OpenGL and Vulkan for Games but there's not much attention paid to OpenCL and GPGPU compute for Things Like Blender 3D's GPU Accelerated Cycles rendering, and other Applications like Dark Table, Gimp, etc. that need OpenCL for GPGPU/Compute acceleration workloads!
There really needs to be a Primer for Linux and How that OpenCL is plumbed into the system and what files are used to get that all working on Both Integrated and Discrete GPU based laptops! So What's the Linux equivalent of MS's WDDM and how does the Linux Kernel get Graphics and the Graphics APIs and Drivers working properly for User Space Applications to make use of.
pp header.txt spreadsheet.csv
cat header.txt spreadsheet.csv > temp
mv temp spreadsheet.csv
But I never got around to it, and it's been a few years since I had to use C at my day-job (which was never great to begin with), so it fell to the wayside.
cat a b | sponge a echo -e "0r header.txt\nw" | ed spreadsheet.csv ex -sc '0r header.txt' -c wq spreadsheet.csv
echo "whee" | ex -sc '0r /dev/stdin' -c wq somefilebecause `tac` was already taken ;)
It also delves into the basic tools and then writes them from scratch. I found it quite inspiring as to where to spend time while learning UNIX.
[0] https://www.amazon.com/Understanding-UNIX-LINUX-Programming-...
For me out of university, it was a co-workers books (some of which I ended up buying) that help me understand the role of UNIX better (I used the alpha machines at university, but was thrown into HPUX/Solaris at work..)
I think this was one: https://www.amazon.com/Advanced-Programming-UNIX-Environment...
Of course a little out of date now, but a lot of the general concepts are the same.
Unix Power Tools helped me a lot as well. https://www.oreilly.com/library/view/unix-power-tools/059600...
How did others learn this stuff?
not to be confused with the C shell, csh/tcsh(1)
I had a very similar idea of implementing the Linux programs in python back then.
Keep it up !!!
sudo ps -ef | awk -v user=$USER '$1 != user && $1 != "root"'many, many years ago i met dennis ritchie at the computer literacy bookstore in san jose. in the Q&A he commented that when he realized a circuit board was essentially a small network, the entire design of unix came to him in a flash. unix is an i/o multiplexer with light security.
Should utilize something similar in teaching FreeBSD and illumos too.