The UNIX Programming Environment (1984) | Hacker News Reader

The UNIX Programming Environment (1984)

archive.org

Skimming this makes me long for the time when the multiuser nature of Unix was really utilized as intended.

I miss being on a system and typing "who" to see who is on, and starting a chat with them with "talk". Or sending mail to other users just on that system. Using "finger" to read updates in their .plan files.

Many of the "social" aspects of the internet today existed in that multiuser aspect of Unix, in a much more intimate way.

Macs and Linux machines are still multiuser systems, but "users" are pretty much just used to provide separate configuration and permissions scoping for different application services. It would be nice to have someone hop onto a terminal on my Mac once in awhile and say hello.

Sounds like you should join a Pubnix!

https://tilde.team/

Very interesting! Thanks for this!

I didn't enjoy my tildeverse experience personally.. there are others like SDF:

https://sdf.org/

What do you enjoy more about SDF? (I haven’t tried either, just curious.)

I just didn't enjoy the social/cultural aspects I encountered on the tildeverse, it was quite immature and dysfunctional based on my experience ~2 years ago.

Yes, SDF.

My primary use is for my relayed gmail, but it's indeed fun to be parked on a box all day with people just like you.

sdf.org doesn't exist, according to Firefox. What is with these websites that can only be accessed by Chrome? And how is that even possible?

It opens with Safari and Firefox (on macOS 12). Maybe “does not exist” is a problem with domain resolution?

I can confirm sdf.org doesn't exist using Firefox but shows ok with Chromium.

On my debian pc. Probably a firefox bug? Oh, wait.. The second time I tried sdf.org it showed up correctly, so I'm guessing it's a temporary DNS name resolution where it fails on the first attempt.. Doesn't make sense!

Works on my fox. Probably something on your end or a temporary issue on theirs.

Here's some I found, that one on the list

https://github.com/cwmccabe/pubnixhist/tree/master/systems

I'm not sure it is entirely that helpful, but it IS a list!

There's also DECUServe for the VMS experience.

https://eisner.decus.org/

> I miss being on a system and typing "who" to see who is on, and starting a chat with them with "talk". Or sending mail to other users just on that system. Using "finger" to read updates in their .plan files.

You've just described a BBS. Some are still around, accessible via telnet/SSH.

>You've just described a BBS. Some are still around, accessible via telnet/SSH.

right, but that misses the point a bit; being connected to others on the system at the time was basically frictionless -- no need to connect somewhere, you were already on the same machine.

saying "hey bob, connect to X bbs" is a bit different; you're not already working there -- it requires user intervention to experience.

You usually didn't have a local computer. Or if you did, it was just about powerful enough to call into a mainframe and act as a terminal.

Unix computers at that time were expensive beyond the finances of most individuals to own one at home, and required special power and air conditioning. That's why one machine was shared among many users.

Presumably GP knows the reasoning for the decline of this sort of usage, but still misses this aspect of the computing environment.

Decline? Maybe, but cloud computing is surely bringing it back.

Cloud computing is ushering in the exact opposite. Each user has their own private cloud VMs; it’s much easier (and cheaper) to spin up and tear down VMs with images precisely customized by each user than to constantly run and maintain a single VM supporting a multiuser system. It’s also a far superior user experience; having root on your own machine is way better than having to ask a sysadmin to update a shared machine (and often getting denied in the process).

Who are the dumb DevOps giving root access to cloud environments?

We give them cloud shell access and Web UI, that is all.

What’s wrong with root access to a private cloud VM? Since VMs are private to each user, there are no security concerns with root access. If a dumb user playing amateur sysadmin accidentally borks their VM, they just tear it down and spin up a new one.

On the contrary, sysadmins who give root access to cloud VMs are smart, and know that having root access in safe, controlled environments does wonders for developer productivity. Imagine not having root access to your developer laptop! I have friends who’ve worked at such companies, and they all said it was hell.

In the vast majority of cases, you don't need root access to develop. You can install software and build and run code in your home directory. Root accesss is often just a lazy way for software installers to avoid the extra step or two of modifying their $PATH because some executable is not in /usr/bin

That precludes use of package managers, which makes life way harder (have fun building all dependencies from source!) It also makes it way harder to link against shared libraries, since you’ll have to update include/shared object paths for every piece of software you build.

There is absolutely zero risk or downside of granting root access to single user machines. Zero, zilch, nada.

Open an IT ticket, and the software shall be provided in SLA time.

Or I can just `sudo apt-get install` and the software shall be provided instantly.

No you won't, assuming the IT people did their job properly, regarding sudo administration.

What you really miss is social aspects of school or work environment where you could form meaningful connections with others without being stunted by corporate politics or ever growing list of sensitivities in society that gatekeep discussions of anything people actually care about. Otherwise similar functionality still exists in corporate tools. You can certainly chat with your workgroup or individual members, and corporate directory usually has a place to type in what you are working on. Plus others don't cause your processes to slow to a crawl by running CPU or I/O intensive workloads. Technology is not what needs fixing in this picture.

sensitivities "gatekeep" discussions of things people care about? seems like recent buzzword soup

Is it? Do you feel free to discuss subjects you most care about or your coworkers most care about without fear of getting cancelled? Not saying politics, just common aspects of human life.

Yeah this is why I abhor non organic team building outside work where the only thing safe to talk about is work.

No, I think what they miss is literally what they described. Because I feel the same way with LP MUDs which had many of the same commands and interactions, despite in fact having friends I can speak to and interact with in other locations.

I read an article about students using shared Google docs to chat in class and (to me) it sounded a lot like the experience of communicating with others connected to the same remote host.

I wonder if the Google Docs experience has an equivalent to the joy of discovering new systems and connecting to them and discovering new societies. I fondly remember some ephemeral connections made to the odd insomniac grad student or bored admin through those methods.

good memories of playing Nethack on the University cs Linux servers. Very rich bones files.

Hey, why's the machine so slow? Oh, Frank is running a kernel build.

Hence why user limits should be configured. :)

What I find interesting is that Plan 9's "who" did not. It's just

ls /proc | sed '{grep out usernames}' | sort -u

so it's pretty specific to whatever machine it runs on, as opposed to who's logged into the installation you've run it on.

On another note, on Tenth Edition Unix, when you logged in from a Blit, something would get pipefiled[1] over your tty and wrap writes from new opens in something that puts it to a window-manager window. Possible because of not having name spaces for files.

[1] 'Mounted stream' in http://man.cat-v.org/unix_10th/2/fmount

Chapter 8 was the most surprising part of the book for me when I first read it. Up till that point lot of the information was mostly what seemed like administrator/user territory. Then BAM! it goes into designing a full fledged interpreter for a BASIC level complexity language, suddenly you are into language design; utterly fascinating how using simple tools like yacc, lex and some simple code (the entire code for the interpreter is published in the Appendix) you can do amazing things as a programmer.

This is exactly what i came to point out!

That chapter named "Program Development" is a masterpiece and a must read by every Software Engineer.

I read it before i knew what Compilers/Language Design involved and it was sort of a revelation in how much you can get done without knowing too much theory but with the intelligent use of appropriate tools, a framework and a guiding hand. Whenever people ask me for resources on Compilers/Language Design i always tell them to first read this chapter a couple of times and grasp it thoroughly and only then move onto other proper textbooks.

A good followup to this chapter is the book Introduction to Compiler Construction with Unix by Axel Schreiner and George Friedman - https://archive.org/details/introduction.to.compiler.constru...

You have to read this so that when inodes come up at your sysadmin/devops social gathering's scintillating conversation you don't have to run to the bathroom and google it on your phone and try muddle your way through without your face turning red.

(inodes just strike me as one of those weird little things like SQL, except for much smaller/easier to learn--just something critical, always there, and for some reason, not understood by a lot of sysad/devops types. I forgot the detail until rereading this book recently. I imagine you can expect the details to vary widely on modern filesystems, but ext4 (descending from and backwards compatible with ole ext2) probably has the concept, and BSD's ffs/ufs, and the idea probably gives you some hazy idea of this general area of the world for a lot of filesystems

I also find the treatment of some old topics kind of illuminating, like the stty command when I do obscure stuff like use serial terminals or try to use a text buffer as a tty--interesting that you can still tell the kernel "hey, do this dirty hack for my terminal when handling characters and things. Thanks.")

Nobody thinks about inodes until suddenly your disk is full, but df tell you it is not ;)

Of course, the same df can show you what's happening with inodes, if used with the -i flag.

This is true! Once I figured out what to search for, df was able to confirm what the problem was for me.

You should be able to see inode usage with `df -i`, no? I usually use `df -Tih` (print the file system type, list inode usage, in human-readable format).

I've had the other case where `df` says the disk is full with plenty of inodes left, but then `ncdu` says there is plenty of space (supposedly).

Lo' and behold, you have some borked process still holding GBs of log files open that you thought you "deleted" -- really unlinked from a directory.

tl;dr: Don't forget `lsof +L1` to find pesky processes that are holding files open that you unlinked from a directory but have yet to be actually marked as deleted on the filesystem [1]. The space on disk won't be freed until the process closes the files or is terminated. (Technically, the open files are associated with a mount point, not the file system itself, which makes dealing with symbolic links across file systems somewhat confusing.)

[1]: For example, see https://unix.stackexchange.com/a/141639/332116)

You have to know to ask, though.

Exactly that took a bit of random walking to find!

Did you have to learn about lsof +L1 the painful way like I had to learn about df -i (every office phone ringing at once, emails filling up with alerts, etc) or were you wise from the beginning?

I learned about the inotify limit last week in a similar way...

Any software engineer should have written a filesystem during their bachelor's degree, and should be well familiar with the concept of inodes.

well maybe not that, but it is very surprising that there are software engineers out there who dont know what inodes are. on my uni this was explained in an OS class in year 2 of bachelors programme. its basic knowledge.

Operating Systems was an elective in my bachelor's program. It's certainly not basic knowledge.

So there are engineering degrees without OS knowledge?!?

Yes, and on the other hand, there are surely also degrees with OS knowledge, but without AI knowledge. And degrees with AI knowledge, but without 3D graphics knowledge. And... well, you get the point.

Hard to do AI or 3D graphics without an O/S.

But easy to do without understanding the OS's inner workings. And there are far more people working on 3d engines or AI than on operating systems.

I would be willing to bet that some OS knowledge comes in very handy for almost anyone who writes software than runs on an OS. I’m sure it’s more important for the 3D graphics programmer to understand the execution environment within the context of directX or OpenGL etc. But it is hard to imagine building any native application without some grasp of cpu scheduling, memory paging, device i/o etc that you’d get in an basic OS course

Really strange perspective. Sure it’d be nice if everyone understood those things. But to think they’re absolutely necessary for any application level programming is… odd.

Not odd at all.

To me if you don't know these things, you shouldn't be writing code.

In most of the last two decades of my career, the OS has been abstracted away from developers by their sysadmins. And things have only gotten more extreme with the emergence/resurgence of managed hosts in the form of cloud computing “serverless” services.

At least Portuguese universities always have introduction lectures to everything relevant in CS, electives are for deep diving, and you always need to do a bunch of them for getting the graduation credits.

Pity it isn't the same everywhere then.

>everything relevant in CS

Literally not possible in 3 years.

Traditional Portuguese degrees in Software Engineering are 5 years, on average people take 7.

When Bologna came around and reduced them to 3, Portuguese universities got around it by offering degree + MSc, so basically business as usual.

Known in English as Licentiate Degree.

https://en.m.wikipedia.org/wiki/Licentiate_(degree)

So, nope not 3 years.

Yes, but we have been discussing bachelor's degrees, not bachelor's + master's.

In Licentiate, bachelor's are 5 years, master 2 years, and PhD another 3.

One of the reasons to Bologna was to try to harmonize the fact that countries like Portugal, someone with a degree would be downgraded when applying abroad.

So we are pretty much discussing bachelor degrees.

Everyone either uses an operating system or builds one.

Few people actually do real "AI".

And lots of people... I was curious so I checked a video that taught JavaScript to newbies. It was full of errors. I would say that this is a common thing, unfortunately.

At my university the compilers course was optional. I always thought it was weird that such a foundational part of computing was an elective. We did have to take an OS course though!

Worse, there are a lot of SW engineers who have no idea how a computer works. One common mistake is that code is executed "instantly". Especially in embedded, this is problematic.

Not every filesystem has the concept of inodes. The FAT family, for example, or ISO 9660.

Nor the Apple file systems, BeOS, NTFS. It’s really specific to the original Unix file system and direct derivatives.

Even though other file systems call them something else (e.g. NTFS uses the term FRS) and their actual fields might be in a different order and contain slightly different data; all file systems use some kind of file table that contains fixed-sized file records.

I am writing an object store that also can manage unstructured data just like a file system. It too has a table of object records that are all the same size. One of the secrets for managing large numbers of files (todays HDDs are big enough that you can create volumes with 100s of millions of files) is to keep this record as small as possible. The FRS in NTFS is 4096 bytes per file. Try storing that table in RAM when you have 200M files! My records are only 64 bytes in size. 16GB of RAM will easily cache the whole table and still give you memory to spare.

https://www.youtube.com/watch?v=dWIo6sia_hw

It doesn't matter, a class covering filesystems would cover the concept of inodes.

You don't need a bachelor's degree to write an ext2 filesystem driver, I didn't finish high school and I've written a read-only one that works on both MSB (68000) and LSB (x86) byte order machines. The format is well documented:

https://www.nongnu.org/ext2-doc/ext2.html

https://wiki.osdev.org/Ext2

I agree. Also every software engineer should have written a compiler, a scheduler, a device driver, a small kernel, an IEEE compliant floating point library and a 3D raytracer in their free time between 30 courses by the time they have completed their degree! /s

Writing a filesystem from scratch is something I would expect from a full 5 years degree entirely dedicated to operating systems, not a generic SWE curriculum

This, but unironically.

And if I didn't do a computer science degree?

then most likely you're not a software engineer, but some kind of coder/developer.

This is a great book, one of my faves. It is a surprise to see it hosted on archive.org when it is still in print, though. Please support good technical writing!

https://www.amazon.com/Unix-Programming-Environment-Prentice...

It's not in print anymore. Your link is a used book.

The link I posted has both new and used copies for sale.

It's being advertised as new as in, "it's not not used"; not new as in, "it's not old".

The edition being sold is 39 years old.

"Being in print" or "new" a is not well-defined in this context. "Being in print" is only a metaphor, since the process of printing is typically completed when a copy of a book is sold (and makes not sense in the case of publishing-on-demand, where it would be equivalent to "is available"); and "new" can mean "in its original condition" or "recently published" (with some uncertainty what time span "recently" denotes).

A more precise criterion would be whether a book is still on its publisher's backlist. As far as "The UNIX Programming Environment" is concerned, the answer is positive. Its publisher Prentice Hall was aquired by Simon & Schuster and later sold to Pearson.[1] On Pearson's Web-site the book is still available.[2]

[1] https://de.wikipedia.org/wiki/Prentice_Hall

[2] https://www.pearson.com/en-us/subject-catalog/p/unix-program...

"The UNIX system is full duplex: the characters you type on the keyboard are sent to the system, which sends them back to the terminal to be printed on the screen. Normally, this echo process copies the characters directly to the screen, so you can see what you are typing"

Ohhhh that is why it is called echo!

The AT&T logo on the cover is a reminder of how many of the foundations of modern computing came out of Bell Labs.

Thanks to AT&T not being able to charge for anything out of Bell Labs, they were quick to react once they were allowed to.

Had it not been the case, we would probably be using something else instead.

> Although most users think of the shell as an interactive command interpreter, it is really a programming language in which each statement runs a command

I’ve read the book in one summer when I was in highschool and this sentence was the biggest light bulb moment for me. So simple and eloquent. This was the point where I _get_ what does programming and writing code really means.

This book is a very fun read and was awesome for understanding why some of the Unix stuff works the way it does

Agreed, this is an excellent book. I would definitely recommend it even today to people that want to improve their understand of unix/linux.

Yes back when there were a few good books that were the references for tech subjects. No Google, no Stack Overflow, no "For Dummies" waste of paper.

I'm currently reading and it feels like I'm learning a lot but at the same time it shows that a lot has changed since then.

I would love similar book but relevsnt for the current systems...

I imagine that would be hard because the scope is so much larger today

I'd love this book if it was rewritten for ANSI C and maybe modern tools such as the ones for NetBSD or OpenBSD at least. AWK and sh still work of course.

This book has an exceptionally high signal:noise ratio. It's long been a favorite of mine, even with its relatively high price tag for a small paperback.

I had to buy this book back in 1985 when working on my CS degree. Like just a handful of other books, it has survived decades of going through my collection of physical books and weeding out ones that are no longer relevant.

Which says as much about the surprisingly lasting utility and consistency of UNIX as it does this book.

Amazingly I have this on my bookshelf, arms reach away, the times before everything was on the internet. Good times. Though whoever stole my ring-bound vi reference guide of that era - cursor you.

I love this book! There is a lot of abstraction today that most software workers benefit from, but I think there is a lot of value in understanding lower-level mechanisms.

I’ve been reading through this for a few months, lots of great content and can’t believe how useful it is so many years later.

This is a really great read. Added to my top 10 books for computer geeks.

I wish this was a history book.

Then you might be interested in

    "Unix: A History and a Memoir"

also by Brian Kernighan

Chapter 5 is the best intro to shell programming I've ever read. If you read it for that purpose, don't skip chaps 1-4.

Reply on news.ycombinator.com