There's No Such Thing as Knowing Your Computer 'All the Way to the Bottom'

I disagree. If it can be built, it can be known, top to bottom.

Not everyone may want to know all the gory details, and nobody wants to get called on to explain it, but it can be, and should be done as a matter of course.

The issue is that the industry will kick and scream to keep from being held accountable to providing the customer with a way to be able to parse that information, and no one wants to hire people to document, because that isn't seen as positive value creation in today's corporate climate. It won't ever be short of a legal requirement to do so. That legal requirement will be fought tooth and nail on the basis of wanting to continue to protect trade secrets.

Nothing keeps a device from having it's actions fully elucidated except an unwillingness to abolish information asymmetry.

The article makes some good points, but I feel that the headline is inaccurate.

It may be impossible to know everything, just as one cannot claim to know "all" of history, but one can make a bloody good go of it.

The main problem I've encountered personally is simply limits on human memory. Things fall out of your brain.

I studied semiconductors, electromagnetism, quantum mechanics etc as part of my Physics degree. I've written emulators, webapps, backend code, etc.

There are gaps. Not conceptual gaps but rather in implementation.

Could I write a top to bottom tutorial? Not without research, and undoubtedly subject matter experts would pick at it, but I believe I have some understanding about every level of the stack and an idea of where to pick that knowledge back up if I need to.

Is it useful?

I mean, surely that's for an individual to decide. Financially perhaps you'll be better off learning the hot new framework of the day and hobnobbing with the cool kids.

But then you could also get rich making wanky adtech optimisation. Or just starting a run of the mill exploity corp.

Me? I just want to know how it works, in endless fractal beauty.

I don't live my life based on what others consider to be "useful".

Not just in programming, either.

> there's No Such Thing as Knowing Your Computer 'All the Way to the Bottom'

NAND to Tetris (www.nand2tetris.org) seems like a pretty good take on the concept.

Another one is Wirth's Oberon system, which runs on Wirth's TRM/RISC architecture, which is simple enough for students to implement in Verilog (or another HDL such as VHDL or Wirth's Lola) and compile to an FPGA.

MIPS is also simple enough to implement in Verilog over the course of an academic term and also has good tool support for languages like C and C++.

Personally I find understanding low-level system behavior to be interesting, fun, and helpful for understanding many irritating application behaviors (often errors and slowdowns) which are caused by interactions with the OS and hardware.

I was recently doing some language evaluation for rewriting an existing system in Go to improve its cross-platform compatibility. One of the things I realized in this exercise was that "compiles to C" was a powerful feature - not because of C itself, since C often gets in the way of the source language's intent - but because it means that the result fits into and leverages all the C-based tooling, facilitating the desired level of platform independence.

And for a lot of projects today, "the bottom" ends with C for precisely this reason - while the fact that C is the chosen language for this task is completely arbitrary, as arbitrary as the status quo hardware paradigm of "x86 on desktops, ARM on mobile". I could imagine a universe where it was an extended form of Basic or Pascal that ruled the world instead.

Likewise, I was also looking into extending Lua with a small DSL interpreter that could accelerate certain byte-level tasks: copy around blocks of bytes, run some common algorithms quickly, define spaces for variables, apply some parameters. At first I thought of this language as a "VLIW" assembly language with a particular focus on having a big bag of tricks, but when I compared my semantics with actual hardware assemblers, I found that having no registers or stack manipulation and focusing only on direct memory addresses changed the character of it so much as to make it a different beast, one more like the "autocoders" of the 1950's: not quite ALGOL, and yet clearly headed in that direction.

All of which is to say - the more you are comfortable with compilers, the more the bottom just becomes the thing you happen to be leveraging to ship product. It isn't "the real stuff" until it becomes the specification language. Some things, like how numeric types are defined in hardware, force the terms of specification way up the stack so that the hardware may be used at its capacity. But that in itself is an "essential complexity" in computing, where the physical realities coincide with the desired programming model. When it's software defining the spec, as in our libc-and-Javascript-driven universe, that's more akin to social complexity, but it produces the same kinds of effects.

Maybe it's not worth learning C as a programming language, but it's worth learning the C standard library, as every language links it in and most of the functionality is available. I'd also say to throw in a cursory reading of the Linux kernel's syscalls as exposed through glibc and a tutorial on how to go from glibc to the kernel implementation of the syscall. It's not strictly necessary for game development or frontend website design, but in games you'll end up learning C++ anyway and with websites everyone seems to migrate into the backend servers eventually or else quit programming and do graphics design.

As far as operating systems, Rust does seem to be a better entry point, with the Redox OS and general community.

The main reason to dig down is to uncover faults and leaks in abstractions, which is something you absolutely will have to do in the fullness of time.

A corollary reason is to improve your "mechanical sympathy" when writing high-performance code.

Many of what we once called "application devs" -- now more commonly "full-stack devs" -- will never have to discover real mechanical sympathy, which is fine. But there is a glass floor for such people.

There is one main reason to learn C: everything can interface with C, and you may need to extend your language's capabilities.

There is one main reason to learn assembly: things WILL eventually go wrong, to the point that you need to debug at the machine level in order to understand what's going on.

Also, many debug tools such as ptrace are primarily accessed through C, and require a basic understanding of the assembler view of the machine.

I realize that it's a bit like saying that COBOL programmers will never have trouble finding work (probably mostly true) but until Linux is rewritten in Rust and macOS/Xnu is rewritten in Swift, C is going to be essential for anyone who has to work with UNIX-like OS kernels and associated drivers.

I would also argue that C is pretty essential for anyone who wants to use EBPF/IOvisor.

This is an excellent article and I rarely see people giving this advise. At the end of the day there is no bottom in general. And it's not clear why that's the case unless you have a reasonable amount of experience in electrical engineering / manufacturering and computer design. An interpreter or a compiler is a computer. Anything lower than that is just implementation details.

I don't agree. At all. Forget C/C++. Build a microprocessor in VHDL or Verilog. This is a common task in many universities with better EE or CSE disciplines. Do these folks understand Java? Or modern C/C++. Well they do much better than the folks that learn at the Java level and work up.

This lives on the assumption (one I often see repeated) that everyone is writing code to the same requirements and implementation details as you. If I'm in a washing machine I;m in total control (or bloody well should be).

None who doesn't care about it will ever learn it just to become a good developer.

Never seen this.

I learned it out of curiosity.