I would not propose this book as one that every programmer should read. But others have posted about books that started with fundamentals and illustrated how computing systems evolved, which brings this book to mind... Henry D'Angelo "Microcomputer Structures".

"Microcomputer Structures" is out-of-print, but I recall the small text starting with atomic physics and building up to an introduction of the Von Neumann architecture.

As a freshman at Boston University in the 1980's, I took this mind-expanding course (probably because it was way above my level) with Professor D'Angelo. The final project was building an interface to a single-board computer.