«Structured Computer Organization » by Andrew S. Tanenbaum. I don't know the newer edition but the 3rd edition is the first computer book I have read and its layered approach is great (and I think, was new at that time)
Check out Code: The Hidden Language of Computer Hardware and Software by Charles Petzold for a slightly romantic take, or Malvino's Digital Computer Electronics for an undergraduate view of computer architecture.
Look up the "system architecture" course in the CS department of your favorite university. Download the syllabus, slides, and required reading list.