What every programmer should know about memory | Hacker News Reader

What every programmer should know about memory

lwn.net

129 pointshaskellito37 comments

Does every programmer really need to know the physics behind dynamic ram?

What every programmer should know is the latency of the various memory types (cache, RAM, virtual) and their respective sizes.

Here's a nice graphic:

http://duartes.org/gustavo/blog/post/what-your-computer-does...

Duartes has a bunch of great articles. Its too bad he isn't writing anymore. Anyone know why all he stopped?

Not every programmer needs to be a hardcore systems hacker; you can ignore the low-level details and lead a healthy productive life. But me, personally, I get a massive inferiority complex if I don't dive into Nasm and GDB every 2-4 months. It would pain me not to know useless things about obscure stuff.

Yesterday I was prancing through reddit when I followed this link: http://pvk.ca/

My stomach sank when I realized I wouldn't have the time to "follow along" (i.e. waste the day reproducing his results, checking facts, researching and following links) because I am about to start a lead-dev/PM position and need to work on SaaS and EA stuff. So I closed the tab and moved on to focus on what matters. It wasn't until 1AM that I realized what I have done; aarongough sent me a link to his toy prototyping language and I have just spent 5 hours reimplementing it in Lisp, in the process re-digging into the Self papers! (and discovering warts, but that's another story.)

I am an Adderall prescription away from leading a healthy productive life, without unnecessary knowledge about stuff.

Nonsense. What happened to abstraction?

To suggest this as recommended knowledge to all programmers is actually making me angry.

My education is in Computer and Electrical Engineering, while I mostly work with high level languages building software for the web. I've traveled up and down the abstraction ladder out of curiosity -- that explains the dual in EE. I've enjoyed it all, but there is zero reason for a programmer to know almost anything from that page. You can make phenomenal software without ever knowing what a transistor looks like on paper; and that's the whole point of progress.

If anything, we should encourage higher level thinking and discovery of more expressive languages and tools.

To me, the mentality that CS should require (near) EE dual-citizenship reeks of "we had to learn it, so you have to too". I highly doubt that knowing how to construct a logic gate from silicon will help me program one iota better.

Heck, even including what the logic gates are and how to combine them to make RAM, for instance, is valueless. So my CPU is mostly NAND... this changes things for me how? If you're going into digital electronics, then by all means... but CS != digital electronics.

No?

My EE classes were about logic gates, flip-flops, and other abstractions. Logic gate chips are abstractions on some complex stuff that you do NOT have to worry about in CS (the real stuff -- resistance, etc).

EE, as served to CS folks, is another approach towards understanding &&, ||, ^, and so forth. You get a firmer grasp on the relationships between logical things. For example, it should be trivial for you to see that !(a && b) == (!a || !b), and you can transform your complex conditional statement to whatever is most readable.

For that matter, should you have to work with assembly, what you learn of these logical abstractions will help with optimization.

Another point... Especially in introductory CS courses and in most algorithms courses, we focus primarily on sequential approaches towards problem solving. In EE, things are, by default, parallel. They have to force sequence (where necessary) with such mechanisms as flip-flops. Exposure to thinking the EE way actually helps with abstract problem solving in parallel software development. For example, software "pipelines" are modeled much after the hardware circuit pipelines. OpenGL? Functional programming? Many similarities.

I remember very specifically coming up with a good heuristic towards solving the big P/NP problem that came from a realization I made from playing with the EE logic stuff.

What about synchronization? Locks? You have handy API for this stuff, but, as evidenced by a lot of bad programming and fear of parallel programming in the last 20 years, people do not actually know how to use these tools, much less how they work. And the EE guys work with this kind of thing regularly; they already have a number of answers that we can use to model our software.

Finally, if you ever have to develop software that helps in the operation of circuit boards, dealing with FPGAs or microprocessors, the EE knowledge helps you and them debug interaction problems (the most common problem being that of timing and how to synchronize all those parallel events). I know... been there... was thankful for the background and was able to help dig into FPGA code to find the subtle problems quicker.

Just to toss in the authority argument, isn't Sussman an EE guy?

So yes, a certain exposure to EE is valuable for students of CS, primarily for the exposure to other ways to approach everyday CS problems.

paragraphs 2-3) absolutely. But that's dealing with logical operations, not logic gates. Physical logic gates are an implementation of those operations, not the other way around.

P5-7) I see this as an inherent flaw in CS education, but it's been a rational flaw: multi-core systems are relatively new. Sequential approaches made a lot of sense at a physical and logical level. At least it's improving, if slowly. And, again, EE is an implementation of the logic, not the other way around. EE itself may provide an application, perhaps one that's easier to grasp by most people (to which I must say: proof?), but so is programming. And for programmers, programming seems to me more applicable, and certainly a quicker-to-test method. How much learning a functional language would help the average programmer is another argument entirely.

P8) well yeah. There you're working directly with things resulting from the electronics at the lowest level. This is very very strictly a combined field, however. The same is true for many fields - they are nearly defined by their crossing of multiple borders. Programmers, for instance, have absolutely no requirement that they learn one bit of user interface design - they may only ever design low-level APIs. Top-level design is partly defined by that bit of art / UI theory crossover. Note also that very few CS majors cover UI at all, at least as far as I've seen, unless they're specifically for UI.

P9) If anything that supports the "I had to learn it, so you do too" mentality.

An analogy is perhaps in order: if I'm trying to make a better wood glue, the underlying operation is to connect two pieces of wood. Will studying nails help me make better glue, just because it also connects two pieces of wood? And since nails are (often enough) made of metal, how about some metal-smithing to go with so I can make my own nails to learn from?

------

edit: I am actually interested in continuing this, there's no malice here (and apologies if it's been interpreted as such), I'm just fairly strongly opinionated on this. I've had a small chunk of digital electronics, and have yet to see a return from it for programming, though I'll definitely dive into an Arduino some time soon. If you think there's something special about general EE education that's lacking in general CS that is best taught from an EE perspective, I'd love to know what it is, because I'll happily learn it. I just haven't found any in my admittedly-very-shallow experience.

Parallelism I'm divided on. There's indeed much more focus on it in EE, and much more needs to be in CS... but I'm not convinced the timing diagrams will help any programmer more than some of the basic algorithms in CS, and my own experience has essentially only served to waste my time.

I maintain that to work well at abstraction level N, you need a good conceptual understanding of abstraction level N - 1. All abstractions leak, and you need to be able to handle it.

So by induction, you need to learn N - 2, N - 3, ..., 2, ... 1? I wholeheartedly disagree. By that logic, why stop at transistors? Let's get down to the physics of how transistors work, or hey, atomic physics.

Ehhh... only partially. If you need to work well at level N-1, then yes, the inductive proof works. They only mentioned knowledge about a level lower than you're working at, though.

My bad, I read it wrong.

I said "a good conceptual understanding." Not a complete understanding.

Actually in case of transistors, it is better to understand how they really work on physics level, because many abstract models of transistor do not make sense.

To work really well, it also helps to understand level N+1 or N-2, respectively--either by implementing a higher level abstraction in a lower level environment, or by working on two levels at once (and thus needing to understand the level below the lower level).

For instance, you might be stuck with using C or Java and thus have to reinvent higher level abstractions through clever use of function pointers or design patterns or something--hence thinking at N+1, implementing at N, and being mindful of N-1 as you implement. Or you may be writing in Python but optimizing code bottlenecks in C, hence when writing Python (N) you have to understand the C (N-1), and when writing C (N-1) you have to understand the assembly (N-2).

That's a false dilemma.

Higher level thinking, more expressive languages and tools don't preclude you from looking at low levels of your abstractions.

Edit: Looks like we were talking past each other. I was talking about the complete article (the non-EE bits are about VM, cache, NUMA awareness, memory management for multi-threaded programs etc.) The tail end of the OP's link, links to the rest of the parts.

I never said they did. In fact, I explicitly mentioned that I've explored other levels of abstraction.

What I am saying is that low level thinking such as depicted in the article has little to no value to the programmer.

How are you so positive about the lack of value to the programmer? What if you are programming at a low level?

Performance-optimization/OS kernel/HPC inner loops/Power-sensitive code (think mobile devices)?

Have you read the article? Where in there do you see any information pertaining to that kind of use? The "commodity hardware today" section is a broad, useful-content-free overview of architecture, without enough information to draw any conclusions. The "RAM types" section has nothing to do with any of your example categories. It helps how, precisely, to know which wire is energized to withdraw a bit from an SRAM circuit?

pacemkr isn't saying all low level info is useless to programmers, just the article's focus.

Yes I have read the article. It is in the seven or so parts. You seem to be talking only about the first page.

Part 5 deals with cache optimization. Part 6 deals with multi-threaded optimizations. Other parts deal with various layers of the memory hierarchy (VM, caches etc.)

I am, yes. Haven't read the whole thing; I may, the later parts look significantly more useful.

I, personally, only take issue with the EE-heavy portion; it only serves to obfuscate what's important with way too much (programmer-)useless info, and maybe to scare off people who could otherwise benefit from the rest of the article. I'm fairly certain that the OP of this thread views it the same way, as they mentioned:

"I've enjoyed it all, but there is zero reason for a programmer to know almost anything from that page. You can make phenomenal software without ever knowing what a transistor looks like on paper;"

implying just a single page (the one linked), and EE-only content.

But one does not need to know the behavior of atoms (if we even do) in order to whittle wood. There's a tipping point for everyone.

Nonsense. What happened to abstraction?

They leak. Not that all of this information is necessarily required for all programmers, but when dealing with a leaky abstraction it is nice to know what's leaking through and why it is so.

That would be one serious leak if a programmer had to even be aware of the difference between SRAM and DRAM, yet alone of their respective schematics. It just doesn't happen. When it does, it isn't a leaky abstraction, it's broken to the core, and knowing these surface level details about electronics wouldn't be enough to fix the problem.

  What happened to abstraction?

I think that's the real gotcha of programming. The programming languages today probably are better described as "good generalizations" of the operations available on a CPU, and the abstraction leaks quite a bit when doing certain things like needing to deal with synchronization. If our programming languages were designed to be perfect abstractions, there'd probably be a subset of the english languages that could described/translate everything to machine code, but that's definitely not the case reality today. So, up till 2010, in my opinion, to get the machine to do exactly something without fail, this article is very relevant.

The physical information on there is valuable to someone. For example, it's required knowledge for understanding memory latencies and BIOS memory timings (valuable information for overclockers). Even if programmers spend most of their time working dozens of layers of abstraction above the transistor, a rough understanding of where their work fits into the "big picture" can't be a bad thing.

As life's, the nature of programming is a paradox: it's about two mutually-opposing views taking place simultaneously.

Programming is very much about computer science's theoretic abstractions as well as about bit-flipping in the physical reality of electric circuits. The trick is that while you must be at least partially aware of both views in order to be a good programmer, you can't program solely from one of them only. CS abstractions won't teach you to work intimately with your computer and, conversely, showing great command of your machine by knowing its eletronic nature and mastering its assembly language won't get you far until you can see on a high level what you're doing.

Programming could be said to be the skill of merging—or making— the theory and practice into a working program, and with the emphasis on the merging part itself instead of what is being merged. For example, at the same time, a good programmer must know both something about CS and electrics while still most of the programming work is about the art of making, which is yet another thing.

So abstractions are there. And they're there to assist you and provide solid ground to build on but at the same time, however, you can't rely on them either.

> "A certain amount of knowledge you can indeed with average faculties acquire so as to retain; nor need you regret the hours you spend on much that is forgotten, for the shadow of lost knowledge at least protects you from many illusions."

--William Johnson Cory

All programmers do not need to know the many details Drepper goes into about cache refreshes and dirty bits, it is true. But most programmers will care about performance at some point, and most of those will be helped by even vague memories about the memory hierarchy and remembrances of how large the performance penalties of cache misses are. Even inklings can be enough to motivate someone to reorganize their loops, or to investigate cache-oblivious algorithms (which are a high-level topic on their own).

If it's something every programmer should know then there should be a summary that every programmer will actually read.

Grep the articles for "conclusion" and read the articles on "what every programmer should do..." in semi-detail. HTH.

It's a cool link, but it gets reposted a lot, last one being 5 days ago: http://news.ycombinator.com/item?id=1496377

That link is a PDF/scribd version. I find it easier to read than a webpage.

Yeah, but every time the discussion is different though. To me, most of HN value is in comments.

If anyone is interested, a PDF of the entire thing is available at: http://people.redhat.com/drepper/cpumemory.pdf

Drepper's site has a bunch more articles, too: http://people.redhat.com/~drepper/

_The former is much faster and provides the same functionality. Why is not all RAM in a machine SRAM? The answer is, as one might expect, cost._

But since the density of transistors is pushing the cost of everything down how expensive is expensive? Does it make sense to product a SRAM-based laptop with 512MB of fast memory?

SRAM uses 6x times more transistors to store the same amount of information, so do your math for a hypothetical "512MB laptop". On the other hand maybe they do exist, did you look?

FTA: It takes more transistors and the packaging of DRAM is more regular. Even if you are willing to pay 6x as much for the needed transistors, you would also need to have your memory chips be much large physically, which, I assume, scales up the cost quite a bit.

In addition to being larger and harder to read data out of, as well as having far far lower yields, it uses easily several times more power to keep on than DRAM does.

Especially if you run it at speeds that would make it comparable to cache.

Well, part of the faster-than-DRAM is because the cache is limited in size. Addressing 512 MB (and keeping it close enough to the CPU) would require a lot more address lines/multiplexing and CPU die real estate [1] that it would be prohibitively expensive and not as fast.

It'll still be faster than DRAM so you still have a point.

[1] In this die-shot (http://en.wikipedia.org/wiki/File:Barcelona_die.jpg) of AMD's Barcelona (4 core) all the regular grid like pattern is the cache (total is about 4MB + 512KB or so.) That takes about half the chip's real estate.

Reply on news.ycombinator.com