The Programmer's Brain (2021) | Hacker News Reader

The Programmer's Brain (2021)

yoan-thirion.gitbook.io

140 pointsrzk69 comments

This looks to be an in-depth summary of "The Programmer's Brain" by Felienne Hermans.

I recently interviewed Felienne for my podcast and something a lot of people may not know is that this book was born out of her struggles to teach python to 12 year olds. Out of that struggle also came Hedy. A Multi-lingual learning language that functions as training wheels for Python*.

The whole journey she went on is wild story about rethinking assumptions about programming languages and education and learning.

https://www.hedycode.com/

https://github.com/hedyorg/hedy/blob/main/grammars/keywords-...

Could you drop a link to the podcast episode if it's out?

Yep

https://corecursive.com/hedy-with-felienne-hermans/

The idea of a multi-lingual programming language is so interesting. The fact that there is a lingua-franca for programming is, ironically, both a huge boon for accessibility and a large hurdle. With a single programming language any programmer only has to learn one language to gain access to the vast majority of programming languages and codebases. But if you are not already a native speaker, learning that common language can be a significant barrier to entry. This isn't new, of course, we've had several eras of different lingua-franca in science, for example (Latin, German, etc.).

Looking at it more, Hedy looks like it solves the latter problem reducing the barrier to entry. I wonder if something like this could be used to also keep the benefit of a common language. It looks like it allows you to choose at the beginning what language to use for authoring code, but could you use it's mapping of keywords across languages to allow a reader to choose the language they view the code in? Would that be sufficient? Maybe not because you still have variable names, function names, etc. that wouldn't have such a mapping. But I wonder how far that kind of auto-translation would get you. And how effective AI-based translation of the remainder would be, especially if paired with some sort of AST-based tooling that helps the AI to use consistent naming for the same symbols, etc. Interesting!

My understanding is they can do AST based translation and then use natural language translation to change variables.

So they can switch the language, but its limited to the quality of natural language translation of variable names.

The RTL langs and the different numbering system, from our interview, seemed like the hardest parts she had to tackle.

I wonder how such a translation system would handle variables like i/j counters. If the target language is anything with a non-latin writing system then I can't imagine what you would substitute. I'm sure there's converse examples too, of single-character variables that have no translation to English.

Similarly, I wonder how one handles non-obvious shorthand like "rsp"

Thanks for sharing hedycode. I've been searching for a tool like this to teach kids coding in Ghana. I'm truly grateful.

From the "Programmer's Brain" title, I was expecting something else.

It seems this might better be titled "How to think like a programmer", or "Skills programmers need to know" or somesuch.

I was thinking this was going to be about what programming (for a long time) does to your brain - how a programmer's brain works, and how that is different from how non-programmers think (about life in general). I used to joke that programming is a disease, not a career...

The full title is:

The Programmer’s Brain: What every programmer needs to know about cognition

To me the most common confusion when reading the code is the lack of documented intention behind the code. You can think of deciphering intention from code as a backwards problem, which is exponentially harder then forward problem (checking if given code is actually doing what's intended). Deciphering intention also gets much harder when there are bugs in the code, and you can't be sure if they are bugs or intentional behavior.

This is quite a valuable comment that I agree with 100% and from what I understand, a highly unpopular opinion. I hope more people see this.

The idea that code should communicate intent, whether by comments or other means, is popular and common. It ends up being somewhat difficult to execute. In practice, you have to write code like you write prose. Keep your audience in mind, and be aware of the contextual information available to that audience, and write code that will make sense to your particular audience, ignoring the needs of non-audiences. The tricky part is when your audience includes people in the future. But even how you account for a future of missing design docs and broken links will vary based on things like team composition, business/problem domain, is this open-source or not, etc.

There is consensus documenting why you did something is good (which is what root comment is talking about). Documenting what you did is commonly thought to be a crutch for writing unreadable code.

While I _absolutely_ agree with those sentiments, I have seen nothing like consensus on them myself (in mostly tech startups, but also fintech, and financial (yes, those are different things)). If I limit it to programmers I respect, the percentages go up, but to _maybe_ 75% tops.

I hate the second thought, because not documenting is clearly not stopping people from committing the unreadable code. Instead we get "my code is self-documenting, I'm not going to write documentation".

And as for the third opinion of "the documentation becomes out of date when the code changes", I would prefer slightly incorrect comments to decipher code rather than no comments to decipher code. Doubly so because I can compare the comments to historical revisions.

A related problem I have as an amateur is the lack of structure documentation. I clone a repo and try to figure the code out, and I am met with complex folder structures hosting interconnected source files that call functions among them in ways I have to disentangle. There is usually no high level schematic, no bird's eye view documentation, no discussion of the overall flow of execution or the main code paths.

Why? I don't understand; twenty-thirty years ago, I was taught to provide visual high level schematics and structural documentation. The latter may be difficult, but surely some schematics with notes on intent cannot be too onerous to draw.

I don't think I've ever seen a schematic for a program in all my years programming. Now that you mention it, it really is peculiar given most if not all other schools of engineering have schematics drawn up at some point or another.

To be fair, all other schools of engineering concern themselves with constructing actual physical objects, not abstract intangible mathematical structures.

To be fair, our school of engineering has moved from parsing payroll and accounts payable and is now controlling physical objects and their interactions with the physical world, so, its probably time to start taking things seriously like our peers

That doesn't make the programmed parts any less abstract, though. That the system as a whole contains physical parts just makes it easier to schematize the system as a whole, not necessarily each individual part by itself.

I'm not saying what should be done entirety other than pointing out its become a serious discipline with real world consequences but lots of it is still treated as bofh doing perl in the basement with no checks or oversight or standard process.

And I'm saying there are actual real-world limitations that determine why things are the way the are, and not merely laziness or unprofessionalism.

Isn’t that sort of what a UML diagram is?

I suspect they aren’t used often because most programming is carpentry rather than engineering. Which isn’t intended as a slight… my life would be much less pleasant without furniture.

It's not just a problem when you are an amateur. This is sth that every project should provide.

But there are also many projects which do. Sometimes you need to search a bit for it. Actually I would expect that most big projects have such documentation somewhere in some form.

- WebKit: https://github.com/WebKit/WebKit/blob/main/Introduction.md

- Chrome/Chromium: https://www.chromium.org/developers/how-tos/getting-around-t...

- PyTorch: https://github.com/pytorch/pytorch/blob/main/CONTRIBUTING.md...

- RETURNN (my own): https://returnn.readthedocs.io/en/latest/getting_started/tec...

- Mold: https://github.com/rui314/mold/blob/main/docs/design.md

And then for some popular projects you will also find some independent overviews:

- Quake: https://fabiensanglard.net/quake3/ (and many more on https://fabiensanglard.net/)

- Linux: https://tldp.org/LDP/khg/HyperNews/get/tour/tour.html

- CPython: https://realpython.com/cpython-source-code-guide/

- LLVM: https://blog.regehr.org/archives/1453

One problem is of course that those documents can be outdated and don't go into much details. But they still will give you important insights and should be a good starting point.

This is so interesting. I've heard that the whole "learning styles" thing is basically bunk, but this really reminds me of that.

I don't find the kind of schematics you're asking for to be useful at all. When I do come across them, I see them as busy-work that people had to make to justify starting some project, or because it's a required artifact. But I never think "oh good, here's a useful thing".

Those are honestly my unconscious thoughts on these sorts of things. But I do realize, if I think about it consciously for a moment, that some people must find this sort of thing useful.

But it's just different strokes for different folks! What I want is a description of the goals of a piece of software - what should I expect to be able to do with this? - and an entry-point, and prose documentation on what each component of the code is and why it exists. But a visual birds eye view of what is connected to what is just not how I go about understanding things.

The thing that both of your goals have in common is that they require software which is architected and not just an interconnected mess of everything calling everything else.

If you do have such a mess, you can't really get a good visual overview (it's approaching a complete graph), and you also can't get prose documentation of code components because there are no clear responsibilities.

Well, I agree that better architecture is useful, but I don't think it's required by my "digging into what's going on starting from an entry point" process. I do think trying to write the prose documentation I described tends to drive better architecture. Trying to write the documentation on a module of spaghetti often what helps me realize there is no through line to why all this stuff is in this module.

Learning styles is bunk because it's effectively a practice effect that's self-reinforcing, not because we don't have preferences at all. So the fact that you have not found utility innately in those schematics (per this thesis, i'm just using it to illustrate the learning concept) would mean that you have not had as much interactive exposure with it, and you therefore decline to engage with them. The bunk aspect of learning styles is that you could deliberately engage with them and doing so (again, in a theoretical learning context, not doing work) in combination with other ways of engaging with the code stack would lead to better and deeper learning than just sticking with what you are comfortable with. Plus the added bonus of you improving your use of the schematics for future learning endeavors. So to your point, in learning, "different strokes for different folks" just reflects the methods of learning you have been most exposed to, and is quite malleable!

Maybe, but as I get older I find this sort of stuff increasingly unconvincing. I've had a lot of time to get a lot of exposure to a lot of different things now, and I have more faith in my own sense of my preferences than in "well you just haven't done the other stuff enough!" now.

I think people prefer different things because they prefer different things.

I feel the same about folder structures. Each language, project, and framework seems to have its own conventions about where to put files. And it's often poorly documented.

I think larger open-source projects are more likely to include schematics since they onboard more contributors. Niche projects, or projects with only a few core contributors, are less likely to spend time documenting high-level schematics.

Thank you for putting a name to something I really struggle with. Learning a programming language in school and hacking together quick projects did NOT prepare me for jumping into an actual codebase with all of its internal complexity. "Structural documentation" would be a godsend to me for most of the repos I look at.

On a tangential note, I also wish I had a better understanding of which files are hand-crafted and important to grok and which are just boilerplate that was autogenerated by a script or copy+pasted from some doc. There are a lot of files that are intimidating to look at, but if I talked to the developer who implemented it they might say "Oh you don't need to worry about that, you just need to include that as config for package X".

> you can't be sure if they are bugs or intentional behavior.

I'm in the middle of a huge legacy codebase and I keep asking two questions:

HOW was this working? WAS this working?

I regularly have this discussion when it comes to comments.

The intention of the code is much more resilient than the details, so comments should generally focus on the why.

(It’s also important to document the code that isn’t there: algorithms that were rejected due to performance, “obvious” enhancements that don’t actually achieve the intended effect, etc.)

Maybe it's me, but I can usually guess intention fairly quickly and also imagine what the thought process was and what kinds of mistakes it probably has.

But heck, I'm old and I've been reading other people's code for decades.

And no one ever talks about this. Doing a code review when dropped in cold in SO hard for this reason.

Yes, I can tell what you're doing here, but is that what you're SUPPOSED to be doing?

Seeing all this stuff I can’t help but think that if programming doesn’t just come naturally to your brain such that you would need all this stuff, then you’re probably better off pursuing something that does come naturally.

I had a similar reaction. Also it looks like a lot of the advice pertains to software development rather than programming (e.g. variable/method naming, onboarding newcomers...).

Has anyone here used flashcards to help them learn a new language?

I agree, but I think a more important distinction is that while 90% (wild guess) of people could get it via pure System 2 thinking (perhaps via an unpleasant grind of working through it slowly), only a small percentage can think this way intuitively and quickly in System 1.

I think there's value in having non-programmers (or experts of any kind) having worked through some non-trivial examples as it may solidify in their mind that what the experts are telling them is legitimate (though, this rests on the requirement that what they are telling people actually is legitimate...which I think is is very much the same problem, but more refined). A classic example of this is COVID, but not only the vaccines aspect.

I think this general idea (a much more sophisticated interpretation of it than I'm giving here) has very broad applicability and importance to advanced societies.

not so sure i agree here unfortunately. i think even if something doesn’t come naturally, if you have passion for it and put in work, then you can learn to think in that way. everything takes practice to get good at and although some things may come naturally, at a certain point hard work beats out raw talent

That's assuming that while the passionate untalented person has been working hard at it, the talented person has just been sitting on their laurels, which doesn't really match my experience. People who are naturally good at an activity tend to perform that activity more, since people tend to enjoy succeeding.

An untalented programmer who has worked their way to competency would probably make a good teacher, because they can impart all the things that didn't work for them, but they're unlikely to be as good as a talented programmer, given equal cumulative hours of practice for each one.

Not always. Lot of smart people realize hard work beats talent.

The real question is how much talent you can replace with hard work, because it's a false dichotomy. Not everyone who works hard is talentless, and not everyone who's talented is completely lazy.

You need both discipline and the "stuff" that you're referring to.

The vast majority of the programming workforce doesn't have both.

If you can only pick one, you're better off with the discipline, because you will earn more money and be able to retire earlier, and/or with better living standards.

This books isn't about that at all. It's more about how our brains work and how we can use it to make our code better (read: easier to understand).

If anything, the book tells the exact opposite: you can write your code so that it's naturally easier to understand.

I'm not really sure where i stand on this - as someone in the throws of self teaching programming (C) - and consider myself pretty smart. I've attempted learning on my own several times. Many times since 2016 and have given up on many occasions. CS50x is one of the first pieces of material which has close to what i consider sane pedagogy.

I go back and forth between thinking Python is better for learning because "hey, less typing, less curly braces". Then thinking - great i dont know wtf is happening here - just magic i guess?? (refer XCXD comic).

I have seen this many times in relation to education and quite simply, the way we teach, often "for speed", and "at university" - do not equate to learning anything. Learning how things works for real - takes time, and is often outside the scope of a "semester"...and quite honestly beyond our current paradigms of cordoned off disciplines.

Programming/algorithmic thinking is more or less hardware accelerated mathematics. When i opened up Hammack's - How to Prove it - and read about sets, saw the notation - AHA moment. Similar things happened when looking at Charles Petzold's Code and seeing details of carry bits in counter chips, and comparing those to literal carry pins in gear based mechanical calculators (in Norman Bigg's "Quite Right").

None of this comes naturally (to me) - and i think many of the learning resources are there - but are not there at the time you need them. Common comments "just do a project"..."get out of tutorial hell"...very well meaning...but even a 1000 LOC text editor is an absolute head fuck for someone who just learnt to print a triangle of hashes.

I've reached out to the creator of the Turing Complete PC game on Steam. I'm certain once virtualising a computer which runs C in that game - will we reach the epitome of programming pedagogy :-)

It's not often that clicking on a too-large-for-the-viewport image with small text makes the text smaller and harder to read.

For some strange reason it was assumed that if you clicked the image, you wanted the image to shrink to fit in the viewport without any sort of zoom, readability be damned. Instead of zooming in on the image to reduce cognitive load.

My brain responds in a predictable way when I encounter visual anti-patterns such as this: I close the page.

Book Review: The Programmer's Brain - https://news.ycombinator.com/item?id=30896120 - April 2022 (54 comments)

The Programmer's Brain - https://news.ycombinator.com/item?id=27734473 - July 2021 (101 comments)

Slightly related, has anyone managed to get some “joy of programming” (for lack of a better word) back after starting to learn a new (human) language?

I’m asking because I used to get that in the past (let’s say 10 to 15 years ago) by becoming interested in new programming languages every once in a while, but nowadays I can’t really do a “let’s discover Lisp/Scheme/Erlang” again, and as there are no new interesting programming languages that I know of (with the exception of Rust, which I might consider but it’s not really in my programming interests ballpark when it comes to its use because I’ve never been a C/C++ guy) trying to learn a new actual human language is the closest I can get when it comes to satisfying that intelectual itch.

I found Nim to be refreshing, too bad it's not more popular.

I haven't had the energy to sit down and learn something new in a long time. Its all similar enough and after 10+years of seeing code, I don't imagine any wow moments learning anything.

Maybe you'd be interested in trying to write your own language or something at this point.

The problem you mention is rather common to senior/experienced developers. One way to get back the "joy of programming" is to study the programming language domain as a whole including Programming Language Theory, Formal Methods, Programming Paradigms and Techniques etc. Most people just focus on the needed mechanics of Programming (because job) without really bothering to look behind the curtain and hence quickly get bored/disillusioned. You have to really motivate yourself to dig deeper. But once you do this, you will begin to look at programming languages in a whole different light.

I found the following useful to get back (ongoing) my "joy of programming";

1) Advanced Programming Language Design by Raphael Finkel.

2) Concepts, Techniques, and Models of Computer Programming by Peter Van Roy and Seif Haridi.

3) Understanding Formal Methods by Jean-Francois Monin.

4) Design Concepts in Programming Languages by Turbak and Gifford.

For my part, I find that Literate Programming is the aspect of programming which is most appealing to me.

https://www-cs-faculty.stanford.edu/~knuth/lp.html

I've been trying to maintain a list of published programs/books:

https://www.goodreads.com/review/list/21394355-william-adams...

Huge thank you for maintaining a list of literate programming books!

1) Applying Cognitive Load Theory to Computer Science Education by Juhani Tuovinen et al. - https://www.researchgate.net/publication/250790986_Applying_...

2) Application of Cognitive Load Theory in Programming Teaching by Xiaoqing Li - https://articlegateway.com/index.php/JHETP/article/view/4418

3) Cognitive Load During Problem Solving: Effects on Learning by John Sweller - https://mrbartonmaths.com/resourcesnew/8.%20Research/Explici...

Also see - https://news.ycombinator.com/item?id=40572984

one of the main things I've generalized from programming:

that we must think about everything as if it were two completely disjointed things: what we or the user or the interface does, and how it works... and the potentially completely 'unrelated' reality of what the computer is actually doing

consider for example, git. git has a lot of subcommands we seldom use directly, but all commands we use a lot are always combinations, or rarely scripts, of multiple of them seldomly used git (sub?)commands

the point being, how the mind of the programmer is all about saying and thinking about "git commit" but actually knowing and considering that it really is many runs of "git hash-object" and lots of other git subcommands behind the scenes.... and so on given how there's gonna be syscalls and eventually assembly instructions moving AND COPYING bits all over the computer

I really love this framing of it. I think this is exactly what I spend most of my time doing.

Although I might say there are three levels I'm thinking about all the time: There are the interface and the "what will the computer actually do" levels, but also a really important one in between of thinking about the future reader of the code.

What's interesting, now that you've got me thinking about this, is that I'm not only doing this exercise while programming, but that it's also what I'm doing during all the other parts of my work. While in meetings discussing what to do and how and in what sequence, it's this same exercise of thinking about how we'll need to interface to the human world, and then how we'll get the computer to do what we need, and then how we'll do both those things in a way that will be comprehensible and maintainable.

So I think you've really hit the nail on the head here. To me, this is the job.

Why is it that this book sits on my shelf gathering dust while I devour books on world war 2 naval history?

It vexes me greatly.

Born too late to ply the seas and too early to ply the stars, but just in time to ply the tomes.

Have you read Facing Fearful Odds? It was recommended to me by one of the marines who survived.

Thanks for the recommendation. Just ordered a copy!

Interesting! I haven't read much on Wake island yet, there's so much to read and learn!

> world war 2 naval history?

Mind to share some titles? As I’m also interested in naval history and strategy, and for better or for worse I’ve stopped reading programming books a good while ago.

Also, and for what it’s worth, I found out that good books on military strategy are also really valuable when trying to apply them to many higher-ish-level programming tasks, so in that respect I’m hitting two birds with one stone.

These 2, in this order:

Incredible Victory by Walter Lord[1]. The story of Midway, fairly balanced.

Shattered Sword by Jonathan Parshall and Anthony Tully[2]. also about Midway, but more research on Japanese sources. Has updated information about some things that are taken as "truth" about Midway. Amazing book.

EDITED to add:

James Hornfischer has some great books about the Pacific Naval war:

The Last Stand of the Tin Can Sailors: The Extraordinary World War II Story of the U.S. Navy's Finest Hour[3]. 6 small escort carriers, and a handful of destroyers and destroyer-escorts vs the heavyweights of the Japanese fleet: Battleships and cruisers. Jaw-dropping descriptions of courage under fire. Fantastic book.

Neptune's Inferno: The U.S. Navy at Guadalcanal[4]. Covers several naval battles, but not much on the ground conflict. Really good book too.

The Fleet at Flood Tide: America at Total War in the Pacific[5]. The late Pacific war, with the invasion of Saipan, and the naval aircraft battle of the Philippines, leading up to Hiroshima and Nagasaki and the end of the war.

----

[1] https://www.amazon.com/Incredible-Victory-Battle-Midway-Clas...

[2] https://www.amazon.com/Shattered-Sword-Untold-Battle-Midway/...

[3] https://www.amazon.com/dp/0553381482

[4] https://www.amazon.com/Neptunes-Inferno-U-S-Navy-Guadalcanal...

[5] https://www.amazon.com/Fleet-Flood-Tide-America-1944-1945/dp...

Thanks a lot!

Oh yeah, on the applying military concepts to software engineering: Shattered Sword talks a lot about organizational/strategic/doctrinal deficiencies and I found the lessons to be very relevant to how teams work. I definitely recommend that book if you're any kind of leader or work in development.

You asked specifically for books-- let me add some podcasts:

Check out https://www.youtube.com/@UnauthorizedHistoryPacificWar An overview of the US Pacific war with a professional historian and WW2 museum director, and a retired submarine captain/commodore who served in the Pacific. Lots of high-quality guests, too, often including Parshall and sometimes Tully from bloopernova's list.

Also, https://www.youtube.com/playlist?list=PLaBYW76inbX4kE52CPUvx... Doesn't have the authority of the previous, but is high-quality and covers the non-US portions of the war much better, IMO.

Types should have been included in the examples. That is very helpful for comprehension.

Very cool to see this on GitBook!

This was one of the best books I've ever read. I'll actually go ahead and say this is probably the only book I can confidently say that every professional programmer* should read.

You know how sometimes you can "smell" something doesn't seem right? Or seems a lot harder [to understand] than it should be? Yeah all those little intuitions we develop through experience is explained using "brain science" in this book.

*: More specifically anyone that writes code in a collaborative environment.

Reply on news.ycombinator.com