For Donald Knuth, good coding is synonymous with beautiful expression
quantamagazine.org
quantamagazine.org
The only exception is that the books only rarely dive into the variable computational costs due to operating on values of different sizes; heapsort, for example, is only O(N log N) time if comparisons take constant time, while, in fact, as key cardinality approaches infinity, key size grows as O(log N). (Which is why O(N) time sorting is possible.) This kind of thing is growing more important as hardware design becomes more accessible; there are probably more people writing VHDL and Verilog now than were writing any kind of software when Volume 1 came out.
So I think using a high-level language would get in the way of understanding those costs, and even to some extent how to implement the algorithm in other contexts.
If you just want to implement a red-black tree or something, then sure, the MMIX code isn't going to help you. But he also presents all the algorithms in pseudocode, so you can use that.
I suspect that isn't the reason this kind of analysis isn't usually done, though. If you measure the running time of a standard comparison sorting algorithm on real data on almost any general-purpose computer from 1955 to 1985, you will find that, above very small sizes, it is quite precisely proportional to N log N, where N is the number of records. The extra factor of log N doesn't show up. Why is that?
It's because almost all of those computers, even bit-serial designs like the RCA 1802 and the LGP-30, aren't bit-addressable; the smallest amount of data an instruction can manipulate is a character, typically 6-9 bits, but often 32 bits. And almost none of them could manage more than 4 gigabytes of data, because that would have been too expensive. So in practice the average time to compare two separately stored keys does not vary by a factor of 100 or so over the data sets the machine could process, as you would expect from looking at key unique prefix length. It might vary by a factor of 2 to 4, but more often was actually constant. MIX in particular had 5-character memory words, but only 4000 words of memory, so its comparison time for internal sorting is quite precisely constant for any realistic data.
After 1985, caches and massive parallelism complicate the picture a bit, initially for largish machines (though Stretch too had cache, such monsters were exceptions) and finally for almost all types of computers, though perhaps not yet the numerical majority of computers, which are probably mostly 8051s and Cortex-Ms and things like that.
Anyway, back to the issue at hand: assembly language exposes all the computational costs by default, which is what you want if you're going to prove theorems about them, while high-level languages obscure them by default, so more of your theorems will be incorrect. And that's Knuth's primary concern.
That level of single-minded pursuit of excellence is the reason we can learn things from books Knuth wrote in the 1960s that are still useful for programming computers that are literally a billion times bigger than the machines Knuth used as his model. It's also the reason he's not done yet, 55 years later, with what he thought would be a single book, done in a year or two.
I don't think it does in the worst case. I'd bet money you could construct an adversarial input that would force you to look at more of each key than you'd have to in order to get down to "real" O(n log n).
>MIX in particular had 5-character memory words, but only 4000 words of memory, so its comparison time for internal sorting is quite precisely constant for any realistic data.
If we're being precise here, then it is certainly bounded above by a constant. But, this is also the same sense in which there is literally no such thing as a Turing machine (all physical machines, even one the size of the universe, are linear bounded automata, at best).
> Anyway, back to the issue at hand: assembly language exposes all the computational costs by default, which is what you want if you're going to prove theorems about them, while high-level languages obscure them by default, so more of your theorems will be incorrect. And that's Knuth's primary concern.
Except, no, you don't. Both you and I just got through explaining why.
> That level of single-minded pursuit of excellence is the reason we can learn things from books Knuth wrote in the 1960s that are still useful for programming computers that are literally a billion times bigger than the machines Knuth used as his model. It's also the reason he's not done yet, 55 years later, with what he thought would be a single book, done in a year or two.
No, it most certainly is not. I'm assuming you've read at least some of TAOCP. Knuth certainly introduced some mathematics and techniques for the analysis of algorithms in TAOCP, but literally none of the algorithms themselves were first published there. This is not a research monograph. It's a reference. The algorithms themselves were published and proven, including runtimes, before they ever made it into those pages.
And, yes, there is plenty that almost anyone can learn from these books and the subsequent fascicles, but it's nothing to do with the computational model. Not one result in TAOCP contradicts the previously published work. We can learn from it because there is simply so much there to learn from. There's a reason that when Steve Jobs told Don Knuth "I've read all your books," Knuth responded that jobs was "full of shit." [0]
---
[0]:https://www.folklore.org/StoryView.py?project=Macintosh&stor...
OTOH, pseudocode is easier to understand, partially because you don't have to think about some of those extra things that don't actually matter in the cost model. For an expository text (and, TACOP is an expository text), I think MMIX comes down on the wrong side of the tradeoffs.
On the other hand, MMIX is actually a very nice architecture for a programmer to write assembly/machine code for — it has lots of registers, etc. It's more pleasant to write in MMIXAL than x86 Assembly or (I imagine) ARM or even RISC-V. So if you just want that part of the programmer experience, it's a great way to have some understanding at all levels.
I'm used to reading books on physics where the authors (even the Russians) give a bit of background and (usually made up) history but when I go to read a book on something like Finite Element Analysis (For engineers) it feels like the author is literally just dumping his life's work into LaTeX without any thought beyond A->B.
Don't get me started on pages and pages of MATLAB or Fortran code without any indentation, comments or variable names longer than two characters. (This is why I believe writing algorithms in real programming languages is playing with fire in textbooks).
Physics textbooks tend to get "unique" when you get to topics like Quantum Field Theory. Basic Textbooks are usually fairly to the point.
"Advanced Engineering Mathematics" is a good book with a fair amount of history (and written by a mathematician so no Engineering-isms e.g. Notation recognizable to other human beings)
It seems more obvious now that I read it. A lot of programmers out there, including myself, are looking for ways to improve their skills and I never thought to myself to write more programs. And not just programs for the sake of programs, but programs that force you to explain to the computer that you understand certain concepts. This man is and will continue to be a huge inspiration.
The Art of War is unrelated but also form an insightful fella. more focused on adversarial confrontations than Art & Fear though
It is so often recommended that I read it to the end thinking there would be some jewel waiting for me. There wasn't.
Sometimes I get the same feeling when coding, when really in the zone with the magical feeling of flow.
All of the students I knew who focused on making 1 good print every day ended up learning more and making better images than the students who constantly tried to rush.
Even the highest quantity of practice is not necessarily essential. My impression is that its spacing out over time is even more crucial.
I have a 3.5 year old, and I’ve been watching him learn all sorts of skills (learning to understand and use language, walk, run, jump, ride a balance bike, solve logic puzzles, build with construction toys, draw, ...), and it’s amazing the kinds of leaps he will make in balance, coordination, speed, understanding, etc. at some specific skill over the course of 3 or 4 months, even if we only practice the skill for 20 minutes once every few weeks.
Somehow the brain is churning on it in the background, and there are sudden leaps in ability which can’t be obviously explained based on direct practice time.
It may be that in painting the cost of accidental complexity can be reduced greatly but in photography due to necessary post-processing there is less on an opportunity.
That doesn't mean there isn't an opportunity here. Being neither a photographer or painter I am most likely not the best person to comment however!
The students who focused on one good print had a lot of prep work. Getting the final step right (for software, shipping and deploying to production) isn't something we should rush. Learning all the steps which lead up to that final one well makes a tremendous difference
Art & Fear: Observations On the Perils (and Rewards) of Artmaking
by David Bayes and Ted Orland
https://www.amazon.com/Art-Fear-Observations-Rewards-Artmaki...
[0] - https://blog.codinghorror.com/quantity-always-trumps-quality...
Your approach might be different than mine.
https://books.google.com/books?id=TuvOng_Yh6wC&newbks=0&prin...
http://www.innonavi.com/wp-content/uploads/2017/04/David-Bay...
Also, it doesn't pass the obvious sniff test, that a teacher would spend a whole semester giving half of his class a terrible experience that uttery failed to teach them anything, leaving them with "a pile of dead clay" as the author claims.
Nice story though.
I'm also wildly guilty of the analysis paralysis... I did try to set up schedules and goals to iterate but it always die before I did any real progress.
Oh and btw, some dude said Wright bros. won the race to flight because they approached it with low cost engineering/lab mindset, rapid prototyping fashion. Other companies with more resources tried the moonshot approach, did 1 so-called plane in 2 years, failed to fly, ran out of resources and died.
I absolutely believe that SpaceX will land a human on Mars in our lifetime. Not sure how long they’ll survive and whether a colony is possible but they’ll definitely make it way cheaper to send shit to mars/moon than it is today.
Also I’m bullish that we’ll make big breakthroughs in DNA and protein folding/interaction understanding. Building with the same molecular machines of life opens up so many possibilities.
It's specially rewarding to do this exercise with utility libraries, as it teaches you about the nuances one abstraction level below (both in terms of learning what is possible but not exposed by the abstraction and in terms of learning what pitfalls exist). Many times I find that it's more elegant to drop down one abstraction level to accomplish something than to add more abstractions on top due to lack of understanding outside my primary abstraction level.
What kind of programs, not for the sake of programs, can you actually write almost daily?
Outside of work, it's easy to write whole programs. Project Euler, LeetCode, etc. These are the kinds of programs Knuth is talking about -- example programs to illustrate algorithms from TAOCP.
Under that category, I imagine something where the resulting program itself is of any, however minuscule, use to me.
A great way to get into this kind of thing if you're stuck for ideas is to go through the Online Encyclopedia of Integer Sequences [2] and just write little programs to print out the first n entries or whatever for a particular sequence. As other comments have suggested, don't worry about making them perfect, just get the numbers right and then move on to the next one.
[1] https://www-cs-faculty.stanford.edu/~knuth/programs/squarepa...
No dis-respect for Donald Knuth (who of course I found orders and orders of magnitude above programmers such as myself) and no dis-respect to people that found those types of programs interesting, but there are also programmers (like me) for whom those programs are just simple riddles, with almost no interest whatsoever.
They're like sudoku or crossroads, interesting and somehow intellectually stimulating but they don't help shape/change/modify the world around us. And I got hooked on programming when I realised how it could help me change the world around me and most of all how it could help me (and others) in trying to make sense on said world that surrounds us.
The question was about the sort of little programs someone could write daily, and the answer resembles crosswords or sudoku, which many people are in the habit of completing once a day.
Would completing one integer sequence algorithm (brilliant suggestion btw!) change the world around you? No, of course not.
Would it make you a better programmer, though? Certainly it would. Which would mean, when you settle down, crack your knuckles, and start changing the world, you'd do a better job of it.
Your objection strikes me as that of a hunter who won't go to the range, because when he fires his rifle, he wants to put meat on the table!
Depends on the sequence, of course. I hear a lot of people would be interested in a fast algorithm for computing membership in OEIS A000040.
I don't think this is accurate: sudoku and crossroads have deterministic known solutions. Solving them is a mostly mechanical feat and indeed somewhat useless beyond the joy of solving.
In contrast, the challenge of creating these sorts of programs is creative and open ended. It is very well possible that someone will come up with an even more efficient algorithm. Or a much simpler one. Or just a completely new strategy.
As to the usefulness, these little toy programs are a constant source of inspiration for solving actual real world problems. A perfect example of this is Algorithm X. One of Knuth's toys that solves Sudoku puzzles. The same algorithm is easily adapted to solve scheduling problems and actually is a major innovation.
You can try to make text-based games.
You can make a bare-bones browser.
You can write a simulator for ..say.. classical mechanics.
Are none of these interesting for you?
The musician doesn't noodle around randomly or repeat uninteresting exercises with the ambition that that in itself is going to constitute a meaningful and memorable work.
Even poets don't write poems just for the sake of poems — they write because it gives them pleasure, because they are moved by some strong feeling or because they have some irresistible urge to say something, etc. (And sometimes because they're being paid for it, or hope to be.) Anyway, here are some reasons (in no particular order, and not mutually exclusive—there's some overlap) for Knuth to write programs:
1. Because it gives him pleasure.
2. Because he wants to find out the answer to some question — for instance, how many knight's tours are unchanged under a 180° rotation?
3. Because he wants to experiment or gain more experience with some algorithm(s) or method(s) that he's learning or writing about.
4. Because he wants to collect data on their performance, so that as a scientist when he makes a statement like “a program using this algorithm finds the answer for N=13 using only 0.3 billion memory accesses”, it is a correct statement.
5. Because he wants to understand something better (e.g. his program for linear programming where he says he understood the simplex method clearly only after implementing it).
6. Because he wants to "debug" or interactively see what sub-components of some algorithm do.
7. Because he has encountered (or been posed) a hard problem and wants the challenge of solving it.
8. Because he has seen a beautiful program and wants to translate it to his preferred style, as he'd like more people to appreciate its clever ideas. (E.g. his reimplementation of the original "Colossal Cave Adventure" game of Crowther and Woods.)
9. Because he wants to provide this program as an illustration of some idea or algorithm mentioned in his books.
10. Because someone has asked him for the program.
One can easily imagine more reasons. Though at the rate of five a week it would be more than 7500 programs over the last 30 years, he has given a few examples on his webpage: https://cs.stanford.edu/~knuth/programs.html (Many of them are in CWEB so to help people who don't have `cweave` installed, I typeset them in 2017: https://github.com/shreevatsa/knuth-literate-programs/tree/m... — I ought to update the repository sometime.)
Calling his exercises simple riddles is disrespectful. It's like calling doing a 5km run every morning, simple walking. The proof is in the pudding, he has impacted the world in a profound way.
But on a more positive note. Which non-cs programmer heroes exists today? I'm thinking John Carmack.
Although I think that many heroes are hidden behind corporations. Those people who contributed most to Google Search or architectured Chrome. Those people who built systems everyone relies on. For example almost every Java project uses Spring Framework, but I'm sure that it was founded by one or few very talented people. And those examples are many, but not very public or exposed.
Do you guys think programmers don't recognise each others work as much as in other fields?
I don't personally have "programmer heroes", maybe it's because I'm 39 years old and as such I haven't believed in cowboy-like programmers for almost a decade now. I did strongly believe though in what Aaron Swartz used to do and in what he used to believe, i.e. an open internet and open data for almost everyone, but that dream died in the late 2000s - early 2010s and I don't think we'll ever going to get close to anything like that ever again.
Maybe you are your own hero?
His latest fascicles are bleeding edge when it comes to a general algorithm that solves a wide variety of problem instances very efficiently. (much similar to his Algorithm X, but superior in almost every way)
I'd like to add that something that I find interesting and inspiring about programming is managing complexity. That's something that's not as easily explored with programs like the above. But for example, you could try to create as much of a spreadsheet program as possible in a few hours. Do that 4 times starting from scratch. How can you organise the code in such a way to make it easy to read, performant, maintainable, and fast to write? What language features will you take advantage of? Maybe try making a part of a word processor, an IDE, a raytracer, an HTML viewer, an FTS engine, a relational database, etc. Try implementing mvc, or reactive, or two way binding UIs. I would consider these sort of a different genre of poem, that let you explore a different problem space. Maybe this might be the genre that you're more interested in?
For example, here is a Gist with a shell script I wrote that, given a set of Maven coordinates, works with a few other tools to build out a docset for use in Zeal / Dash.
https://gist.github.com/michaeljmcd/5564758537946963e946806e...
I had to dig into some corners of Maven I didn't know well to pull it together and it formalized some random bits of minutae.
Nothing elegant or crazy, perhaps, but it is a bit of coding apart from the normal day job that taught me a couple of tricks.
Whenever I start at a new company, before I get thrown into the heart of things and become "too busy" to write these kinds of programs, I like to write things that automate the boring stuff.
Some recent examples:
- a cli for inspecting, adding comments to, and updating the status of JIRA tickets. because our hosted JIRA instance was god-awful slow and their web interface is a mess. - a tool for monitoring things and pinging me with macos native notifications when things crash or complete.
Was my net productivity positive for writing these tools? hard to tell.
There are some companies full of engineers whose actual job is to write tools like this; I'm not sure what they would do for fun :P
I think there are a lot of organizations that would be better served devoting more resources to the meta work given the labor and infrastructure costs. I would go out on a limb and say the capital expended on the majority of projects I've worked on in my career had a smaller ROI (because it was negative in most cases) than would have been the case had the organization just invested in improving internal tools and processes of their existing systems.
We’d rather direct engineers to improve the effectiveness of a single system by 200% (even though it impacts 1% of revenue), than to impact all (or many) systems by 5%.
Yet, most of the impressive software we admire today was built in the environment you prescribe. C and UNIX, famously, but also git, Go, Rust. I imagine many others.
How much has C improved productivity, over Assembly and COBOL? How much has Borg or BigTable?
The issue, IMO, is that no ambitious manager wants to invest the social capital in defending and advocating for these teams, because they don’t look good until they look exceptionally good.
So a little while ago we needed to log some info to figure out where 2 parallel sets of calculations were going wrong. So I wrote some logging that output the info to a file instead of the main log, and then wrote a simple parser to read them in and display them side-by-side in a nice hierarchical tree structure. It made something that normally takes us a week to work out only take a few hours. It's something we'll never ship to customers, but helps us enormously.
I'm finding myself writing more and more stuff like that these days.
(The Writing Life by Annie Dillard)
ArraysOfConcreteParameterizedTypesAreDisallowed
The body of this class is a proof of this fact.
> “I think that there is a moral to this story, namely that it is more important to have beauty in one's equations that to have them fit experiment. If Schrödinger had been more confident of his work, he could have published it some months earlier, and he could have published a more accurate equation. It seems that if one is working from the point of view of getting beauty in one's equations, and if one has really a sound insight, one is on a sure line of progress. If there is not complete agreement between the results of one's work and experiment, one should not allow oneself to be too discouraged, because the discrepancy may well be due to minor features that are not properly taken into account and that will get cleared up with further development of the theory.”
> Scientific American, May 1963.
By that token, coding should be beautiful first, in elegance of structure and forms. And then we can iron out the bugs.
I tend to agree.
- selecting for the H index, that is, what will be most cited, leading to strong convergence around "consensual" or "popular" research, at a huge detriment to diversity of ideas
- one negative effect is a major push for incremental publication (equivalent of a LOC KPI for coders... it's just wrong). “What did you publish this year? How many times was your name in a paper somewhere?”
- putting funds and institutional support behind people you like as opposed to people with the most promising ideas. That's pretty much the story of string theory, and see how despite the failure they are still "winning" politically, occupying top positions and feeding the mainstream vulgarization, whereas those who "lost" decades ago are still AWOL even as their ideas are freshened up by a new generation, whose future against the institutional pyramid is all but guaranteed.
We might still be there (a situation of total stagnation, since the 1970s, at the theoretical level) by 2040 if something doesn't change profoundly in the way university politics have such a compelling handle on research, thus on our progress in science in general.
My 2cts but I'm parroting, people who now speak up in academia.
What I do know about, ML, is all but dead currently in MIT, Stanford, etc. It's just monkey coding applying whatever the industry likes for short-term benefits (again this "incremental"-only approach, playing it safe with 0.001% gains that are, sure, predictably doable). There's no actual research on "AI" let alone the general notion of "intelligence". Not even in neuro. I fear we've lost sight of the importance of basic research, as basic as it gets in a given field, because industry deals run the show in terms of funding. Yet look what Bell Labs or MIT did in the 1970-80s, surfing on the last wave of fundamental breakthroughs of the 50-60s. These days are long gone.
[1] Knuth, D. E. Things a Computer Scientist Rarely Talks About. Center for the Study of Language and Information, Stanford, California. 2001.
"A person’s success in life is determined by having a high minimum, not a high maximum. If you can do something really well but there are other things at which you’re failing, the latter will hold you back. But if almost everything you do is up there, then you’ve got a good life."
That first sentence is really profound in my opinion.
Take for example Kurt Cobain (whatever you think of him and his music). He had major success in music, but his life was a disaster and not something I'd hope for a loved one.
I do agree that it seems like the best art is created by "tortured artists".
What is life for? There are fundamental questions here and I don't think the answers are obvious as you seem to think.
J. S. Bach had some hardships in his life, but he wasn't tortured.
J. R. R. Tolkien, similarly, was not tortured.
da Vinci's life doesn't seem to have been horrible either.
I think they stand as strong counterexamples to the idea that tortured artists make the best art.
Certainly it would have been a traumatic experience.
Definitely a relevant point. Thanks for mentioning it.
Taking your utility function, your definition of success, from someone else, is only one step removed from being jealous of someone on Instagram.
Think, and judge, for yourself.
But surely also be inspired by others? OP simply said "I like this". I agree we shouldn't blindly follow or resent others, but that seems a stretch from "like".
Please allow people to like and utilize the good ideas of others. Thankfully, we don't have to invent every wheel we use.
It's true we don't have to invent every wheel. But it's also true that we don't have to treat invented wheels as gospel. There's more than one standard for living a good life, which depends on the person.
There will be many concerts, where you are just at 70% of your peak ability.
As a professional musician, you have to make sure your 70%-version is good enough that people gladly pay for seeing it.
Or you just do electronic music, so that's not an issue!
I saw Brock Hampton at a festival in 2018. Hadn't heard of them before, and they were described to me as a boy band so I had low expectations. Their performance was bizarrely wonderful - they were all painted blue and wearing orange jumpsuits. Anyways, they have a DJ with a Macbook and a bunch of rappers. At one point mid-song they have a bar of silence, and I swear I saw the DJ lean forward and hit spacebar to pause the track and then resume it after.
Also, a lot of other live electronic music performances have almost zero showmanship, so there's that too.
(Speaking as an electronic musician)
To expand upon what you said: it's amazing when you take that trick, a kickflip for example, and you're so confident in your consistency, that you apply it to something new. Kickflip to rock to fakie, kickflip down a set of stairs, kickflip to 50-50, and so on. The consistency you gain from that trick opens up the door to a whole new world of combinations.
There's so much about skateboarding, the culture (or counter culture in some cases) around it that translate so well to programming/hacking (and life in general). The trials of learning something new. The acceptance and overcoming of failure. The respect you shown and earned when you're at the park and you or someone else puts down a sick trick.
One of my favorite creative minds in skateboarding, Rodney Mullen, talks a bit about the relationship between hacking and skateboarding in his TED talk. [1]
What's your opinion on this?
As Tevye might say, You're right!
> it's better to leave weaknesses be and instead utilize your strengths
You're right!
> They can't both be right.
You know, you are also right.
- - -
Or maybe this illustrates that overly simplistic "solutions" to the complexities of life are just that... overly simplistic, that is, not solutions.
That's a lot different from having a shallow understanding of some technology you use or not understanding your employers business that well.
Same with Sir mix-a-lot and baby got back.
They have some pretty low minimums.
If you make one great work - you can coast on that forever.
I remember Harlem shake is a thing but can't remember who Baauer is. I know about Sir Mix-a-lot, but couldn't recall any of his tune.
I think you need more than great work to get me remembered forever.
The Beatles certainly achieved that, in year 3000 some people will probably talk about them. I'm not so sure about Vanilla Ice.
Knuth is famously proudly terrible at email and "keeping on top of things". Does that make him a failure? Of course not.
I find this inspiring enough that I am going to clean my toilet now.
(My toilet looks and smells much better now.)
The uniform isn't for "efficiency". For cleaning one toilet, changing in and out of uniform would ruin the efficiency of having a pocket for 409 spray.
The uniform is for fun.
"Jill and I got uniforms that have a slot where the 409 cleaner fits. You go over there and squirt and feel good cleaning the toilet!"
When I read this, my first thought was about how Knuth hasn't used email since 1990.
"I have been a happy man ever since January 1, 1990, when I no longer had an email address. I'd used email since about 1975, and it seems to me that 15 years of email is plenty for one lifetime."
Source: https://www-cs-faculty.stanford.edu/~knuth/email.html
If you're into the math side of this, I'd highly recommend the book "Concrete Mathematics", which Knuth wrote with Graham and Patashnik. I got it to get better at solving problems over at Project Euler, but it's a fun read, like TAOCP. As deep as it is, it's amazingly light-hearted, down to the graffiti in the margins from the students who proof-tested the material in a Stanford course.
You think that's it?
"Now comes the fun part: Buttons to push that will take me from one desktop to another.
Being a control freak, I am not trusting FvwmButtons to find the correct button layout; I'm building it myself. The goal is to have a 64x64 ASClock at the upper right, preceded by four 32x32 buttons that will aim my display at another desktop, all above a 16x128 CPU load display. Geometry-wise, I consider it to be an 8x5 grid of 16x16 squares (although I could have regarded it as a 4x5 grid of 32x16s)."
When I see the picture in the article, I was wondering what kind of editor he is using
Only one of the people has read the book, with even Knuth saying that he hasn't read it. Of course, he is joking.
But this is a fun read.
I have a BS & MS in CS, and most of the material in TAOCP was still new to me. Even the stuff I thought I knew like hash functions was covered in a new depth I never would have imagined.
This would be an iconoclastic route into programming, to say the least.
If you are looking for a more approachable or practical book that gets you programming and introduces some theory I recommend “Classic Computer Science Problems in Python” by David Kopec.
Reading TAOCP to get into programming is sort of like reading a physics textbook in order to build a doghouse in your backyard.
https://www-cs-faculty.stanford.edu/~knuth/fasc2b.ps.gz
I just managed to finish the first Alg. P (plain changes); it was really hard, considering that I didn't understand the algorithm and the description is made for Pascal like languages.
Unfortunately, I didn't learn how to program! I thought that what I was missing was knowledge about algorithms, and that was occasionally right but mostly wrong. Worse, algorithmic textbooks bias you to look for the one weird trick that makes your apparently complex problem simple. But usually that trick doesn't exist, and when it does you usually have to solve the problem the hard way first before you understand the problem well enough to find it. The process of debugging, refactoring, optimizing, and testing that gives rise to the final polished form of a program cannot easily be inferred from what remains. Books like The Practice of Programming and Code Complete were much more helpful, but you can't learn to program by reading books, any more than you can learn to play baseball or win lawsuits by reading books.
I did eventually learn to program pretty well, though I'm not yet a master of the craft like Knuth, Jeff Dean, Rob Pike, Walter Bright, or Norvig. I did it largely by a practice described in this interview: writing new programs every day. I also learned a lot from pair-programming, which taught me both to read other people's code (we had collective code ownership) and to write code others could understand. My main obstacle was not ignorance but perfectionism and lack of practice.
Is there a math book that gives the historical context (what is the exact problem this piece of math is trying to solve)?