Current draft of The Art of Computer Programming pre-fascicle 6a [ps]
www-cs-faculty.stanford.edu
www-cs-faculty.stanford.edu
Side note: I've always thought Knuth looks delightfully like a resident of Whoville. https://s-media-cache-ak0.pinimg.com/236x/45/16/e3/4516e3256...
I will not be able to read and understand this book fully and I probably won't even try, unless I'm locked in a cell with it.
Although I've helped build software used by millions, what I did was stitch together things that are too brilliant (or insane) for me to fully understand. And I assume that most of my colleagues did the same.
There are pylons of brilliance on the shoulders of which everyone builds, and Knuth must be one of the strongest.
Thank you Mr Knuth for holding so much weight on your shoulders.
I will add that learning algorithms in the way he presents them has been hugely successful in really understanding working with abstractions. Not just inventing the cheap abstractions of giving names to things; but really understanding how you can abstract some problems onto others and use common tools in solving them.
But I can't possibly allocate so much time reading these (long and complicated) books, because, well, all these problems are solved and implemented by people smarter than myself.
They're in the libraries, codecs, standards or embedded in the programming languages.
Problems I'm trying to solve are further and further away from that deep abstract level - and more in the realms of social psychology, perception, etc. Thanks to Knuth and others, I can move away from the abstract to the concrete, from the computer to the human mind, even if I couldn't possibly explain the fundamentals in such detail as he does.
Even this single section within a single chapter could be used as the basis of an entire course.
The primary benefit of TAOCS is that it's self-contained and all written in the same voice/style. It's much higher quality than a lot of other texts on the subjects it covers, but a given chapter is not necessarily that best text for the given topic.
It would be really cool if there were a "TAOCS" elective offered every single semester, just for the sake of fun. I would love to teach a course like that. I think it would work really well for e.g., an honors program ("to get honors in the major, one option is to enroll in this course for 4 consecutive semesters")
More generally, I think the policy that courses should use TAOCS where a chaper exists for the topic at hand would certainly increase the quality of courses at non-top CS programs (certainly, it would have at mine).
But in general, there are often texts that do a better job at covering a topic. E.g., this is a really interesting chapter, but I think there are probably better introductory texts on SAT (entirely subjective, of course).
Which is to say, if I know the area well then I will probably not use TAOCS. But if I don't then TAOCS is a safe choice.
I agree programmers should be able to read and understand Knuth's books, but I don't think a CS degree is preparation. Would you think someone needs a CS degree if they already have a good understanding of the books?
My complaint with college is that it seems to be mostly fluff. If you show up and pay the bills then you'll get your degree.
> Would you think someone needs a CS degree if they already have a good understanding of the books?
To be a programmer? No. But, at least to me, programming related matters is but a small part of computer science. Which makes Knuth's books such a great resource: it goes into detail in that small, yet quite large, field of programming.
I guess what I'm saying is that an undergraduate CS degree is just a vocational degree. College does not prepare you to read TAoCP. If you want a solid CS foundation you have to self study.
I think my undergraduate CS degree contained exactly four courses that I would classify as pure vocational in nature: the first class everyone had to take was to learn to program in Delphi; a class on functional programming; a class on GUI programming; and a class on object oriented programming. After that, all classes were either:
- mathematics, from analysis, statistics, writing/reading proofs, to loads of discrete mathematics
- formal methods, from proving correctness of programs, modeling, to different methods to analyze complex systems
- theory, from automata theory, language theory, compiler theory, database theory, relational algebra, to complexity and computability
- odds and ends, such as ethics, philosophy, history
Then, each trimester, we also had an engineering project where we would have to apply theory. There were no vocational classes associated. If we had to, or wanted to, use a tool, language, system, we had to learn it by ourselves first. Nevertheless, these engineering projects can be seen as vocational in nature too, of course. Still, that makes for about one vocational class a year (of about 15), and three engineering projects a year.
So, when people talk about undergraduate computer science degrees, I always assume their experiences are not dissimilar to mine.
* writing * accounting * organizational psychology * small amounts of business stuff * lots of EE, digital and analog * how to read and write research papers * how to research * how to prove theorems (important if you are inventing your own) * linear algebra, calculus, stats and probability, discrete algebra * physics I-III, plus 1 semester of experimental physics * chemistry (I,II, no organic) * a bit of philosophy * circuit design * numerical computation
It would be a Herculean task to take all that up in your spare time to the level I did at school (I still learn on my own time, so you have to catch up not just with what I did at school, but everything afterwards). It's 4+ years of opportunity to bump against really good minds. I went to professors and said "I want to learn x". (think of the course catalog as a suggestion!) That got me, among other things a co-authorship with a professor on an academic book. All of that has been relevant to my career except for the chemistry. Even with that, my first job could have used it if I had stuck around more, and it is useful to know if you want to be scientifically literate.
Or, you know, you can punch your time cards, do the bare minimum, and graduate with a pretty useless certificate.
College as voc tech to learn to manipulate the LAMP stack is probably a poor choice of time. If you want to be able to take a job to compute cancer statistics, program robots, build digital interface cards, perform computer vision, simulate the ocean, write aircraft wing simulations, model and research traffic flow, work on medical devices, you almost certainly need more then votech and/or self learning, IMO. I've done most in that list professionally, and friends of mine have done the rest (no rockets in that list, but I was offered a job to do that, turned it down).
There are multiple paths to life, but the chance to just think and learn for awhile is pretty incredible if you can afford it.
THIS IS THE BEST COMMENT YOU WILL READ TODAY, IF NOT THIS WEEK!!!!
Sorry for the all caps, but "truer words were never spoken".
So, anyone reading this who hasn't gone to college or who hasn't yet graduated, heed those words.
You will thank the poster perhaps 30 years later.
Thus, follow your heart- dwell not on what you did, nor on what you do, but listen to what you need. Long after, look back, and see, more clearly, what you need now.
The year being over, I asked my friend if his brother is done with the books. He got back to me saying that his brother asked if I was in 1st year, and that he was surprised when he learned that I graduated and still studying.
I had a chance to talk with him yesterday (well, this morning at 4 AM), we were seven people at my friend's.
I said that not working harder than I did back in college is a mistake I am paying the price for today. I urged him to work harder than he is for he will later regret it with hot tears. I said that right now, he's in a cocoon where nothing is expected of him except to study, which he should do vigorously, relentlessly, and constantly. He has no other obligations or pressure other than to learn and become excellent. I said if I was asked to kill ten people to be able to go back a decade, I would kill an additional two to show how thankful I am for the bargain.
I tried to convey the message because he didn't fully grasp that time wasted is never, ever coming back. That once it's gone, it is forever. That it is the only thing we own only for its very duration, not a second more.
I hope he understood. There was a fleeting expression of realization after the chuckles that I hope will grow.
There are a lot more problems that need to be solved at higher levels these days and many of them aren't even in computer science. That's were I spend most of my research time - the coding/cs stuff is easy now.
The point I was trying to make was that Knuth is one of the granddaddies of this whole reality that we populate and that was my attempt to express my gratitude.
I'm infinitely curious about everything, but at some point in life you realise that you can't go too deep in all directions, because it's a fractal of knowledge out there, so some things you just have to trust to be true and move towards new grounds.
I have one of the first printings of the three first volumes but have not read all parts. I did do a deep dive in the floating point, as we at one point had to write that at MWC for an early compiler for the non-8087 class of computers. (Good work, Tim M.!)
Seibel: Have you read Knuth's, The Art5 of Computer Programming?
Zawinski: I haven't.
Check the book.
Not sure what book I have in mind, then. I know there was one where they focus for many interviewees was whether or not they read other people's code.
And, I should add that I don't see it as a failing that people don't read some literature. I do think it is more approachable than it often gets credit for. To the point that I wish I had tried to read it years ago. That said, people that are being obviously productive should remain so using whatever strategy they have used to this point. :)
Apparently TAOCP has a lot in common with the Wheel of Time.
I have a ton of respect for these huge, decades-long, life-defining projects and the people who undertake them (Robert Caro's LBJ biography is another example.) So the end of this project always receding further into the distance makes me wistful, even if the quality of what gets put out is still excellent.
If you want to frame one of those cheques, get in fast.
Is it reasonable to study the second edition, or should I buy a copy of the third edition? I've been a hobbyist programmer for 20+ years, and I'd like to strengthen my understanding of basic data structures.
It is really worth studying even though sometimes you will be discouraged when answering some of the problems in the book and that would be normal. In fact, reading Donald Knuth's book is difficult but you should not feel bad about it.
Here is a quote from Peter Seibel's book, Coders at Work:
"I try to give the key ideas and I try to simplify them the best I can, but then what happens is every five pages of my book is somebody's career.
In other words, there's still so much more beyond any five pages of my book that you can make a lifetime's worth of study, because there's just that much in computer science."
What I'm saying is that there's plenty of literature on SAT Solving already, and some of it is probably better.
You need contributors who can dedicate enormous amounts of time, a good wiki system (most can't cope with automatic cross-references to equations or sections, and don't provide special formatting or indexing for exercises, proofs, etc.), and editors who can combine the efforts of the contributors and produce something coherent.
I'd like to see software that can handle this kind of text. ScholarlyMarkdown[0] is trying to do it, and something programmable like Pollen[1] would be easily extended to support TeX-like features.
The software won't solve the time sink problem, though. Knuth is like a monk, sitting in his room pushing out manuscript pages. Who else is willing to dedicate the time?
[0]: http://scholarlymarkdown.com/ [1]: http://pollenpub.com/
Creating a quality computer science wiki with quality editing seems like a fine thing to do but "TAOCP as wiki" is just a contradiction in terms. You will invite certain comparisons that make you look foolish from the beginning.
I would love to contribute to a new, more open TAOCP that strives to cover a wider range. But I think it will be difficult to organize this.
gswin64 -sDEVICE=pdfwrite -o fasc6a.pdf fasc6a.ps
You can get ghostscript here: http://www.ghostscript.com/download/gsdnld.html and you will need it on your path.
If you want nicer fonts you can download your preferred latex distribution (miktex is easy to install but big) and your preferred perl distribution (ActivePerl maybe). Then you can run pkfix (which you may need to install) on the ps file like this:
pkfix fasc6a.ps fasc6a-pkfix.ps
Finally you can run
gswin64 -sDEVICE=pdfwrite -o fasc6a-pkfix.pdf fasc6a-pkfix.ps
Which will give you a pretty nice pdf.
Proponents of RSA, DHE crypto - take note.
“At the present time very few people believe that P=NP. In other words, almost everybody who has studied the subject thinks that satisfiability cannot be decided in polynomial time. The author of this book, however, suspects that N^O(1)-step algorithms do exist, yet that they’re unknowable. Almost all polynomial time algorithms are so complicated that they lie beyond human comprehension, and could never be programmed for an actual computer in the real world. Existence is different from embodiment.”
But would the existence of such "step algorithms" be fully equivalent to P=NP or a limited case applicable to only a subset of NP problems (i.e. one's thought previously to be NP)?
I don't know the answer and am genuinely curious.
One way to think of this is in terms of galactic algorithms, which are algorithms that incrementally improve the exponent in a polynomial time algorithm, but have constant terms so huge that they're effectively useless, practically. He's suggesting that there could exist a polynomial time algorithm for an NP problem which is similarly useless - but with the additional caveat that we won't even be able to prove theorems about it!
I would be very interested in hearing of industrial/practical applications of those. Anyone with some experience?
I would say that SAT solvers (or SMT solvers, depending on the problem domain) are applicable to almost any kind of problem where the first thing that comes to your mind is implementing a brute-force search using backtracking. SAT solvers implement lots of tricks that are likely to leave naive backtracking algorithms in the dirt (knuth's chapter covers lots of them)
One silly example I like is the time I used a SAT solver to find the solutions for a little Flash puzzle game.
https://github.com/hugomg/hexiom
My favorite part is how I could add some symmetry-breaking predicates to greatly speed up the solver on highly symmetrical problem instances. Those instances are very hard for hand-written algorithms and a SAT solver let me work around the symmetry problems in a highly declarative manner.
Algorithm W on page 79 (WalkSAT) is the workhorse.
Lemma L and Algorithm M page 82 were my favorites.
Also, is there a condensed version of his work that preserves the spirit and rigor?
> Also, is there a condensed version of his work
> that preserves the spirit and rigor ?
I am not sure if this is useful for you, but Sedgewick's and Cormen's books are among more digestible alternatives.
Regarding if it's justified, you might want to look at Knuth's forword to The MMIX Supplement. In short, he says that it's important to have some understanding of all the layers in the computer, and thus learning assembly is time well spent even if you never use it as such.
The problem is...?
On the other hand, it's Knuth and TAOCP, so I can't really complain much.
Books.... that's what our forefathers used in the distant past, right.
"Note: However, I have personally approved ONLY the PDF versions of these books. Beware of glitches in the ePUB and Kindle versions, etc., which cannot be faithful to my intentions because of serious deficiencies in those alternative formats."
I prefer electronic books simply because they take zero physical space in my apartment. I do wish we could fix the issues.
Also, I agree with him! ePUB and Kindle markup/typesetting just don't look as good yet, and they certainly can't handle complex mathematics with any amount of grace.
Pdfs on current generation kindles (300dpi) look great, actually, as long as they are computer-generated. Scanned documents can be hit-or-miss, though.
Can it handle figures too?
[0] https://empslocal.ex.ac.uk/people/staff/mrwatkin/isoc/crank.... (this an an excerpt from a paywalled NYT article here: http://www.nytimes.com/library/national/science/020999sci-ma... )