Numerical Recipes: The Art of Scientific Computing | Hacker News Reader

Numerical Recipes: The Art of Scientific Computing

numerical.recipes

119 pointsbrudgers50 comments

I think that NR exemplifies how scientific software should _not_ be.

Despite the availability of the source code, already the usage of NR programs itself is seriously restricted, even after license purchasing. Distribution of the software is not allowed (neither in the original of modified versions), which prevents proper verification and advance of computations. Adaptations or copies of the source code (extracted from the software or from the book) constitute a copyright infringement. Academic citations alone do not avoid such infringement. Rather, specific exceptions should be asked (and may not be granted).

Scientific codes widely used in physics often have comments such as 'this is taken from NR' (clearly violating the license/copyright). It shows that what most of the scientific community expects is everyone to be allowed to use/modify/distribute scientific codes, given proprer attribution. In the same way as everyone would feel free to write down and modify the standard model Lagrangian in their papers by citing appropriate references, and to publish their results without any of the above restrictions.

exactly, the cognitive dissonance between code and other research results is striking, sometimes. Then again, this just mirrors the expectations in the scientific community and the capitalist industry.

Could you elaborate? Are you suggesting that something about capitalism prevents industry from taking advantage of research?

it's competitive instead of cooperative

Agreed, I naively bought this book. Then I found out you were not allowed to use the source code in the book for commercial purposes without purchasing a separate very expensive license.

Definitely, recommend you stay as far as way from this as you can. It's my own 1-star review on Amazon.

They don't own the algorithms. If you read their descriptions and turn them into code yourself, you're good. So "this is taken from NR" could actually be perfectly valid.

And they're code doesn't exactly conform to best practices anyway, so if it's small (e.g. a random number generator), then implementing it yourself is a good idea. And if it's big (e.g. nonlinear optimization), you should use some other off-the-shelf implementation anyway, and just use NR to understand the theory behind it, strengths/weaknesses, etc.

From the book:

Copyright does not protect ideas, but only the expression of those ideas in a particular form. In the case of a computer program, the ideas consist of the program’s methodology and algorithm, including the necessary sequence of steps adopted by the programmer. The expression of those ideas is the program source code (particularly any arbitrary or stylistic choices embodied in it), its derived object code, and any other derivative works.

If you analyze the ideas contained in a program, and then express those ideas in your own completely different implementation, then that new program implementation belongs to you.

The code is next to the explanations in the books. Who decides whether your implementation is "completely different", especially if you admit that yours is not a clean-room implementation, since you read the book? Do you want to expose yourself to a legal risk? Or hire a lawyer to determine what the legal risk is?

It is also not that clear all of the algorithms are fully original clean-room implementations written by the authors --- the techniques in the book are well-known and previous implementations do exist, written by other academics. As such, claiming copyright on some the codes is not morally defensible, even if legal.

It's just better to not read the book at all. Since you cannot use the codes as is, if you want to learn about the theory behind it, it's better to pick a different book.

> it's better to pick a different book

In my experience and opinion, Numerical Recipes hits a pretty unique spot by explaining enough about the ideas behind and how the algorithms work, that one can start implementing them (with or without reading the code) but not getting theory-heavy and full of equations and proofs, like a real numerical analysis textbook.

Kahaner, Moler, Nash: Numerical Methods and Software is a similar book, maybe even better, but it's from 1988 and never updated, and it's not nearly as widely known. Also it covers a smaller range of topics.

The point stands, though - NR gives a nice introduction, and allows you to dabble with code and algorithm. But the NR code is didactic, it is not production quality (nor is it intended as such). Together with the restrictive license terms, that means:

For anything to be used in the real world, in production, you're better off taking code from somewhere else.

What alternative to use depends on the topic/algorithm in question. But NR should just be a starting point, not what you actually use.

Yeah, that's how I remember it too. NR was a great reference in the 80s, but then some time (after the 2nd edition '92) things started to change and now they've changed a lot.

First, I think the publishers started to see that there was money to be had as the copyright/left wars took off and the book left academia (where you weren't going to get any money suing them anyway).

Second, as the web grew (late 90s) "programmers" started just copy-pasting "code"/html from wherever they liked without understanding or attribution, and that is something that still exists today... although I hope that more people recognize it as a problem to be stamped out.

So now, NR is somewhat of a relic that oldsters remember as a useful tool, and a new generation see as another symbol of a problem they're trying to get rid of. It's really both.

I remember there was quite some critique of the algorithms used in NR and why you should not use them [1]. Not sure if they fixed the issues in the third edition though. Some alternatives can be found here [2].

        "It is naïve to hope that every computational problem can be solved by a
        simple procedure that can be described in a few pages of chatty prose, and
        using a page or two of Fortran or C code. Today's ambitions for
        correctness, accuracy, precision, stability, "robustness", efficiency,
        etc. demand sophisticated codes developed by experts with deep
        understanding of their disciplines. We have long ago outgrown the
        capabilities of the simplistic approaches of 30 years ago."

===

[1] http://www.uwyo.edu/buerkle/misc/wnotnr.html

[2] https://web.archive.org/web/20021015200910/http://math.jpl.n...

Yeah, even when I was in grad school some 15-20 years ago these recipes were at best "okay" for very basic work. Then again my work required use of non-trivial linear algebra, which (at least at the time) NR's chapters on that topic said "better to use something pre-built".

Even in 1970 Forman Acton was saying 'use someone else's numerical linear algebra'.

This is really just an example of geeks overcomplicating everything. The majority of applications don't require the bleeding edge speed or accuracy, they can get by just fine with this book. One example that comes to my mind is the Lanczos approximation given by NR to estimate the Gamma function. With the coefficients given in NR you can be accurate to seven significant digits or so. While there are better approximations available, seven sig figs is plenty good enough for me. NASA allegedly uses 15 sig figs, so I guess if you are literally a rocket scientist then NR isn't for you. The benefit of NR is that it has a lot of workable solutions in a single volume. I save time by having a single source solution that makes it easy to look stuff up.

TL;DR: Perfect is the enemy of the good. Unless your project actually demands the best possible algorithm, get your answer faster from NR and be on your way.

Seven digit accuracy is only single precision, and if speed-of-implementation is your concern (and you're not in a situation where you have a c99+ libc available), there's a perfectly good license-unemcumbered version in cephes.

I think that criticism of Numerical Recipes is still fair. Most of the topics in this book are the subject of decades of research hundreds of papers. Expecting all this to be condensed down to a few pages with the 'good enough' answer is unreasonable. Before the internet and the wide availability of good software packages (say in the 1980s), I could see the appeal to the practicing scientist or engineer. My boss wants the inverse of a 3x3 matrix -- I'll use Numerical Recipes. Today, anyone wanting to implement any of their algorithms is wasting their time -- excellent tested code already exists and may already have bindings to their language (e.g., numpy/scipy).

The real problem with this book is it is sometimes the first and only introduction to applied mathematics. I first saw it in 2005 as an undergraduate physics major. Later, when I mentioned to my applied math PhD advisor, he was appalled. I was at first surprised by his response, but after finishing my PhD program and spending a few years in the field, I agree with him.

Unfortunately, applied mathematics has not produced a competing generalist manual to the breadth of our field. In part, this is because the field is large and complex; to even understand the basics of modern algorithms requires a level of mathematical sophistication not taught in undergraduate math (see, e.g., GMRES). There are good books that cover specific topics in Numerical Recipes, e.g., Trefethen's Spectral Methods in Matlab and Trefethen and Bau's Numerical Linear Algebra, but they (rightfully) don't provide a pithy answers as to which algorithm to use. These books showcase different algorithms and illustrate the situations where they are or are not appropriate. This latter part requires significant significant sophistication to understand. Implementing an algorithm, easy. Choosing the right algorithm; aye, there's the rub. And that is where Numerical Recipes fails.

I always liked the Boost model (if I'm thinking of the right library), where things aren't just open source, but where there's some expectation of active peer review.

I'm not sure how to incentivize peer review, but that seems like the right solution: to not only put out code that's openly reviewable, but where you create some incentive for people to review it. Maybe that's the point of paid third-party audits, or rewards for identifying bugs, but maybe there's some other options that could be encouraged more?

The GNU Scientific Library is probably a better option, for reasons of both licensing, maintainence & evolution and correctness, if you're writing problem-solving code in C.

https://www.gnu.org/software/gsl/manual/html_node/

GPL not lesser. Link with this only from code that you can and want to licensee as GPL.

I managed to use their code (C version) for Linear Programming - i.e. Simplex in production for an automotive assembly line back in the 90s.

Best of it: I was calling the C module from COBOL...

This book has two problems:

- It's a collection of numerical algorithms and their implementations, not a systematic study of it. Meaning the style is "if you wish to do this, you use that" and so on.

- It developed a bad reputation in the 1990s for the authors disallowing use of the implementations in the book without some kind of contract or permission. I'm not sure if the authors still do that but if you use any of the implementations from the book, you would always be concerned.

A book (not the only one but one that I learned from) that solves both problems is Scientific Computing by Heath (https://amzn.com/0072399104). (Well it doesn't provide implementations in any programming language, but if you understand the algorithms you should be able to implement them yourself).

Finally, I would like to say that numerical-algorithms is the bigger of the two main areas of algorithms (the other being, what I call, combinatorial algorithms), and not only that, it's extremely relevant today for computer scientists in a world where the trend is to go from traditional programming to data science and machine learning. (I'm of the belief that CS departments and book-authors of the world have fooled their students for decades by teaching them only combinatorial algorithms (CLRS, Knuth, Skiena, that sort of stuff), and calling the class "Algorithms". This is a big reason CS students today have difficulty getting into machine learning theory, and CG theory, unless they go to grad school).

Be sure to check the license if you want to use this for anything.

NR performed poorly, in my experience.

Many years ago, I tried using NR for optimization problems. The functions were dog slow, because they tended to set aside space at the start and free it up at the end, and that burns up a lot of time for functions that are called frequently.

The alternative, used in the much-faster NAG and IMSL systems, was for the user to set aside memory before calling any functions, and to hand functions "working memory". This scheme demanded more of the user (yielding bad problems when you gave insufficient memory!) but it delivered much better performance.

For my particular problem, a particular NAG routine (name now forgotten) was almost magical in finding solutions in a relatively small number of iterations. As I recall, the NAG code was closed, so I never knew how the authors got such good results. I imagine the secret was in the realm of clever numerical analysis, as opposed to clever coding.

Relative to other systems, NR had the advantage that you could see the code. Quite often, the code provided examples of how not to do things. But at least you could see the code.

As for matters of licensing, I think they are covered very well in other comments in this thread.

In my experience, there is immense value in NR as a set of curated numerical algorithms. What other criticisms dismiss as "chattiness" is oftentimes an altogether more useful explanation as to the background and rationale for the algorithm's inclusion than your local numerics professor is likely provide.

The best way to use NR, IME, is as a reference that guides you through the huge body of literature to arrive at some specific method recommendations, like a more helpful combination of Wikipedia + Stack Overflow. Use those references to help you understand the algorithm in original form, and where the NR authors have made changes to the algorithm or parameters, it's good to understand why.

I wholeheartedly agree with the parent's description of their code as "how not to do things". This I think is less true from a strictly numerics perspective than it is true of the general structure of their code, naming, interfaces, etc. I would throw much of those aspects out if implementing their methods.

I had a numerics professor on my grad committee ask for NR to be removed as a reference in a journal paper we were preparing, on the basis that it is "not taken seriously" among the numerics community. He wanted the source references cited, i.e., the original books and papers in which the method I was using were detailed. However, NR tied all those pieces together in a very coherent and approachable way, and gave helpful recommendations on implementation details. People steeped in the field don't appreciate the encyclopedic nature of NR, which is not to be lightly dismissed for engineers and practitioners. I have not seen a numerics prof-blessed text in the vein of NR that gets the right level of depth for practitioners and presents enough opinion/judgment on state-of-the-art methods to actually be of value. If such a thing exists, please enlighten us!

In terms of code implementations, I also wholeheartedly agree that if you can find a high-quality vetted library in your language of choice whose license terms agree with your code/intent, use them! That said, there are definitely areas where NR presents better methods than what is in some of these libraries (ODE solvers are one such area I'm familiar with).

Lastly, note that NR has evolved a fair bit over the years. Don't base too much of your opinion on prior versions.

I came across the first edition (1986) of this book on one of my father's bookshelves while visiting last solstice. He'd acquired as a library discard when USRD was shut down in the 1990's (i.e. it was headed to the dumpster) and it had sat since. I knew nothing about it, but it looked interesting.

Last night, I was looking for something to read on my bookshelf and the now my copy caught my eye. But before reading it I was curious about it as a book...i.e. was it rare and so I googled it up, came across the link, and posted it here since its history was interesting. And then I shut off the computer and started reading.

There's a lot of criticism here about the licensing model. While I agree that GNU or MIT might be more utopian, I think the premises of the criticism are largely premised in historical counter-factuals.

The book was published in 1985-1986 (see the Wikipedia article) [2]. There was no GNU license, no BSD, [1] and as best I can tell from a few minutes googling, no MIT license either. It was the time of the Unix Wars that led to our contemporary licensing landscape. The big choice was between public domain and proprietary licensing (and the practices of proprietary licensing were typically a long way from where we are today).

All this is to say that the code in Numerical Recipes was an improvement in the landscape at the time it was released. A person could look at the source code for the price of a book versus the cost of a commercial license or dealing with a black box. A programmer could kick the tires of Numerical Recipes before buying. A paid license also provided an incentive to improve the product and that produced code in Fortran, Pascal, C, C++ and a host of other languages...all with the ability to kick the tires.

It is also worth pointing out that the code in many many books is not released under an open source license. The typical case is for code in books to be published with strong copyright claims and all rights reserved. Even today, the availability of an explicit software license is an improvement over a number of currently published programming books.

Again, there are utopias I'd rather live in, but the licensing of Numerical Recipes is better than most commercial operating systems.

[1]: https://en.wikipedia.org/wiki/History_of_free_and_open-sourc...

[2]: https://en.wikipedia.org/wiki/Numerical_Recipes

blast from the past ! just looked at the page, kind of surprised to see this:

    'Numerical Recipes in Java™!'

i was/am under the impression that it was either FORTRAN or C++ in HPC/scientific computing arena :)

edit-001 : slight clarification.

Over the years, they've also had source code in Fortran 77, Fortran 90, C, Pascal, BASIC, Lisp, and Modula 2. More history: https://en.wikipedia.org/wiki/Numerical_Recipes

There is (at least) a FORTRAN and a C edition. The interesting thing is that it is not a simple translation. The FORTRAN edition is completely (text and code) written using 1-based indexing, the C edition uses 0-based indexing. If I remember correctly the foreword to the C edition discusses this topic at length.

This shows us that the war between 1-based and 0-based indexing was not decided for a long time.

Most languages outside the C family throughout the 60-90's, allowed the developer to select the lower and upper bounds, the Algol family even allows enumerations to be used as indexes.

Still not decided. Matlab, and more recently Julia, also adopted Fortran’s stupid convention.

Can we stop calling it stupid? It's different. I prefer 1-based in Fortran where I deal exclusively with matrices, and 0-based in any more general programming.

Here's Dijkstra arguing the case well:

http://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EW...

I see no argument there, only a subjective statement "0 ≤ i < N is nicer". To me, 1 ≤ i ≤ N is nicer because it is what most people think when they see sequence 1..N. In addition, the last number of the sequence indicates the count of the members.

> In addition, the last number of the sequence indicates the count of the members.

This is only the case for 1..N, not for I..J, when 'I' is not one. In this general case, the number of elements is 'J-I' if using zero-based numbering and exclusive upper bounds, which is more convenient. Zero-base also simplifies index arithmetic with modulo operations, such as when you are mapping between different index spaces.

It's not so much that it's "nicer". It's that it's simpler. In his "Simple Made Easy" talk, Rich Hickey quotes Dijkstra saying "Simplicity is a prerequisite for reliability".

Here's some more grist to the mill, currently on the HN front page:

https://news.ycombinator.com/item?id=13547317

It's not stupid when writing mathematics-related code. Few things are more annoying than having to mentally translate between the 1-based indexing in every formula, theorem and equation to the 0-based indexing in a program.

I agree (based on Dijkstra's arguments, I'm not smart enough to come up with my own) that 0-based indexing is the more logical, general choice, but 1-based indexing is more convenient for about 99% of the code written in Fortran, Matlab or Julia.

It’s usually better to start from zero in mathematical formulas too (in many cases simplifies them, and most of the rest of the time makes no difference). It’s just that mathematics as a field is very slow to change, and the cultural tradition of starting with 1 is thousands of years old.

> It’s usually better to start from zero in mathematical formulas too ...

No, it's not. There is no universally preferred indexing in mathematics. The indexing scheme that is "natural" changes with the particular application.

There are of course instances when 0-based indexing makes more sense [1], [2]. But, there are cases when starting at 1 is more natural, as well [3]. Moreover, there are instances where starting with negative indices is more natural, say with the coefficients of the truncated Fourier series, or with the weights of a finite difference operator or of a cross-correlation operator.

Fortran supports all these conventions, which allows for a more direct translation of the mathematics to code.

[1]: https://en.wikipedia.org/wiki/Geometric_series#Formula

[2]: https://en.wikipedia.org/wiki/Taylor_series#Definition

[3]: https://en.wikipedia.org/wiki/Harmonic_series_(mathematics)

> but 1-based indexing is more convenient for about 99% of the code written in Fortran, Matlab or Julia.

You could make the same argument for octal literals over the scary hex numbers with letters mixed in. 1-based indexing was invented by mathematicians using conventional notation without fully appreciating the problems it causes in computing systems. There is no reason to continue the insanity.

It has nothing to do with not fully appreciating the problems it causes computing systems - many of these notations predates the existence of computing systems or even the though of them. Mathematicians use both zero based and one based systems, sometimes due to mere convention but more often where it is natural to the problem domain. Natural notation has a lot of power to help understand your results. I agree sometimes it is a pain to translate to a physical representation in a computer, but it's not like "just always use zero based" is a good enough answer either.

> Matlab, and more recently Julia, also adopted Fortran’s stupid convention.

Julia, as far as I understand, consciously mimicked Matlab there, to facilitate transition.

Lua also uses 1-based.

The code in the c edition uses 1 for the index origin, via a utility library that provides vectors and matrices that provide Fortran-like indexing.

"Besides, the determined Real Programmer can write Fortran programs in any language."

- Ed Post, Real Programmers Don't Use Pascal

Page 18 and 19 of the second edition of Numerical Recipes in C [1] support your comment. There is also a discussion on the books forum [2] that suggests that all C editions are 1-based and the current C++ editions are 0-based.

This doesn't fit my recollection of having the C edition with a foreword that described how hard and error prone it was to convert everything from 1-based to 0-based. Maybe my memory plays tricks on me. I'm sure I had the C edition and not the C++ edition and this was before 2000.

[1] http://apps.nrbook.com/c/index.html

[2] http://numerical.recipes/forum/showthread.php?t=65&highlight...

My experience with "NR in C" is 2e. There were a number of routines I wanted to run on data stored in native C arrays, so I painstakingly re-wrote the NR functions to use 0-index single dimension C arrays.

At least on bio-informatics, Java and .NET are pretty strong.

I have worked with customers that are heavy users of either IMSL or Alea:

http://www.roguewave.com/products-services/imsl-numerical-li...

http://quantalea.azurewebsites.net/

Mostly used on their in-house desktop solutions for data manipulation.

The Numerical Recipes code isn't really meant for HPC, it's for small scale individual researchers.

Yeah, from the ones I've seen they tend more towards avoiding shooting yourself in the foot with numerical issues than performance. Numerical methods are often like cryptography in the sense that there's lots of gotchas that you really need to be an expert to avoid; NR is meant as a shortcut so you can get something that works without needing to consult a PhD in numerical analysis.

I have been using NR for a long time. At this point I have had to make so many modifications that it would hardly be recognized coming from NR.

Reply on news.ycombinator.com