Stephen Wolfram IAmA On Reddit (3pm EST)

Thanks for posting this. I'd read it years ago but had lost it. It's excellent and sums up everything that's wrong with SW.

I'd love to ask him to respond to that article. He won't.

It might be good reading, but I don't think its a good review. What follows is my review of the review, which is rather long, but I've been meaning to write something for a while, so here it is.

The review is filled with a lot of Shalizi saying how bad Wolfram's book is, but when it comes time to justify his criticism he seems either to get bogged down in something quite orthogonal to what Wolfram is actually saying, or to recap some history without relating it to his original claim.

I feel like a good-faith review should at least try to understand what the original author's major claims are, state them, and then weigh them. He hasn't done this. In fact, I suspect he hasn't read very much of the book at all. Of course, he doesn't have to, but then he must refrain on passing judgement on the core ideas, especially when the core ideas aren't actually intelligible to him.

Here's a list of concrete criticisms from Shalizi's diatribe that I was able to identify. I follow each with my own thoughts, as someone with mathematical training who has both read the book and been to the summer school (http://www.wolframscience.com/summerschool/).

0) "Wolfram's main discovery and idea is that "simple rules can produce complex results", something that is neither novel nor helpful."

Actually, Wolfram's discovery is that the very simplest non-trivial programs that incorporates time and space can be arbitrarily complex (loosely translated, 'universal'). Furthermore, after doing exhaustive enumerations of many program families, it is demonstrated that most of them have this property.

Those qualifiers make a difference; because now Wolfram is studying the zoology of programs, instead of the piecemeal taxidermy that was attempted before.

1) "Wolfram's chapter on perception recycles the production rules approach to cognition."

This is a bizarre conclusion. It seems to be taken from about two sentences out of a chapter of around 90 pages. Actually, to the extent that Wolfram offers a theory of cognition, it is closer to Jeff Hawkin's hierarchical temporal memory.

Now, there are several interesting claims and ideas (perhaps wrong) in the section on perception, but Shalizi doesn't have anything substantive to say about them, for them, or against them.

2) "Wolfram doesn't understand evolution. Also, programs can't explain evolutionary adaptation."

The first claim is unlikely.

The second claim is true, but once again Shalizi seems to have misread (or not read) Wolfram. Wolfram doesn't claim that programs are responsible for evolution, but rather than lots of complexity comes 'for free' without natural selection having to incrementally produce it. Natural selection can preserve these useful programs once they have been discovered, but they cannot be constructed piecemeal.

3) "Wolfram is dabbling in toy computer models, nothing more."

It's easy to perceive it that way. I think this is just a side effect of how wide open the space of programs is. It is an orchard so large, with so many low-hanging fruit, that the logical thing isn't to construct one extremely good snowflake model and write a book about it, but to show that even an absurdly simple one can get you further than most numerical models do, and then go on to show the same thing for a dozen other problems.

No particular example is meant to be both novel and definitive. Rather, the whole collection of examples is to demonstrate that a connecting thread of extremely simple programs runs through very many scientifically and mathematically interesting questions, and that a small amount of tugging on each one can yield some disproportionately impressive answers -- even if the models are overly simplistic.

4) "Wolfram fails to define complexity other than visually."

Shalizi misses the point about metrics of complexity, thinking that the lack of a single definition of complexity is a bad thing. Instead, it represents a new approach. Let me explain in more detail.

The basic premise is that any particular (practical) complexity measure boils down to using a particular program to shortcut or predict the evolution of a process. Wolfram argues this is both true of hardcore statistical analysis and "folk analysis", i.e., looking at stuff visually.

Recall that Wolfram claims that most non-trivial processes are "irreducible": you have to run them to find out what they do. Therefore, definitionally, they cannot be shortcut by any practical complexity measure. That means that most complexity measures will agree on all the interesting cases: they'll all 'break a tooth' on the nugget of irreducibility and claim them to be maximally complex.

So if the principle of computational irreducibility is true, then we have the conclusion that devising complexity measures (while a fun game to play) is not actually that crucial to whole enterprise of understanding complexity.

5) "Wolfram doesn't understand relativity and/or Bell's inequalities."

I don't know any loop quantum gravity, so I can't comment on how much deeper it is than Wolfram's work on trivalent nets.

Actually, Aaronson's claimed disproof of the network universe really hinges on whether you allow weak, long-range connections in the graph. It's more of a stalemate.

6) "Wolfram tried to use Cook's result with crediting him."

There is no evidence for this claim -- from what I've heard, Cook broke an NDA he signed that required him to wait for the publication of the book to talk about the result, and then broke another agreement by trying to publish it. But ultimately, it's a "he said, she said kind of thing" -- we cannot conclude anything about the character of either of them.

7) "Wolfram gets the history wrong."

No examples of actual errors are provided, although "indefinite examples are available on request". 'Indefinite' is a good choice of word.

8) "Wolfram's style is unpleasant."

I happen to think that most of it is written in simple, straightforward, unpretentious language -- contrary to much academic work. Of course, this isn't an academic work, and wasn't purporting to be written in academic style.

From history, we have two diametrically opposed examples of style for large scale scientific works: Newton's Principia, which was so technical, dry, and austere that it took quite a long time for people to actually start reading it and absorbing the ideas; and Darwin's Origin of Species, which was straightforward, readable, and sold out on the first day.

Stephen Wolfram isn't exactly known for his humility.

Well, it's either true or it's not. I don't see how humility would change anything here.

If they found bugs in TeX, they should have reported them to Knuth. If they didn't want to "bankrupt" him, there could have turned down Knuth's check or just not cashed it, like the vast majority of people who have received Knuth's checks for bug reports. I'm sure Knuth would have greatly appreciated receiving the bug reports (and in fact would still appreciate receiving them now, if in fact those bugs exist).

You seem not to be aware that he's joking.

So he's doing an AMA every two months?