Zed Shaw should stick to his area of expertise. His angry, opinionated rants on topics he clearly lacks deep knowledge of are perfect examples of the pseudo-intellectual junk that litters HN from time to time.
Also, just FYI: I studied statistics in business school both BS and MS, so most of my background is in applied simplified stats. This includes several graduate courses in stats in the sociology department. In addition I have extensive experience since about 2000 (so 10 years) applying statistics to problems using the R language.
Now, when you write this blog post, I'll expect to see your full Ph.D. level CV laying out all of your experience, publications, and applications of statistics just to be fair.
1) I find your writing style juvenile and distasteful. Maybe some of you yanks actually like that kind of writing. Personally, such writing makes the reading intolerable, even if the content is interesting.
2) Still regarding 1): if you already have the fame, why do you keep writing like a disgruntled teenager desperate for attention?
3) Interestingly, I don't disagree with the points you made in the article, I just don't think that insulting your audience is the best way to make your point. Maybe some people like it. To me, it just sounds vulgar.
4) Statistics taught in undergrad and business school is pseudo-Statistics. I don't mean to be pedantic, but the truth is that only graduate-level, rigorous, Mathematical Statistics counts as "true Statistics". One does not need to know the entire field (that takes at least one lifetime), but one needs to know the fundamentals very well in order to apply them.
5) Mere mortals can only be experts in one or two things. You have programming and guitar. Sorry, but you still are no Statistics expert on my book. 10 years of R is impressive and valuable, but if you don't know the foundations, you're a technician, not a statistician.
6) My field is not Statistics. I am no expert in Statistics, and I am not qualified to write about it. I can write a blog post on Algebraic Geometry, but you would not understand it, so there's no point.
7) Unlike you, I have no interest in revealing my true identity. Unfortunately, there are evil people in this world, and whatever I would write on HN under my real name could be used against me later on. If you're not in that position and can write what the fuck you want on your blog, then I must say that I envy your freedom.
To recap: writing like a teenager does alienate some readers, and calling a non-expert an expert is something to be avoided.
In this and a few other cases, the angry tone is a lightning rod that got more attention than a quiet "please guys, will you learn some stats?" If you have a better way to do it, then do it.
On the other hand, your post sounds angry, gives no insight as to why / how he lacks knowledge (which would have been interresting) and use condescending expressions like "pseudo intellectual garbage"
what could you possibly use from this article? if you're really interested in learning stats concepts and tools you can be better served by blogs with actual content.
here's a good one: http://incanter-blog.org/
Contrary to what many short-sighted "pure" mathematicians think, Statistics is hard. You can't explain statistics in a bunch of blog posts, not even if you are a true expert. You can't condense all that knowledge in a blog post, one must read the books, though painful that may seem.
No, that's a silly approach. If your, say, launching a web app advice like this [sic] is useful. You dont need or want a deep understanding, you just need the pointers to analyse stuff right.
(as it stands I never mind Zed's tone and when I first read this article it provided some useful info to me in a simple format :)
Also r.e. your first post - from what I recall Zed can be considered some sort of an expert on this stuff.
Get this: you need deep understanding of the fundamentals, otherwise you're not doing Statistics, you're doing Voodoo Magic. If you learned some Statistics from Zed's post, then you must know even less than he does. But then, one of the many symptoms of ignorance is the false belief that one knows. Do you even realize how many years one needs to become an expert on any field, especially a tough field like Statistics?
It seems your definition of "expert" is a lot higher than mine; which is fine. Lets use a different word: experienced.
However with that said what Zed wrote in that post stood up to extra research by myself and I actually understood what he had to say.
If you wish to claim it is wrong and he has no clue, fine; but can you please - for our benefit - explain why and specifically where he is wrong?
For me an "expert" is someone who has at least 10 years of experience in one field. An expert on Statistics must have carried out extensive data-analysis work, or must have published original papers. Reading books gives one some understanding, but only when one must solve a problem, does one realize that one's knowledge is utterly superficial.
Zed is no superman. His area of expertise is not Statistics, it's something else. I am not saying he's stupid. All I am saying is that he has not invested the many years of effort into studying Statistics to earn the title of "expert". By calling him an expert, one insults all the true experts out there, the one who do not write blog posts using juvenile language because, you know, they have better things to do... like spending time with their friends and family...
I do think 2 things are happening here though. I think firstly Zed's style doesn't really appeal to you - which is fine, I can see how he wouldn't gel with lots of people. As a result the second thing happens, his content seems irrelevant.
As someone who quite enjoys his approach :) I found quite a bit of useful content. I dont think the point is really to infuse a deep understanding of statistics - it's to correct some common mistakes hackers like us make when playing with stats :)
For example I scanned the blogs of those 2 names you mention and their stuff certainly seems interesting; but frankly I didnt notice anything massively relevant to things I might need to use day to day in my "startup". Zed's stuff I've already taken to heart and corrected some of my "work practices" when dealing with stats.
As to the rest I think it is just a case of differing definitions of expert :) as I said "experienced" is a better word to use - assuming our definitions match :P.
The good news is that there seems to be an opportunity here: write Statistics for the non-statistician. Not everyone who needs Statistics can take 2 years off to learn it. Back in the 1970s, Digital Signal Processing was an advanced grad course... now it's a basic undergrad course. Maybe Statistics will go the same way, becoming more and more prevalent, and less and less ivory tower.
Sometimes it's just fun to read someone go nuts about something like statistics, and it may give you motivation to go learn more about it so that you can understand the whole thing.
But Zed's article, juvenile as it was in tone, had undeniable passion about statistics. And it was enough to keep me reading through the whole article and pay attention to the examples. As it is, I will be doing further reading on statistics and so Zed's article has been incredibly useful to me.
"Maybe it's just me, but Statistics is a beautiful field, and such intellectual beauty should be all the "psyching up" one needed."
It's just you. Nine out of ten people think statistics are nothing more than boring numbers made up on the spot.
If you can't see the beauty in it, you've probably been taught pseudo-Statistics. Don't feel bad. All the Math that engineering students learn is kind of pseudo-Math. All one learns in high-school is BS. If you want to learn something, here's my advice:
i) Don't use modern, over-designed textbooks.. you know... the thick expensive ones with many colors and boxes highlighting the formulas one must memorize.
ii) Instead read the classic books from the 1960s, many of which are published by Dover. They look boring at 1st sight, but their content is rich. Another option is to get the old Soviet books from the 1960s, which tend to be forgotten gems. The Russians are the best at marrying theory with application.