Programmers Need To Learn Statistics or I Will Kill Them All - Zed Shaw | Hacker News Reader

Programmers Need To Learn Statistics or I Will Kill Them All - Zed Shaw

zedshaw.com

68 pointsjamiequint19 comments

Rant from Zed Shaw of Mongrel Fame

I'm a huge fan of statistics for reporting, but I think his ideas about how to use it in practical development is mostly broken.

Their team needed statistical analysis and charts to solve a 60 second response time bug? That's just a total lack of basic debugging skills.

Eliminating all confounding and spending the time to test your application's performance characteristics in a statistically valid way rarely makes sense. If lives are on the line fine, but in most cases the reason for even attempting it do so is bureaucratic. Release early, release often.

I mostly agree with you, but in the article, the slow component had a subsecond response time with a 60 second deviation.

I think women are better programmers because they have less ego and are typically more interested in the gear rather than the pissing contest

Here's a statistic: 99 percent of all achievements in computer science come from men. But, hey, I'm no statistician, so maybe I'll just trust some off-the-cuff twaddle from random jerks.

I try to show them how to properly graph a run chart and they're indignant.

Just wondering: Is this guy hacking device drivers? OpenGl? Maybe he's tweaking the last bit of sweat out of microsoft's C++ compiler? No...he's clearly doing CRUD work. Performance issues in CRUD work are solved by RAM and memcached, not screwing around in Excel. That's why your co-workers are indignant: You don't understand your trade.

If we want to only verify one single query on a database then what's stopping us? Stupidity that's what.

Because there are 6000 queries, and not all of us can spend all-day-all-night screwing around with Excel.

The classic example of this is the Pet Store

"The Pet Store" isn't the classic example of jack.

Then I hit the DB2 database and about crapped my pants. Almost all of the queries performed great, except one query that had sub-second response on average, but a 60 second standard deviation!

...my own statistics are a little rusty. If the average is .5, and the SD is 60, doesn't that mean that certain runs should be choking so hard that the log file would...just freeze. I mean, break out the Excel if you want, but I've seen log files choke before, and it's usually obvious what the problem is: The entry at the bottom.

"It's not the database, it's IBM's DB2 configuration.

LOL. DB2's configuration being a part of...what then?

The next day we had IBM fixing the problem (turned out to be a single update index command) and we all kept our jobs.

Our hero! (Self-proclaimed)

That's what a proper analysis method can do for you.

Or just watching the log file.

Is this guy hacking device drivers? OpenGl? Maybe he's tweaking the last bit of sweat out of microsoft's C++ compiler? No...he's clearly doing CRUD work

This guy wrote Mongrel, which is a very solid HTTP server - the most common way to deploy Ruby applications these days. Also he was one of the guys on ruby-lang who discovered nasty bugs in Ruby's threading implementation (everybody else just preferred bitching about "unreliable" Ruby threads and memory leaks)

Great. An a-hole with credentials.

Still, he's clearly doing CRUD work in the provided example. It's not my fault that he wasn't on the list, but to show I'm a good sport I added him:

http://en.wikipedia.org/wiki/List_of_programmers#S

> the most common way to deploy Ruby applications these days

Say what? Are you sure you aren't conflating Ruby with Rails?

No I am not. You can use mongrel with any framework you want.

Framework?

Not every Ruby app is a Web app. I've written dozens of Ruby apps, but only some of them run on the Web.

> Here's a statistic: 99 percent of all achievements in computer science come from men.

Ummm....wouldn't that be because ~99% of all coders are men?

Ummm...no

raw numbers

http://64.233.169.104/search?q=cache:_0JxL7T75-sJ:www.bls.go...

versus prominent programmers.

http://en.wikipedia.org/wiki/List_of_programmers

I would suggest if you strip off some of the dubious or ceremonial entries ("ada") the picture gets worse still.

Computer Programmers .... 3.9%

I count 9 women out of about 230 which comes out to...

3.91%

What is the point you are trying to make anyway? Are you going to go on to tell me black people suck at programming to because they don't compose 25% of that list (but do compose 25% of graduates in CS from the stats you linked)?

Get over yourself.

Women comprise 25.3 percent, not 3.9 percent.

fair enough, I misread the chart, but my last statement still stands (with numbers swapped 3.9% black, 25% female). Your point?

His point was that female programmers aren't superior to male ones. Remember he quoted "women are better programmers because they have less ego". Which is a huge load on both counts.

Replying to that by "no, men are better programmers than women" doesn't seem the way to go. Especially in response to a rant about faulty statistics skills. That argument with 99% whatever certainly was extremely faulty (I can't even begin to describe how faulty, it is just waaaaaaay off). It just goes to show that some carefulness with statistics would be a good idea. Anyway, that response basically proved the point of the article.

I'm surprised that nobody has posted a link to the R language archive site:

http://cran.r-project.org/

R is a GPL project; there are free builds for the major operating systems, including a nice graphical environment for OS X.

R is an incredibly useful language -- with only a little bit of effort, you can do sophisticated statistical analyses that would take weeks to implement in other languages. It's also a great tool for developing/prototyping statistical machine-learning algorithms. Highly recommended.

To quote the article: "I really can't blame them since they were probably told in college that logic and reason are superior to evidence and observation."

Albert Einstein was criticized by the physics community for ONLY using "logic and reason" to explain what he was proposing.

Don't be hasty and dismiss half the tools we have because YOU find them useless.

The guy says programmers are shitpumps because they are men who are guided by primate dominance reflexes not reason, and what do the programmers do in response?

Start a pissing match of course.

I saved his page so I can study up. Also, my dick is bigger than yours and I could kick your ass.

My gosh what is this Digg? there seems to be an increase in the number of trolls recently.

Reply on news.ycombinator.com