37signals report card

The specifics of the data are not important here. 37s has a real knack for creating a continuous stream of positive buzz using status updates like these in addition to their blog posts. It's less about their average customer rating or whatever increasing by 3% this year, and more about knowing the company is still alive and doing well with a good product and happy customers. What's particularly smart is that, excluding their books, their articles and reports aren't long or comprehensive, but have the minimal amount of information to offer enough value to keep them relevant and in a positive light -- like when you kick your feet periodically in water to stay afloat. It's not easy by any means. I think it's quite clever.

I am not sure if average page load times are a good way of measuring site performance. I wish they published numbers like median, 75th percentile and 95 percentile load times.

Please see here for why average is not a good metric. http://zedshaw.com/essays/programmer_stats.html

It is especially dangerous for 37signals to do this because a large percentage of web developers look up to them for best practices.

"92% gave us the highest Customer Service rating."

This seems somewhat disingenuous, considering that the rating scale was only out of three, and the "highest" level is used to show any satisfaction whatsoever. Their copy implies some sort of rigor to their rating process, and suggests that the "highest rating" is somehow significant and indicative of inordinate quality, as would be implied by a five or ten out of ten. A three out of three when the bottom two ratings indicate dissatisfaction or displeasure isn't really saying much. I'd think that it's fair to assume that the majority of most customer bases like the product in some way. A more realistic description would be to say that "92% of our customers were pleased with our service."

It's like saying that 92% of students in a pass/fail class earned the highest grade. It might technically be correct, but it doesn't mean anything.

While there has not been much downtime, I have noticed a big spike in their Twitter account apologising for some bug or other.

In fact, for last month or so, every second and third day, I see a tweet from them in my timeline saying "Sorry for disruption in ________service of <product>"!

They continue to be a great example of a company that tries to communicate when something is an issue (or is going great).

It seems like the critics of which metrics are highlighted are missing the point... the average person who uses their products or plans to sees a company who is willing to communicate how they are doing.

If you need to dig into more detailed stats, they're not doing a good job with how their service performs. The whole point of their services is so that you don't need to worry about those things :o)

I think one useful metric to track would be similar to the stock market's "52-week high." As an end user, comparing to the last month isn't very useful. But knowing, is this the top mark they've had in the last 12 months? That provides richer context.

At https://starthq.com we are working on producing similar report cards for all the web apps out there. For the web app developers out there: is there any other information that you would like listed?

Good to see a mix of quantitative and qualitative measurements in there. While uptime and customer satisfaction are probably correlated, there are probably more factors influencing the latter.

They're making a bigger deal about their company than their products. Am I missing something?

I thought this might be a survey or request for users to fill-out a report card, not an informational page. We tried Basecamp once. It seemed like an expensive implementation of something we could easily setup ourselves. Not sure why everyone is so 'ga-ga' for 37signals...

Which NMS is tracking these numbers?

That paperclip annoys me too much to read the page.