The laws of shitty dashboards
attackwithnumbers.com
attackwithnumbers.com
This annoyed me a little bit. A check engine light is the perfect component for what it does. If it's on it means that something may be seriously wrong and that you're probably too stupid or ill-equipped to fix it.
If it was something simple, easily detectable and fixable it would have its own light, ie the door is open, you're running out of gas.
In this day and age we can have something more informative than a check engine light. Tell me what's wrong, or at least have a "verbose" option I can enable that will let me see WTF it thinks the problem is.
I think the check engine light is a giant scam to get people to take their car in for service when there is nothing wrong with it.
As far as a verbose option, if you pull into an O'Reilly's or other auto parts store they will lend you, for free, an OBD-II scan tool with which you can read out the exact error plus any corresponding sensor data(freeze frame).
If your check engine light isn't flashing, it's actually because your vehicle has detected an issue with your emissions equipment that means that the engine control module is not able to regulate emissions to within EPA requirements. There are various reasons why this could be the case, but it usually indicates an imminent failure.
For example, mine is currently on because it's detecting a lean fuel condition in the cylinder bank when under high load, which means either that my fuel injection system is not able to keep up or that my throttle is letting too much air into the intake manifold. My car is still drive-able, but I definitely need to have it looked at because my fuel pump or injectors could be going out or my throttle could be faulty.
The point is that it's not a scam, it's providing you useful, predictive information. You just need to know how to retrieve it.
So, for a trivial cost you can get your verbose option. Most people are not going to have any knowledge or desire to have this information, so it's a silly feature to include.
You will need an Android phone already, but these days that is pretty common (particularly for technical people). Just get the error code and Google it, it becomes pretty obvious what it is.
I know my 2007 Lexus has those programmable options like beep durations and such, but I have only found two solutions - paying the dealership $60 to change them, or buying a $15 android app that changes them. I could do it myself, but the codes (Lexus Personalized Settings) are secret. Did you find a free alternative to modify your Prius settings?
http://priuschat.com/threads/geniii-prius-custom-pids-for-to...
But the short is someone else has done the hard work so you don't have to.
Pro tip: As the others stated, they make good obd-2 readers for cheap and you can also go to many auto parts stores and they'll use theirs to give you the code print out for free.
The point that I was trying to address is that if there is a critical warning that you can't do anything about, there's no more info needed than 'stop and seek help' which is perfectly served by a simple light. Or do you want a 1 page problem statement to flash up on the dash and explain why you really need to pull over. The only reason you would do such a thing is if you wanted the user to make the evaluation of whether or not it is important or something that can be ignored.
Also the note about the tachometer struck me as a little odd. The tachometer can tell you if your vehicle is running at a high idle, if the bumps you're feeling are misfires or the road, and it can also tell you whether you've accidentally left your vehicle in 3rd instead of drive after coasting to a stop. Finally, the tach can be used as a raw assessment of the load you're putting on an engine and you can optimize your driving habits according to that. It's not useless information and I much prefer it to an extra cluster of dummy lamps for all that stuff.
Perfect example of this are the people who drive with an automatic and somehow manage to stay just below a shift point much of the time, meaning that the engine is at a higher RPM than it needs to be and could be consuming more fuel and wearing faster as a result.
Only that's a BS reason, because you could always have the "simple reliable" light AND a more detailed explanation in a panel next to it.
I could tell when it was idling high (for example on cold days) simply because I had driven it a lot. I didn't need a tachometer for that.
If your car is misfiring the check engine light should come on. These days a car misfiring can have serious consequences...
The previous owner of the above mentioned 4Runner ignored her check engine light. By the time I got to it, the engine, 02 sensor, and catalytic converter were destroyed. If your check engine light is on, scan the codes and make sure that it isn't anything serious.
Result? Manufacturer manuals ALL OF THEM will tell you: "Just keep on driving, if it doesn't go off after a few days/a few drives bring the car to a mechanic."
Seriously, go get your manual and check what it says.
I mean, what the fuck? How the hell is that good in ANY way, shape or form?
> If it was something simple, easily detectable and fixable it would have its own light, ie the door is open, you're running out of gas.
I've been in new vehicles with a single light for "low tire pressure". But it wouldn't tell you which tire. Sure, I can discover this by manually reading the pressure on all the tires, but the system has to already know this information. It's just incapable of displaying it.
The average consumer can just take it to a tire shop and let them figure out which tire has an issue.
Most technical articles are written by someone blinkered by their specific experience which they feel is so wonderful that they should share with the world.
Save the internet from such dross and write it on a piece of paper, roll it up and shove it up the ares your talking out of.
Writing in the public view is a great way to get feedback on your writing and the subjects you discuss—e.g. corrections, further education, etc. For some people, it also increase the pressure to improve.
Vanity is only one possible reason for writing in the public view, and it's an exercise with large potential gains. Your perspective is entirely off, and you've decided to take an opportunity to attack someone who was either brave enough or indifferent enough to risk being attacked in the first place.
However I didn't come to Hacker News to read a stream of consciousness consisting of the word "shitty" and random screenshots.
By the way, the correct word in this situation is "you're"
I don't have a HN account and bored on a train, its a comment not an published and promoted article I've written, trying to dismiss my opinion based on grammar and typos, hmmm okay you've secured the moral high ground here really well,
You have no idea what your users will decide based on the data you are showing them. But you somehow assume your users will know.
My eternal gratitude to anyone who can tell me what to do with session duration at the hourly level. “People at 4:53AM on Monday stayed longer on the site than at 11:36AM”? So what?
Just because you don't know what to do with the data doesn't mean the same applies to everyone else...
1. How do you measure success? -- or -- How will you know that life is good? How do you know that the sky is falling? 2. If you see this number (something specified in #1) go above or below a certain value, what are you going to do? Anything? 3. When and where do you need to have the info from #'s 1 and 2 to actually take action? i.e.-Do you need to be inside the warehouse? On your phone? At your desk? Daily? Monthly? By the minute? Push?!
1 tells you what to put on the dashboard, 2 tells you how to prioritize the information (something that's important but not actionable will be something that can be found, but isn't prominent or on display above-the-fold), and 3 tells you the form factor/latency that's needed.
I've built a lot of dashboards over the years. Some have completely changed businesses. Some have languished in obscurity. Most are used fairly regularly (at least monthly), but don't actually add much value beyond time savings of having the numbers automated. The ones that really changed businesses and provided some benefit beyond just time savings have had clear answers to those three questions.
I've seen dashboards get tied up in internal politics. The list of recipients of "quality" dashboard is a line in the sand vs the "quantity" dashboard political group. The recipient list and who controls it is far more important than the data contained in the report. Whats important is who reports to who, and why.
A fifth question is does anyone in the chain of command even remotely understand basic statistics like error bars and standard deviations? If the only purpose of the report is to loudly trumpet when pet division B beats divisions A and C thru G, then a very high std deviation / error rate makes stack ranking give the predetermined "correct" result more often. I've seen this personally in "metrics as a teambuilding exercise" where whats actually produced is a weekly report that every division will get to stack ranking win at least once a quarter to meet the Morale Improvement goal on some exec's goal list. Again a PRNG gives better data than real data, if the goal is to give everyone a participation trophy.
Oddly enough, I'm the only one that uses it and I've dropped everything they've asked for from it to declutter.
The purpose of a dashboard is to give instant feedback on something that requires it. (Driving a car, airplane, nuclear reactor, space station, server load) i.e. if something is wrong you should be able to see it right away so you can take action.
The dashboards we see today are more like vanity boards which feed our need for new information but doesn't really serve any other purpose.
A really good read on the subject is this one
http://www.kaushik.net/avinash/smart-analytics-dashboard-mod...
1) http://www.kaushik.net/avinash/digital-dashboards-strategic-...
2) Information Dashboard Design by Stephen Few - http://www.amazon.ca/Information-Dashboard-Design-Effective-...
I can't figure out if he doesn't like the phrasing (instead of "last year," maybe) or thinks the time period is not useful.
Generally speaking, especially near the beginning of the year, I often find "last 12 months" to be far more useful than "this year," which he didn't call out.
That said i've seen "this year" used to mean last 12 months as well...
IHMO, it clutters the options with options that are not useful (do you often look at your 14 days spending?) and already included in other options that are close.
Same thing about 3/6/12 months. Looking for seasonal trends is not stupid.
3 months = a quarter
12 months = a year
New Relic comes to mind, and it's a tool I've found hugely useful.
While it doesn't instrument full system monitoring (though it's getting there), it provides numerous system and site metrics, monitoring, and a useful degree of logging, that's hugely useful.
One of the biggest gains for us came when it implemented JVM heap monitoring. This is possible through jconsole, but jconsole is a steaming heap which if it were actually made of sht would be useful as it might provide fertilizer. It's a Java app itself, has no persistence, must be running to tell you what you need to know, presents its own security vulnerabilities (if you can attach jconsole to your JVMs other JDK hacks can as well), and more. Given the critical nature of heap and GC operations to site performance, having the insight through NR, and not having to rely on desktop jconsole sessions (for each member of the admin team individually, oh yeah, forgot that one) was a huge boost.
And the NR team both understands the tech they're monitoring and works with clients. So many of the stats provided are* actionable.
Wily has dashboards about overall application stats that you can configure, as well as dashboards (?) on a per application level. So I can take a look and see the threads on one JVM are getting stuck and also some SQL statements/service calls are taking longer then expected on that same JVM/overall application. Also useful in dashboards on a per datacenter level to help figure out what's going on.
I'm genuinely curious, because I don't know any better, how do other companies monitor their applications? Anyone else have any comments about Wily monitoring?
Other data are more useful for the marketing folks (e.g., geocoded data), though they can also point out abuse or other issues.
Tachometer has no use? Tell that to professional drivers.
Real time stats are worthless? Not unless you expect a spike in your server load (?) and need to react to it immediately.
If you can't find a use for some number, others may. That's what controls are for. You take the set of data your app (or whatever) has to offer, then you toss in some controls (your job is to make them intuitive so user does actually use them without tears) that can operate on and present the data and then everyone gets to choose what they see.
Bottom line is that the only way you can figure out that something is right for each individual is to give them a choice. The whole no one needs X seems far-fetched (to say the least). Sure you need to talk to users, but they most likely will express different opinions.
Also, don't build more dashboards. What? So we shouldn't improve on our mistakes, right? The concept is so bad that we shouldn't even try because no one (again, assumptions) can build good dashboard. C'mon..
The most critical piece though is: is the data being presented actionable?
Can I do something with it, do I need to do something with it, does it tell me when I need to do something?
Think of the most common dashboard: a car.
It tells you how fast you're going (speedometer), how much fuel you've got, how far you've gone (trip/odometer). Indicators for services or features enabled (lights, hi-beams, turn indicators, hazards). Problems (engine temp, problem lights). Maybe a tach. Automatic transmission indicator, if applicable.
Other chrome is possible: outside temps, compass, and comfort/environment controls (heat/AC, sound system).
But essentially it's a few crucial indicators which either give you current status or tell you there's a problem.
It's not a bad model to start with.
As many of the other comments in here have noted, the design of a good dashboard is often "don't design one, you need something else."
If you've ever run service where you're anticipating a large traffic spike and you need to monitor server stats, real-time statistics are invaluable.
"Will a decision be made in real-time based on this information?"
There are things which automating them would take lots of man-months and tens of thousands of dollars, so much that the effort is not worth it from the savings -- or from any "hidden assumptions" you discover.
I don't believe it is possible, but it seems like an extremely useful feature.
YES! If you're building a dashboard, don't answer the question "what data do I have?" or "what does the brass say this should be?" But instead, get out and talk to users, find out what data is most important to them, think outside the box, throw some different ideas out there and see what sticks with users.
Of course if you are marketing an analytics product, give the customer a realtime dashboard. It's useless but it makes them feel powerful and in control. The news writers would certainly have thrown a fit if we tried to take away their useless chartbeat.
I think many have recognized the demand for dashboards and sprung a cottage industry around it. That is demands often are perverted and sometimes it just comes from an exec wanting to see some "action" or gaining "visibility". They have VC money to spend and will spend money for moving "realtime" colors on the screen.
For the dashboard creators, that is all they need. If someone buys is it. They will keep making it.
On other hand, to disagree with the author. "So what?" People want shitty realtime moving colors because they look cool. Heck, have you seen the crap people pay for in app stores, farmville type games on Facebook and so on. One can criticize the providers and consumer of that crap. Yet they are happily transferring money and product between each other.
Yup. I can't even begin to imagine the amount of time lost on executive or PM useless whims (and how much good could have been achieved with that time and resources).
Sometimes we just want a moving graph that looks important, so the boss knows we're working on something.
While the OP hits lots of points squarely, I strongly disagree about the "no one needs real time". In particular, any service that does onboarding or signups would be really well served to track new users through the getting started process.
I live in the custom dashboard that I built for my startup.
Screenshot: https://www.evernote.com/shard/s16/sh/c8cdeadc-643d-4028-b58...
It tracks every single signup from provisioning through to successful setup. It lets me easily see if people are flailing trying to get things working and if it looks like they are I send them a personal email like: "It looks like you might be having some issues with picking an email approver address, can I help?"
Having this real-time insight into customer issues lets me provide much better support and from an ROI basis is incredibly worthwhile.
The actual service: https://addons.herokuapp.com/expeditedssl
This is probably the only reason many dashboards exist. They're not there to be useful, but to provide proof that the gears are turning behind the system.
This is so arrogant. If your customers are not interested in target demographics then ok. Generalising this to everyone is not ok.
That said, I have an amazing PM who has customized the living heck out of our TFS Dashboard such that it is useful.
Most fun of all is seeing our "daily bug resolved as fix rate" and "daily bug incoming rate". Seeing them as flat numbers in boxes is, IMHO, more useful than seeing them as on a graph.
But yeah, the dashboards I see other teams using? The worst is a bug tracking dash that is updated once every 4 hours. During crunch week, it serves to do not but spread chaos and confusion.
Assuming the author meant car, this would be very dangerous (i.e. irreversible engine damage): http://en.wikipedia.org/wiki/Redline
Dashboards are an astrological tool. Here is an elaborate and complicated process you don't understand to generate numbers that are devoid of meaning, to dilute responsibility when you make a decision that turns out to be wrong.
The article misses this point entirely. A good dashboard from the end user perspective can be used as numerical backup for any arbitrary decision at any time. The author just doesn't get it. Thats why the author is confused by the "just keep adding stuff until I can always use it to justify whatever I want to do".
This is how dashboards are used in practice, this is their actual reason for existing. This is why money is spent on them.
The article is like a debunking of astrology, "well see here, based on the gravitational constant and the distance to this orange vs venus, the square results in ..." and the boss replys with "shut up I don't care about reasons I decided to go to war with eastasia and we've always been at war with eastasia and my astrologer always had my back, and apparently you don't, so lets discuss the effect of all this gravitational formula stuff on your career prospects vs backing me up which you're paid as a yes man to do..."
The dirty little secret of the business intelligence / dashboard industry is that no one logs into them.
A daily email helps with this problem, as people tend to read emails, even if its only a glance.
Nothing real time, no dashboards, static data, but customers love it.
1) Don't just throw all the numbers you have a page. 2) Talk to the users to find what they need measuring 3) Don't multiply controls and options, show fewer specific things 4) Consider what displays will lead to specific actionable insight 4) Don't use specific ranges just so data will change 5) Real time data are mostly gimmicky, don't add it unless it serves a specific purpose 6) Consider if you even need a dashboard in the first place ...
It mentions the OMTM, One Metric That Matters. When you focus your effort to one number, things get much better.
The final key with Tableau is the possibility of endlessly refining the data. So you can star with basic dashboard view, but you can narrow it down to the things that really do matter. It's not a dashboard, it's a data discovery tool when used by experienced analyst and great fun for not so experienced guys too. If we compare it to other popular similar tools like QlikView.
Btw. Many of QlivView dashboards are just those top results of Google Dashboard search.