Ask HN: Develop your own dashboard for metrics - yes or no? | Hacker News Reader

Ask HN: Develop your own dashboard for metrics - yes or no?

11 pointsjoshdotsmith16 comments

I'm wondering whether its worthwhile to put time into developing your own dashboard for metrics. This would be for the kind of internal metrics you likely couldn't acquire from external APIs/services, things like hourly updates of total # of users, # of users with friends, # of users with a profile photo, etc. You could then also measure things like % increase over X amount of time, or % difference of growth rate between X and Y time periods.

For me, this would involve writing some SQL queries, running an hourly cron job, and designing/developing something that makes use of some jQuery charting plugin. I'd also probably like a way to export the data in CSV format. Total time involved is maybe a couple days at most.

Is this worth all the effort involved? Do you have experience with doing something like this?

View on HN

Is there value in doing this? Absolutely. Figure out the core metrics for your business, measure them, pay attention to those metrics, and figure out how to improve them. Going one step further, I recommend creating an A/B test platform for yourself where you can easily measure how A/B tests impact the metrics you care about. There is a lot of value to be derived here, and in the end it is going to be an open-ended commitment.

Is it worthwhile for you to do this right now? Only you can answer that. These things are certainly not part of a minimum viable product. But if you find yourself looking at certain figures over an over again, then by all means build a dashboard.

Pro tip. If you're still small, you can probably query your whole database interactively without any noticeable impact on your site. Throw together a prototype based on doing that that before you start to get fancy with things like regular cron jobs. It is faster to put together, and it is easier to make tweaks. Only after you know you've got the right stuff should you migrate the queries to cron jobs and refactor your dashboard to hit cached data.

Yeah, I'm almost done with what I'd call our MVP (and that's after spending a long time working on a very full-featured site that just wasn't as compelling as the MVP itself), so these metrics are going to help me figure out where to head next.

I'm not sure yet how I'd go about building an A/B platform, but its on the docket of things I want to do.

Also, the reason I'd be running a cron job is to have hourly data for all of these metrics (to look for those changes over time).

See http://elem.com/~btilly/effective-ab-testing/ for my advice on how to put together an A/B platform. Sections 2-4 are the most relevant.

The most important detail is the warning in slide 34 that you should not use the person_id to set the A/B test version. The reason is that if you make that mistake, then you can only run one test at a time, and can't easily drop versions of tests later. On the other hand if you make assignment random, then you are free to run tests in parallel, which is very useful for trying out lots of ideas. (Particularly when you have less traffic.)

Thanks for that!

I actually just signed up with Optimizely today and really love the way they run A/B tests so far. I need to think a little deeper on how to integrate these tests with the rest of my metrics, but it's a really good (and easy) start. While I haven't reached a level of real statistical confidence yet with my first test, I've already been able to increase engagement with a single change on my landing page by 35%. Wow.

clickable: http://www.optimizely.com/

At the risk of coming across spammy, I'm the founder of a web service called Geckoboard [1] that might be useful here.

It integrates with 3rd party services such as Google Analytics, Freshbooks, Pingdom, Mixpanel, Zendesk, Uservoice etc. and we're pushing out new connections regularly.

Where it might be useful for you is that we also have an API that allows you to use your own scripts to get data that's in your database onto the status board. You create a script that outputs XML or JSON in a certain format, add a widget to your Geckoboard that's configured to poll your script at regular intervals (support for webhooks is coming) and the data appears on your dashboard that's accessible anywhere you can get on the web.

We're in private beta but will be offering paid plans very soon. I'll reiterate my open offer to HNers: if you would like to check it out the beta then just use the email address in my profile and tell me your HN handle.

[1] http://www.geckoboard.com

Heh, I actually already use Geckoboard, and I have created my own custom widgets for it. It's an awesome service, to be sure; I plan to continue to use it.

If you revamp your charts at all, let me know, as I'm really trying to get some more intense data into my dashboard.

Great to hear you're using it, thanks!

We're planning a major revamp of all the widgets. In the meantime you can use a custom text widgets (which accept images and inline styles but no JS) to render whatever you want. Drop me a line if you would like any help getting something up.

Is it worth the effort involved?

Yes absolutely it is. I always tell the developers that I work with that the most important thing we can do is figure out a way to "peek" inside our systems. Keeping statistics o "background" metrics and displaying them on a simple webpage using plots can be very useful in discovering problems that would be otherwise impossible to notice.

There are many things which are easy to QA because you can visually see them. On the other hand there are many things which are hard to ensure top notch quality because there's nothing to "see". For example, if your system gets sluggish how many users were viewing your site, what was the CPU usage, memory usage, etc. Having cron jobs or other statistical capturing systems which you can use to store and plot data can help you understand when a problem happens. Then you can look for similar patterns and debug the issue.

I work with a system which is rich in algorithm and we keep lots statistics on every major algorithm. That way, if we modify an algorithm or add data to our system we can see if there's any abnormal behavior. We've learned what acceptable is and how to identify problems by looking at the plots we generate with the captured data.

I wouldn't do anything overly complicated. We use the google chart api, easy to use and very fast. http://code.google.com/apis/chart/

Yes without a shadow of a doubt. These are the most important metrics to look at in fact. It will be helpful as well to identify a (one) core super metric unique to your business as an overal measure.

E.g. Blogger - Number of blog posts per day Flickr - Number of photos uploaded per day

After that once you start breaking down into AARRR, you can start defining more mini metrics to watch and optimise at each stage of the process.

Be careful with data overload. Less is more. Is finding out the number of users with a personalised profile picture meaningful to your business RIGHT NOW? If not, shove it and move on to the next metric.

Less is more!!

Not pimping , but my post on this exact topic was recently on Hacker News , http://contrast.ie

Awesome post: http://www.contrast.ie/blog/evolution-of-a-dashboard-design

Yes. It's worth it.

Don't jump into charting plugins. Just dump the data you need on some admin screens and start watching it. If you need a chart, add it later when it hurts to not have it. Just like any feature, add it only when there is a need for it - your dashboard is an MVP for you.

You can create enough to get started in a couple hours, not a couple days.

I've been building my own in Sinatra. Haven't got everything I want yet but if you're interested in the source, it could save you some time on the design side of things. It's all CSS3/HTML + Basic HTTP Auth. (Screenshot with different color "themes" displayed: http://i.imgur.com/8h9jK.jpg)

Yes it's worth it. We made our own and it's the best thing we ever did. It really illuminates your business and allows you to make better decisions. We used Fusion charts for the charting bit.

Yep. I'm even building my own dashboard that sits on top of the google analytics API to give me quick access to more sophisticated analysis than GA allows.

Just create some excel spreadsheets and import the data into them from SQL. Then you can munge it all you like using pivottables.

Reply on news.ycombinator.com