Listen to your community, but don't let them tell you what to do | Hacker News Reader

Listen to your community, but don't let them tell you what to do

codinghorror.com

71 pointsmgrouchy22 comments

Also, not all community members are created equal.

I used to be a mod/product lead for a major IT community forum. We had a group of 300 or so hardcore posters (which the ad team referred to as "inventory builders") who we took behind the curtain and showed our prototypes before we released them. They often had some interesting feedback, but we quickly fell into a strange trap:

The changes the hardcore users wanted didn't drive usage -- posting or reading -- when we implemented them.

These guys were already spending every damn spare moment posting in our forum, so they had no more to give and all the esoteric bells and whistles (segmented follow lists, advanced karma scoring, multi-tiered private messaging) they wanted were completely irrelevant to the casual user.

When we shifted to testing changes that drove casual usage, we saw much better results. The hardcore group wasn't against it, generally, except for those petty few who saw the forum as their private playground and didn't like the riff raff wandering in to post. They were just dumbfounded as to why we were wasting time on these "beginner" features.

You don't want to alienate the devoted userbase, but data -- not anecdotal feedback -- is what should guide your decisions.

Recently I deployed a new design for my web app that I thought was much better than the previous.

A few of my users complained and I decided to rollback the changes. Nobody has complained about the rollback but I still believe the new design was better.

I find it hard to know how to decide when to push further against the community sentiment. Facebook has done that countless times and it has worked for them.

I guess this is where the founder's vision comes into play.

How do you decide what to keep and what to ignore?

You have to figure out what feedback is meaningful and what feedback is merely change aversion. If the users have a valid point, then maybe consider reverting. If you believe it's just change aversion then you have to stick to your guns for the greater good.

Just like you said with Facebook, people will gripe about every change. But they gradually will appreciate the improvements as they get used to them.

This is a good point. Sometimes I definitely experience change aversion. But usually if I'm upset about a change it's because there was something that used to take one click that now takes three, or some kind of new complication or a feature that has been completely removed. It's even more frustrating to users if they make these kinds of points and you respond with "Change Aversion!" The trick is knowing the difference (or knowing when some complicated/removed features are worth overall UI improvements)

Yea I know the feeling, I think if you remove a feature without providing any replacement then that can be tough on the users. Outright removal of features is something I try to avoid unless I felt like the software was moving in a directly that wasn't right.

If something takes more clicks because it's a rarely used feature, that may be acceptable if the reason is to improve a more common feature that now requires less clicks.

It's ultimately up to you to decide the direction you are going to take. If you really want a challenge, wait until you do an entirely new port or total concept change after people have been using your software for several years. They often get really cranky about that, even if the new software is miles better.

Show us the redesign vs original design. The best way to tell if a design is "better" than the original is to put both version in front of brand new users who haven't developed usability habits yet.

Sometimes when you make an app the first time, it's badly designed but people get used to the bad design. Then when you improve it they have to relearn everything and that's why they're pissed off.

Another thing you can do is "incremental design". Change things little by little over time.

^ This! My company had a website that predated Google (seriously) when we updated it you can imagine the shock people had after using the same site for like 14 years.Incremental design in this case was out of the question it had to be forced. They will get used to it soon enough.

Remember the famous quote from Henry Ford: 'If I had asked people what they wanted, they would have said faster horses.'

Change for changes sake is not good however there are times where you NEED to change and you have to face the criticism and stand your ground. The trick is to know the difference.

wundergound.com?

FYI - Henry Ford likely didn't say that (although I love the phrase): http://www.quora.com/Quotations/Did-Henry-Ford-actually-say-...

For a good example of this, think Digg vs. Reddit

Do you mean Digg has pushed a big design change and Reddit just does small changes (if any)?

A couple of years ago Digg did a 'drastic' design change and released all of the features at once, and a large amount of the users moved over to Reddit. The Digg design looked nicer, but the function and usability/ease of it changed..

I think this is a great answer.

Option 1: Compare old and new design with new users. Posting the designs in Mturk could get some unbiaised answers.

Option 2: Gradual design. Don't shock your users. Change things gradually.

Thanks Chris!

All change is unwelcome at first. It's unfamiliarity. You have to understand this or you'll fail every time.

The way to get around this is to make gradual changes.

Surely some changes can be bad.

Facebook Beacon was removed after some time for example.

I'd like to know how to figure out what is the fastest to know if a change is for the better or the worse.

In the case of design change, I wouldn't expect a direct change in the metrics, it would impact on a longer term though.

If a vocal minority complain about the new design, but still carry on using the app then you probably don't need to worry. Look at the data.

What data would you look at for a in-app design change?

First, I'd just look at the simple overall engagement metrics that matter most to you. For example, if time spent on your web app is most important to you, then see how that was affected by your new design. You could compare the users who say they don't like the new design with the rest of the users.

It's likely that some specific features or actions will be affected by a redesign, so track that using Mixpanel or something similar, and again compare old vs new behaviour and complaining users vs others. For example, did people stop clicking on some buttons after the redesign? Did they start buying much less?

A/B testing?

I think A/B testing would take too long to show the effect of the design change. Design helps your users like your app a little bit more and I think it would show in retention, but not in sign up conversion.

What metric would you measure?

I believe this is quite true. I also believe that the loudest members of the community (or work, family, politics etc) are the ones you should never, ever listen to. The loudest people tend to get the attention, but I firmly believe you should not indulge them and create a broken system where the loudest, most obnoxious people start calling the shots.

This is particularly why I like Ubuntu these days. It seems they have a vision and are sticking to it. Most current Ubuntu users are going to be quite happy, but there will be a few very vocal community members who are essentially caught in a "who moved my cheese" scenario and they will be quite loud. Unity is turning out to be the best DE in all of Linux and they are doing something different, bold and it is clear there is a unified vision there. Had they listened to the community, they would still be doing GNOME2 for the next 10 years and the status quo would remain...no movement, no growth, no real innovation.

Another community that seems to get it right, as much as I am not a fan of the whole community, is Rails. DHH made some tough calls (Coffescript and not going to HAML and Rspec) and there were quite a few people who hated those decisions, but DHH either had a vision or made a random call (hard to tell in his case) and stuck with it. Seems to be working out for them.

And, lastly, Apple. I really, really dislike Apple these days. I think they are going Evil pretty quickly. BUT, they stick to their guns when everyone else tells them they should be doing something different. Let other people build mac hardware? Open up iPhone/iPad development off Macs? Flash on the iPhone/iPad? App store taking 30% from in app purchases?

Overall, I think the message is this: Have a vision, understand what you are doing and execute to your vision. You might be right, you might be wrong, but you can't please everyone so know what you are trying to do and do it. Hot or cold but not lukewarm.

This is especially important in the era of MVP where you are showing the public the absolute bear minimum and all further progress is done in the public's eye.

Reply on news.ycombinator.com