Review HN: Feedback Army | Hacker News Reader

Review HN: Feedback Army

47 pointspclark22 comments

I thought I'd try out feedback army on my new marketing website. To ensure that what I'd written was in line with what users understood it as.

For $10 you get 10 people to anonymously browse your site.

You give it a URL, and you can ask the users upto six questions, the default questions are pretty great. Along the lines of "what is the site about? how does it work? how do you get it?" etc.

I expected them to give one liners - but holy cow, I got entire paragraphs. They left comments on blog posts, they clicked and interacted with every element.

I guess the "negatives" to this is you can't guarantee quality testers - do I know if they even checked my site?" I used clicktale to track where they clicked, and you can even reject reviews that are nonsense.

I've only got 8/10 "reviews" so far, and they're really excellent. I'll be doing it again before we post it on Hacker News :)

http://feedbackarmy.com

I'm not affiliated in anyway... maybe this was the startup I inspired, thinking about it, I'll have to check Hacker News archives, I just thought you guys would appreciate this service.

too lazy to read: for $10 you get 10 users to beta test your site, and they answer your questions about your site.

Feedback Army was actually inspired by an HN post a long time ago. I wrote (and submitted) some history on it, here: http://killall.dashnine.org/2009/06/how-to-find-new-startup-...

That said, I'm an HN user too. If anyone has any questions you can contact me via the site, call the number, or send a carrier pigeon.

Thanks for the review too. Always appreciated.

P.S. I just found the original thread, http://news.ycombinator.com/item?id=362459 yes pcclark, you and a few others were the inspiration for this project.

you should team up with wuffoo so we can make real (stunning) forms with all the appropriate "out of 5..." style questions.

maybe make that a more detailed survey?

I discovered something new today with vhosts.

I clicked your link to feedback army from your dashnine page, and it took me to AtD:

http://www.feedbackarmy.com./

(Thought you might like to know - click the link!)

I think I'd always choose UserTesting ( http://www.usertesting.com/ ) over this. You get the same kind of feedback, can ask questions/give tasks, and you get to see/hear them go through the site in real time.

I'm not affiliated with UserTesting, except as a customer.

I wholly agree with you on the benefits of watching a user interact with your site, but I find Userfly : www.userfly.com 's delivery model to be more cost-effective. HUGE con against userfly is no user narrative, this proves to be a big downfall when it comes to iterative development, because correlation =! causation.

Yeah, I've used it too. Got great feedback and it only took about 3 hrs to get all 10 responses. Definitely worth $10.

Seconded. Cheap, fast and was blown away by the quality of the feedback.

only issue is people aren't going to create an account and give you the feedback you really need - inside your site. Homepage/etc feedback is easy to come by, by comparison.

That being said I have used Feedback Army once before months ago and received decent to good feedback quality. I guess I just need to ask better questions.

Excellent for initial usability testing.

The only gripe I have with crowd sourcing apps is the inability to distinguish those who write truthfully vs who lie (they might write legibly to get their response accepted, but the response is a lie). It's not a problem with this service, just a nasty side effect of the crowd sourcing and using people.

I've read about this a few times, but never really knew what the quality would be like. It's good to hear it's pretty well done.

I've only used it once previously, but I was also surprised at the verbosity of the responses. (both a good and bad thing)

I saw one example result that said the user came from "Turk". Presumably, the site just generates ten Mechanical Turk tasks that pay, say, 75 cents, and keeps the other 25 cents for itself. I wonder if you could just put the tasks up yourself and get the same reviews cheaper, or if the site does something more than this.

I address this on the FBA FAQ, question 10: http://www.feedbackarmy.com/questions.slp

In short, here is the value add of Feedback Army:

- it's easier to get started with and use - results are available as an RSS feed - FBA worker form now has built-in grammar and spell check courtesy of http://www.afterthedeadline.com

If you don't like a response you get, it's a one click rejection process. Mechanical Turk lets you reject responses too, but it's a few more clicks/pages than one.

You know what else what be a cool service for them to offer? Feedback from experts. Obviously it'd be more expensive, but if you could submit the site to be reviewed by a panel of UI experts, software engineers, etc., I think that'd be a pretty worthwhile service as well, especially for startups.

For the slightly-more-expert take, you might like a site like this (seems like 99 designs but for usability reviews):

http://usabilitytest.com/

Also I saw this service advertised before:

http://www.usabilityfeedback.com/

I've never used either of those, but I have used Feedback Army and can recommend it as a way to get sanity checks with multiple people. Sometimes it's hard to tell if a certain stock photo is "creepy" unless you launch or use a service like FA, so it ends up being well worth such a low fee (for us anyway).

This may come off as corny but - who is more expert than real internet users?

- I suggest that by being an "expert" (whatever that means) you would have too much 'domain knowledge' to know what would work best from the average internet user.

- sounds like the biased sample fallacy http://www.nizkor.org/features/fallacies/biased-sample.html

- I think best idea would be for a/b split testing perhaps using the 'army' to configure your different tests..

What I meant by "experts" is people who could look at your site and give it a quick lookover according to their specialty. Web software engineers might look at a site and see code that doesn't validate, or AJAX that could be optimized a bit; UI designers might suggest that you reorder some of the buttons on your page for better ease of use, etc.

It was vexing that the "3. See results!" link does not point to the example results page. I tend to read stenciled-text graphics last.

Feeding back the feedbackers.

I'll blog my results

Used it before and got great feedback. Highly recommended, but I think when we did it initially it was slightly cheaper. Still very much a bargain.

ah, this was the original "introducing feedback army!" thread (not by me, but I guess I'm inadvertently connected..) : http://news.ycombinator.com/item?id=785489

infinite loop!

Reply on news.ycombinator.com