UserTesting.com - 15 minute video of a user testing your site for $19 | Hacker News Reader

UserTesting.com - 15 minute video of a user testing your site for $19

usertesting.com

90 pointspchristensen27 comments

I've used it and it is good. Highly recommended. The videos you get aren't super great quality but they're useful and the price is right. The only complaint I have is that I'm a tad jealous I didn't start this site.

I wonder if one couldn't put together a software package that Mechanical Turk participants could use to generate similar (if lower-quality) data?

Why?

The main deliverable in user testing is quality. It's much better to have 5 detailed pieces of data than 100 that don't say anything at all.

A smaller sample means a greater chance that it's not representative. If 10% of the population is colorblind (or left-handed, etc), you stand a greater-than-50% chance that a random group of five people won't identify a problem for this segment.

A larger number of available testers means more fresh eyeballs, hence a better chance of repeat queries yielding accurate first-impression data.

Feedback from people whose job it is to evaluate websites may not be representative of the average site visitor. They may be more familiar with common site layouts or have specific pet peeves.

Probably depends on what you're looking for. When you're just starting out a site, just a couple users will point out all sorts of user interface weaknesses that'll keep you busy fixing for a long while.

I think there's room for both. Turkers might simulate the casual browsers who happen upon your site. There could be valuable data buried in what they do in a short period of time.

FYI, I've already done this. I started the project last week and am still testing it (it works though). http://www.feedbackarmy.com/

Visit the site, give it your URL and some specifics to focus on, dump a credit card number into paypal, and subscribe to an RSS feed for the results.

Ha, that's great. I was thinking more along the lines of a piece of software that runs alongside the Turker's browsing session and keeps track of what they see and what they do with the mouse cursor, perhaps augmented by a video of the user's face taken by webcam, if available. But even simple feedback -- as you have built -- is extremely useful.

We've used it in the past, and will in the future - BUT this is not rigorous usability testing of the sort that you'll get in the lab. They're self-selected and they don't do well with loosely defined objectives (even though they tend to be North American native English speakers).

If you have access to a Mac and can recruit your own volunteers (college campus, etc), I'd recommend getting Silverback - http://silverbackapp.com/ . Not only do you get the screen capture, you also get to see frustration/delight on users' faces. Plus, it's cheaper if you plan on doing four or more tests.

I was curious how much the reviewers got. A little digging reveals that they get $10 per review.

I've used it - really helpful. Wish they would send their testers eye tracking monitors - that would rock.

Agreed, but do they not show the cursor?

Also, do they have a "more users, fewer minutes" option? I know I'd rather have 15 users for one minute each, or seven for two minutes each. After that, most users will have decided to stay or flee any ways. It would be good to get a sense of that early decision-making, especially across a statistically-reliable sample.

Tried it out -- shows the user's cursor fine in my experience.

Cool. They could just give their testers instructions to use the cursor as a pointer to where they're looking. Or you could put that in your instructions to them. It's a decent enough proxy for eye movements any ways.

Have you seen footage from an eye tracking session? A user can't explain or point his mouse to where he is looking. The eyes move quickly and somewhat unconsciously.

Can anybody share a video session that they themselves commissioned? I'd like to see how the quality of feedback compares with the "official" demo of tripadvisor.

P.S. Somebody else mentioned eye tracking - I think commercial solutions for this are pretty expensive, but I've seen a few hobbyists roll their own using low cost webcams, though I'm not sure how they fair in terms of accuracy.

P.P.S. And somebody else mentioned just embedding a screen grabber directly onto your website - I'm sure it's achievable but with a browser extension, i.e. Mozilla. I'll have a play around and see what I can come up with.

It's possible from Javascript without extensions. I did a prototype of this, and while streaming the mouse x&y coords to a server was trivial, the hard part was making sure that what the user is seeing is the same layout I'm seeing myself.

For example if the user has a larger font size, then the layout might be different, so those coordinates would not mean what I think they mean. Faced with this I abandoned my own project on this, but I noticed that sites like http://robotreplay.com/ have appeared, not sure how they dealt with that issue.

"Don't Make Me Think" ( http://www.squeezedbooks.com/book/show/9/dont-make-me-think-... ) has a good section on cheap usability testing.

I really like this idea, and I'll probably use this for my own site currently in development.

Have you considered selling an embeddable recorder that people can use right on their site? Why or why not? i.e. I pay you $xx for a google analytics style plugin that I can embed and have any user that wants to provide feedback...

Just trying to understand why you chose to implement as it currently stands.

Once again, really nice.

It's a cool idea. I'm not sure that selling mainly on price is a good move business-wise, though.

There would probably be demand for a more expensive, higher-quality service. Someone else in this thread mentioned adding eye tracking. It also might be useful to add multiple trials from random individuals and create some sort of aggregate statistics. These are both things that you could charge a large premium for.

What is cool about this site from a business perspective is that it is a low-capital, low-cost way to do things with just one person, much like girlinyourshirt.com

It's a cool idea, but I will be reluctant to use it. Sole reason being, the method they select personnas to do usability testing isn't transparent and I'm not sure how much useful feedback will be from those personnas.

I like the idea. I'll probably use it at some point.

How do you guys think they find testers? It's specific to demographic and such, so i think it's beyond mechanical turk?

I love it, will definitely use this.

But please add an affiliate program.

They offer a white label program:

Can our usability firm get a white label version of your service?

Yes, you can resell it to your clients with your logo (not UserTesting.com) so you can use your brand and set your pricing. Email us at resellerusertesting.com and we'll get you started.

I'll do this for $10. :p

Reply on news.ycombinator.com