Survata (YC S12) replaces paywalls with "surveywalls"

"You probably don't need a degree in experimental psychology to understand that forced recruitment gives you invalid/biased/low quality data."

Interesting point. Anecdotally, I am less honest with my selections if I'm taking a survey specifically to get a reward (i.e. contest, Kongregate points, access to a site), as I usually just want to get it done with as fast as possible.

However, if it has tangible results (i.e. will tailor my experience on the site like Netflix) then I am likely to spend more time choosing honestly.

There are cases where surveys are useful, but none of them hinge on getting accurate results. The two that come to mind: push polls, and exploiting sample bias to make yourself look good.

You seem to be evaluating this company based on either A. how it would work for you personally, as someone who isn't interested or B. an inaccurate assessment of how it would work for other people who aren't you. When discussing a company you typically try to avoid both.

For instance, many girls like dolls, but if you don't like dolls that doesn't make dolls a "No." business, and it's not illuminating to hear why dolls are icky. A better discussion would be how and what makes a new doll appealing (or unappealing) and whether it can compete with Barbie.

Clearly the online survey industry is making some money. Certain parts of the web, which may not target you, rely on surveys. Combining surveys and paywalls, if not perfectly novel, is at least interesting. The combination raises a host of issues and potential problems which your comment ignores.

Your being off-base would not quite matter as much if it wasn't so rude to the team that worked on this. Did they do a good job in their implementation? I don't know, it hasn't come up yet.

Thanks for the interest, and sorry for the delayed response.

We certainly hope that the surveys are not annoying. We know that some people will prefer to pay, but others (like me) would rather take a few seconds to complete a survey than pay. Our hope is that the surveywalls will let users access content that otherwise would have been beyond their reach. And at the same time, quality content publishers will be able to make money off their work.

As to the researcher side, we agree with you that nothing beats revealed user behavior when optimizing web sites or apps. But it's not cheap or even possible to A/B test in other situations.

Suppose that you're a restaurant owner and want a new sign for your building. It's not practical to purchase two signs and see how alternating the signs affects business on different days. And it's also not in budget to spend $10k or more on a traditional market research survey.

Or if you're a politician, you can not wait until voting day to see which of your various ad campaigns worked in different districts. You need proxy measurements.

Companies already ask these types of questions using traditional approaches like panels and phone polling. We provide a cheaper way for them to do it online. All approaches have built in biases, and we're working to account for the biases and quality issues in online polls. As you point out, we'll have to do that to succeed.

I'm curious to know what bad personal experiences you've had in conducting surveys to dismiss them completely. When was the last time you ran a survey and why did it fail to deliver the results you were expecting?

Also, please can you take it easy on the tone? See this discussion - http://news.ycombinator.com/item?id=4396747

Edit: Analysing server logs can tell you what people did, not why they did it. At least in my experience, surveying users directly about their motivations and needs has been very enlightening and resulted in a direct increase in sales.

And I think the solid relationships with big publishers is the big part that's hard to replicate.

That said, you could probably more easily approach smaller (more local?) publishers that are more willing to experiment to get traction. Is that what you're planning on doing?

Thanks for the interest. Google Consumer Surveys does have a similar model to ours. There a few things that differentiate us now:

1. Our minimum spend is only $10 vs. $100 for Google; lower minimum brings in a group of people who want to get a "quick read" on an issue (e.g. ask 100 people an opinion on a new logo design or tagline)

2. Starting soon, we'll be offering advanced behavioral targeting. For example, you could select a target audience of active young mothers, Honda car owners, or online shoppers.

3. We're allowing multi-question surveys (up to 4 questions). Each respondent answers all questions in your survey, which enables cross-tabulation of responses (e.g. looking at how respondents who answered "Yes" to question 1 respond to question 3). Google is currently focused on 1-2 questions at a time.

In the coming months, we hope to differentiate ourselves in other areas, in particular data quality.

The article mentions some ways they plan to catch bogus data, but I agree that I'm a bit skeptical they can make it work consistently.

I'll second this, we put together a system that did this around 2008 and while it worked out well initially within less than a year it became a loss for our company. We couldn't charge much for the data as even with large amounts of filters and detection techniques for bogus data and users the results were so bad no one was willing to pay for them any more. Within a year we scrapped the project and ripped it out of our ad system. People are more than happy to answer a survey to get to paid content, they are also more than happy to supply bogus answers to the questions.

That being said, I'll be interested to see what, if anything, makes this different than all of the surveys running in the affiliate marketing/lead generation space.

We agree that maintaining data quality is the biggest challenge and really the crux of this business. We have initial filters that look for incorrect answers to "checker" questions (e.g. What browser are you using? In which time zone do you live?), an outlier response speed (too slow or too fast), etc. And we're working on a more sophisticated system to really address this.

Data quality with initial partners has been good. We'll be working hard to keep it that way.

Thanks for the support, Dan.

I agree that the lightbox experience is common, but I view the surveys as fundamentally different than advertisements. Part of our idea is that your attention is more valuable than a banner/video ad. If I had 30 seconds of time from a smart HN reader, I'd rather ask for advice than show a commercial. Hopefully getting at that valuable opinion will allow you to get better free content online.

And, just to be clear, we're keeping the surveys fully anonymous; so we will not collect or tie to an e-mail address or other personally identifiable information.

And demo that survey sticky on your homepage. I've answered side-bar surveys before and I'm more honest in my answers than when I'm trying to get behind the wall.

Thanks for the suggestion. We'll be moving to and optimizing for mobile soon. (I'm a Survata co-founder).