Show HN: Roundtable – Survey fraud and bot detection API | Hacker News Reader

Show HN: Roundtable – Survey fraud and bot detection API

21 pointstimshell9 comments

Hi HN, we're Mayank and Matt of Roundtable (https://www.roundtable.ai). We launched our Survey Simulator as part of YC S23 (see Launch HN: https://news.ycombinator.com/item?id=36865625), and ultimately decided to pivot because we couldn't convince users to trust our results and our value proposition wasn't clear.

We conducted tons of user discovery interviews, and a lot of big players talked about the pain of manual QC and bot detection. We launched a survey fraud and bot detection API (Alias; docs.roundtable.ai) that determines whether a bot is answering your survey or not. Our API embeds Javascript into survey programs (Forsta, Qualtrics, jsPsych, etc.) to track user behavior and make a judgment on whether the data is from a high-quality participant.

You can use our demo, and try to break it, here: https://www.roundtable.ai/hn-demo. See if you can generate LLM-based text without any flags going off! Feedback welcome!

Here is some of the feedback. In terms of bot protection you would get a 1/10. Perhaps, a 2/10 on a good day.

First, your payload is being sent using base64 encoded using the default alphabet. You only need a simple atob() to decrypt your payload.

Second of all, your bot detection script is very readable making the job of the attacker relatively easy to reverse.

Third, but not least, you do not have enough signals/fingerprints which means that your false positives are going to sky-rocket.

Bots are not dumb, they are programmed by real humans, your site is extremely easy to reverse. You need to add more obfuscation, more signals, better client-side protection in order to quality for real "bot detection".

Source: I reverse antibots for fun and profit, is literally all I've been doing for the past 2 years straight.

Thank you for this feedback! We'd love to contract you in the future to try and break our system

hey, I'd be happy to be contracted to break your solution as well

the arms race between bot and anti-bot is fascinating and I think I could reasonably overcome barriers like it, so HMU!

What are the privacy guarantees you give to users subjected to your tracking? I don't see anything in your terms, and the javascript tracker seems to collect everything the user does on the page without any privacy-preserving techniques applied.

We only track typing behavior in pre-specified text boxes. We are thinking of having version 2 track more data, but we need to determine the privacy implications of that

From the docs it appears this is only for open-ended survey questions.

Do any tools exist to detect suspicious behaviour for e.g. multiple choice surveys? I'm thinking of tab switching to Google for an answer for instance.

Correct, it's only for open-ended survey questions! We are building functionality for multiple choice surveys, but it seems a bit harder to build

Mayank and Matt at Roundtable are two of the most special people I’ve met. No telling what RT looks like in 3 years.

I'm assuming you invested in roundtable right?

Reply on news.ycombinator.com