Hacker News Reader

Comment by foolrush | Hacker News Reader

I suspect that the cavalier sentiment OKCupid has approached the discourse with will not go well for them.

Will be interesting to observe the sociological takedowns of this post.

Unlikely. Facebook went overboard with it after hiding it from their users. OKC has always done this with their data, although I am more than a little disturbed by them showing deliberately bad data to their users.

Also who the hell though it was a good idea to have a dating site without pictures (even for 24 hours?)

And, probably more to the point, Facebook is Facebook and OKCupid is OKCupid. And even if, as OKCupid says in their post "But guess what, everybody: if you use the Internet, you’re the subject of hundreds of experiments at any given time, on every site. That’s how websites work." the fact that Facebook did it ignited the Internet's collective anti-Facebook sentiment. (Of course, it's also the case that Facebook's experiments were more oriented toward their own advertising than improving the site experience for users.)

Unfortunately, while showing what we would consider "intuitively bad data" may seem like a bad idea, only actually empirically testing it can we measure and quantify the exact impact that bad data (either shown purposefully or accidentally, or unknowingly) will have on users.

It may have in fact turned out that what we intuitively think as bad data results in better matches or better experiences. I think experimenting is worthwhile, so long as it is done in the open as they have been doing.

I remember reading about this concerning Netflix. They used to have a competition on who can come up with a better recommendation algorithm. They eventually decided not to implement the best one because what they were getting was good enough, and since most people were streaming instead of getting a dvd, the more recommendations the better. They could just try and if they don't want to see the whole movie, rate it and instantly pick another.

I'm pretty sure they didn't implement it because it was an extremely impractical solution.

Edit:

"We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment." - https://www.techdirt.com/blog/innovation/articles/20120409/0...

>Streaming has not only changed the way our members interact with the service, but also the type of data available to use in our algorithms. For DVDs our goal is to help people fill their queue with titles to receive in the mail over the coming days and weeks; selection is distant in time from viewing, people select carefully because exchanging a DVD for another takes more than a day, and we get no feedback during viewing. For streaming members are looking for something great to watch right now; they can sample a few videos before settling on one, they can consume several in one session, and we can observe viewing statistics such as whether a video was watched fully or only partially.

I wouldn't say that this makes you wrong, but I'd say you are partially right.

> I am more than a little disturbed by them showing deliberately bad data to their users.

I actually find this reassuring. It means they're sanity checking their algorithm to make sure that it works better than random pairings, or isn't outright wrong.

I'm willing to bet that Facebook has always done this with their data as well. You have to, in order to determine not just if your algorithms work, but whether they should work and what they should do.

Also, Facebook didn't 'hide it from their users' so much as 'not tell anyone they were doing it'. And for the average OkCupid user, I'm willing to bet that 99% of them have no idea this sort of thing happens. They log onto the site, do some stuff, and then leave.

Personally, I was as surprised by the idea that OkCupid outright lied about compatibility scores to see what was up as I was about the results, but it doesn't bother me, and I'm the type of person who's read all their blog posts anyway.

Reply on news.ycombinator.com