Comment by ribs | Hacker News Reader

Comment by ribs | Hacker News Reader

Parent Full thread

It really worries me how many people are so quick to forgive him and tell him so.

In my family if someone cheated they got called a cheater and suffered consequences. At least, they would have, if someone did something like that. But my parents didn’t raise mendacious villains.

Look at this crap on Twitter:

“Everyone makes mistakes. Thank you for the apology”.

“Kagglers will still love to have you back”

“It's great that you realize your mistakes. Looking forward to see your comeback with more cool DS solutions and ethics than before.”

“Thanks for doing this. It's okay to make errors in judgement, we've all been there to varying degrees. Y'all be gonna be fine.1!”

Those are the worst. I’m not so crazy about these below, either, although there’s just a hint of steel in them, at least:

“I’m glad to see that you had a change of heart after sleeping on it and that you will be returning the prize money. I hope you will consider donating to or volunteering at a local animal shelter as well. Atonement here is more than returning the money and apologizing.”

“I hope this can be used as a teaching moment as well. Many people clearly look up to you because of your work. What can we learn from this? Something to ponder in the days to come.”

> In my family if someone cheated they got called a cheater and suffered consequences.

He was publicly shamed, banned from Kaggle and he lost his job (and I guess he's basically unhireable right now).

I wonder what would be your family reaction, if these things don't sound like consequences to you.

Losing his job is one thing, but maybe it's time he got a job doing something else.

I'm thinking Kaggle should look to Epic/Fortnite for how to handle cheaters.

A lifetime ban would not be out of the place here, considering it went on to win the competition with no admission until caught.

Kaggle should be looking to the police on how to handle cheaters. He stole 10,000 dollars, he defrauded h20.ai, etc. That's "go to jail" level crimes, not just "get banned from a competition and fired" level crimes.

He gamed their poorly designed game. He didn't hack their servers.

I’m not saying that Pleskov is a sociopath, but I’m beginning to see how sociopaths manage to make their way in the world without getting punished.

Its a down side of all these leetcode type systems where people game them for status.

Imagine you have 100 sociopaths, all of them aiming for the top, all able to recognize each other in a crowd. Now you both compete and cooperate together to get to the top; the ones caught are written off, the uncaught ones sooner or later achieve the goal, even if only 10 out of 100 make it there.

Some headhunters specifically try to identify high-performing sociopaths for top management positions.

> Some headhunters specifically try to identify high-performing sociopaths for top management positions.

[Citation needed]

You're not too far off, it usually starts with the family attitude.

Any sources for this? You can't just claim things like this. At least now without getting some reaction like mine here.

>In my family if someone cheated they got called a cheater and suffered consequences.

You picked a random sample of ratio'd comments from a platform which is big on performative wokeness to strengthen your opinion, which still fails to explain how Pleskov managed to zugzwang his way into pulling off a Kobayashi Maru style move and the failure of the platform to monitor such abuses. Only time will tell, whether or not he has avoided being a part of the Dark Triad; without condoning his behaviour ─ what more can he do to atone for his sins?

I'm confused by this reaction here - this was a _brilliant_ hack of the system and I think his work should be celebrated. Was it in keeping with the intention of the competition? Of course not. Were lives threatened by their creative solution to the competition? Also no. At the end of the day it was a fun, inventive approach to a made-up problem.

So what's the problem?

That's not a hack, that's blatant cheating, their solution litterally looked at the anwsers. It bypassed the ML model prediction, so that's not a ML solution, which to my understanding was the constraint of the competition. And in the end, that solution is useless for the adoption site, since the objective is to get adoption predictions (the animal has not been adopted yet). I'm confused too, by how can anyone think it's an acceptable behavior.

Their solution made use of the data available to them in a "prize" that was nothing more than a made-up competition for fun. Did the rules expressly say somewhere that one could not make use of available datasets?

Kaggle rules routinely restrict use of datasets to approved ones, almost certainly yes but you would have to check to be certain.

The prize was $10,000, not "fun".

I think you misunderstood that the team in question scraped prospective test profiles from the website in question outside of the competition data flow. They blatantly cheated the competition process using data that was not supposed to be available to them.

For this particular problem, they were working on pet adoption timing prediction algorithms. The proposed solution wouldn’t work as efficiently in production as one of the other competitors. That leads to inaccurate predictions on pet adoption times, which increases costs for adoption centers, and maybe euthanasia rates among kill shelters. So animal lives would be impacted by this cheat.

More importantly, Kagggle does competitions across dozens of industries. If a culture of getting better at hiding your cheat pervades the platform, that could impact finance, transportation, and medical research. In those scenarios, lives would either be threatened or at least subject to sub-optimal systems.

The problem was that there was a prize at stake, and everyone else was playing by the rules.

Not trying to be argumentative here, but is what they did actually against the rules? I'm not at all familiar with this competition which might be why I'm not quite so worked up about this, so maybe I have misunderstood something. Did the competition really require that they only train on the provided data?

I compare this to my favorite sport, Formula 1 racing. In F1, teams of engineers with nearly unlimited budgets spend an absurd amount of effort doing everything in their power to bend the regulations (the "formula") to squeeze out some extra advantage.

For example, in this past season Ferrari was suddenly outperforming the pack (and their own recent performance) and it was clear something had changed on the car, they had power in places they didn't before. What finally came down is a clarification of the rules around fuel-rate metering, without directly calling out Ferrari. After the clarification, Ferrari power was back where it used to be. Nothing more was said of the matter by the FIA.

What we all _think_ happened is that Ferrari, knowing the fuel rate meters ran at 10kHz, discovered they could pulse their fuel pump so that the low-end of the flow rate cycle happened during that sampling interval. This means they could increase their overall fuel rate beyond what was technically allowed, due to how that technical requirement was being measured on the car (and reported back to the FIA).

Is it in keeping with the spirit of the rules? Of course not! Does it make for an interesting engineering puzzle on top of an already-exciting sport? Sure does!

Clearly I'm in the minority here, but I think this sort of problem-solving approach can be useful. If you're looking to compete against a field of entrants who are all looking for obvious and well-understood approaches to solving the problem at hand, I think sometimes the best solution to stand out is to look where the other teams aren't looking.

The issue is that the entire reason the competition exists is because the company is sponsoring it and putting forward the prize money so that the top performing models can then be put into production, thereby solving some problem the company has. This type of cheating is dishonest and against the spirit of the competition, but it also defeats the entire purpose of the exercise. Simply keeping a lookup table of answers for the data isn't machine learning, and will not generalize into a production system. As stated in the article, without these hacks, he wouldn't have even placed in the top 100.

To use your F1 analogy, this isn't the equivalent of tweaking the cars in whatever way possible is within the rules. This is the equivalent of completely cutting across the grass and bypassing 90% of the track, which is indeed illegal and would get you penalized.

> this was a _brilliant_ hack of the system

Jesus fucking christ. He fucking ran MD5 on some shit he pulled down from a web crawler.

Over-training a model on the validation set would be a lot more "brilliant", and even that is a dumb script kiddie level hack. Maybe finding an algorithm that computes weights s.t. the preimage of the training algorithm on the training set matches the result of training with truly random weights using the validation set. That could be a "clever hack". And even then _brilliant_ would be.... a real fucking stretch.

That reminds me of another _brilliant_ hack I had my kid do at a youth checkers tournament. I got him to swat the other players pieces off of the board so the opponent would eventually run out of pieces before my child. I think the opposing kids parents were biased against me just because I'm an inventive genius.

Reply on news.ycombinator.com