So in one sentence, a user name containing "catbot" is a legitimate signature of an actual work of malware Valve is trying to mitigate.
HN, you got trolled. There's nothing wrong here at all.
HN, you got trolled. There's nothing wrong here at all.
Precision is the ratio between correctly-identified cheaters and the total number of people matched by the signature.
Recall is the ratio between correctly-identified cheaters and the total number of cheaters in the system.
In this case, the signature had good recall but poor precision, as those threads describe cases that make it plausible to have a "catbot*" username without cheating.
(I said "had" because the recall rate is going to drop when the string gets changed by the cheat authors. So long-term, the signature is poor on both recall and precision.)
The most problematic issue I see for Valve is not the lack of measurement of precision and recall for their VAC ban signatures, but rather the inability to obsess over their customer experience and react properly when relevant issues in their products are reported.
How so? Of Steam's users that are running linux and have 'catbot' as part of a username on their system, what proportion do you think are running a cheat bot? 99%? 99.9%?
What level of precision would you say it not 'poor precision'? 99.999%?
So whilst you have a reasonable point, it's not relevant to the point I was making, which is that the precision of the rule (after implementation, and before discovery) probably exceeded 4 nines, which doesn't seem like 'poor precision' to me, without further detail on the requirements.
And BTW I have no opinion about whether this was a good or bad move by Valve. I don't have sufficient context for that, and my comment was intentionally focused to try and understand what the person to whom I replied thought would be acceptable precision.
It's like treating a disease with antibiotics, eventually the disease builds a tolerance and you need to find a new drug. By using multiple drugs you can hopefully kill it before it can resist any given drug, but in practice tolerance builds regardless of your best efforts.
These cheats are actually big business nowadays, people have a financial incentive to work around them as quickly as possible.
https://github.com/Kr4ken-9/cathook
Golly, hack authors are just so concerned for the well-being of random internet strangers, how dare you doubt their sincerity! /s
Plausible or possible? All heuristics have potential false positives (e.g. maybe someone puts the actual catbot binary in the blockchain, and now all Bitcoin users get banned??). I don't see any evidence in the GitHub thread of someone accidentally hit by this. And I see evidence of one person legitimately being hit by this (someone claiming that they are running the bot, just not with the same Steam account).
So I think this heuristic is working properly. (Whether to allow any theoretical risk of a false positive is another discussion, but, Valve is a business, it seems totally reasonable to me for them to risk it.)
Unfortunately, it seems that Valve's software doesn't bother to actually check whether the machine has any malware installed, it bans anyone who has any account on the machine that matches a regexp. Innocent or not, who cares, right?
How "effective" this approach is is very much documented by the fact that as soon as the botters caught wind of this they have changed the username they use and Valve can play a game of cat & mouse with them now.
It is not the first time Valve got caught by doing dumb/lazy stuff like this. E.g. Steam was known to erase your entire machine (or at least what your current user had access to - so most likely all you data) because of lazy programming and hardwired assumptions (see here: https://www.theregister.co.uk/2015/01/17/scary_code_of_the_w... ). Their response has been pretty similar too - "Works as intended, go away, we don't care".
Yay ...
Devoid of all context, no it's not. When a malware is known to exclusively use a particular and uncommon pattern, yes it is.
If you remove all context then nothing is ever a signature of malware. Maybe there is a legitimate reason for a third-party program to be writing your memory or injecting DLLs, you never know! But in practice those are red-alert triggers for any anti-cheat.
This is a poor solution: it's the equivalent of blocking spammers by IP address. It works but will also lock out legitimate users.
You take what clues you can get and you push the fix. This is what all their users not named catbot want.
Comparing statistics (accuracy, reaction time, chat messages, what have you) server-side to a not cheating player population seems like a very promising solution, given it doesn't involve the game client (which is assumed compromised). I don't see how that has anything to do with the halting problem.
There is no bulletproof solutions.
That said, its a really shitty signature with lots of false positives -- catbot is exactly the sort of strange names I assign to hosts or projects.
...and also doesn't look too out-of-place as a gamer handle --- in other games I've seen players with variants of 'cat' and 'bot' in their names (catman, botman, catfish, fishbottom, etc.), which didn't seem to be cheating either.
If it was a name with more "entropy", like jH3bzKflAC5, then the case for it being a signature might be far clearer.
In any case - both are all over the place and in online games there are often crazy names that look like the came from /dev/random. It'll also be easily circumvented now that it has been discovered and it was dumb in the first place.
Valve really dropped a ball, it's not like they don't know how crazy, meme fulled or troll-y usernames get on their platform.
Outright VAC ban (that's a big deal) for something like that is absolutely idiotic (extra inspection that doesn't affect anything for legitimate users - sure).
After all the banwave only hits once.
The mentality that drives the use of aimbots is a disgusting scourge on humanity.
Their chat filters did not allow the string "cock" in chat...and then they put a bunch of nasty cockatrices in the Dreadlands and in the Overthere, along with quests that asked you to seek out their eggs, feathers, and beaks, thus giving you a reason to use the string "cock" when talking with others.
I think signature is a strong word. This is kind of like comparing the color of the ink used to sign for something and ignoring the actual signature.
Who suggested otherwise?
A dozen or so people in this thread?
What happens if the cheaters decide the change the 'triggering' username to something far more common? Guest, admin, local, user, etc. Now valve is allowed to ban everyone who has that username on their system?
HN is as much your comment as any other.
eg this guy: https://github.com/Kr4ken-9/cathook is kr4ken on HN, there's probably others here who are smart enough not to link their black-hat identity with HN.
This is literally blackhats bitching that they got VAC-banned, 100% legitimately, and then concern-trolling us about what-if scenarios about some hypothetical innocent user who might have been swept up with their script-kiddie customers.
Can't say that such opinions really have any place here, IMO, considering that they are the ones who created the problem in the first place. Maybe it's not worth shutting them down, but they certainly shouldn't be celebrated or legitimized in any fashion.
Scorn is an entirely appropriate community reaction here, and a mod tipping the scale in favor of black-hats is not really appropriate IMO.
Our community has its strengths but some subjects like politics provoke a suspension of critical thinking. In this case, small time hackers being punished by a big company in a seemingly overkill way for a seemingly innocuous slight.
Ideally, a moderator should be aware of the biases of the community and adjust the discussion accordingly. Or at least when the lies being perpetrated are as obvious as it is in this thread.
My point is the orthogonal one that it's incongruent to snarkily diss HN when you're as much a part of the community as anyone else is. It makes no more sense to say "HN got trolled" than "HN debunked getting trolled". People don't do this to be informative, but rather to superciliously position themselves above others. That degrades the spirit of the community here, which is already so fragile. That's why I reply to such comments the way I did.
It's like saying "<Insert Community / Country / Organization Name>, goddammit!"
I personally don't mean that I'm better than everyone in said community, but that I'm disappointed with the prevailing sentiment, and so feel the need to signal my separation on the issue.
It's shorthand "I don't identify with <Community> on <Issue X>".
People do indeed abuse this rhetoric to feel superior to others, e.g., "Reddit is a hivemind!". But on issues where the community clearly falls prey to its biases, a little snark now should remind us to avoid those collective pitfalls in the future.
Anecdotally, the people who screech about and around VAC bans are the same people who will ruin online games with botting, scripting or just general douchebaggery.
Basically, no one notices VAC if they aren't trying to screw over other gamers.
Those who do so so repeatedly, often resort to comment threads like this one, venting vitriol on the off-chance the more-gullible will confuse it as signals of a larger problem.
Steam has many issues: poor customer service, a wash of bad content, a lack of focus on making games.
But being too hard on those poor, upstanding cheaters? Not an issue.
Or reinstall my TF2 and get banned so then I can tell you to stick it.
It's VAC that's doing the trolling here. Person's OS (not even Steam) username is not a cheat (and Steam frankly has 0 reason to examine its name in any way for any purpose, for installing games just pasting will do and for everything else there is a Steam username).
This is bullshit, will be defeated now instantly that it came out and on the level of dumb antiviruses that delete exes (i.e. my compiled Pascal and C++ programs) and claim the reason was "unknown low spread program" (yes, that happened, because clearly, unknown program = virus).
VAC has sainthood status now, Valve social media staff often tells people it's 100% foolproof and bans are never appealed, people who claim they were unfairly banned are shitted on by the community and it's undisputable (although there were overturns i.e. when CoD players got banned by Steam's mistake[0]) but they do crap like that?
It's like Destiny 2 bans last year, they were all "hand made by staff and 100% confirmed cheaters" and then a couple hundred got overturned and claimed to be "software error".
What next? Banning everyone who has a GitHub account or knows git because the aimbot code is hosted on GitHub? Banning people from sites for having HN accounts because it's a "hackers" site?
What planet do people like you come from?
[0] - https://kotaku.com/5597038/valve-bans-modern-warfare-players...