Hacker News Reader

Comment by kelnos | Hacker News Reader

I can only assume that this isn't the result of a human flagging Pushbullet like this; I expect it's an automated system. And if that automated system can make a decision to flag the extension, it could also include in the email specifically what caused that flagging to happen.

At this point I'm really starting to become unsympathetic to the idea that they can't tell you what you're doing wrong because it'll enable people to figure out ways to work around the flagging algorithm. I kinda just don't care anymore. If you're going to run a platform like this and capriciously threaten to kick people off of it, it's pretty antisocial to hide the reasons why.

(And yes, I know, Google has no obligation to do anything differently here. But I can perfectly well think they suck for their current practices.)

View on HN

I agree, and will go further in that I believe they actually should have such an obligation: it should be a consumer right to find out why you were denied access to a platform--or a restaurant, or a barber shop, or a wedding cake maker, or whatever else that someone is refusing you service due to, as a business operating in the public--and that whatever reason that is must be something that either 1) has nothing to do with the customer and can be shown to have nothing to do with the customer (such as "we do not have enough workers to take more orders and we didn't do anyone other than you") or 2) is clearly and obviously "correctable" (so like "because you are gay" doesn't count); if the platform (or service or business) refuses to tell you, or their reason isn't "correctable", there should be heavy penalties attached for what amounts to opaque discrimination.

(FWIW, I appreciate the idea that eventually it would be useful to be able to ban a bad actor entirely from something to exclude the possibility of future harm: the past tense of "correctable" was "avoidable", and it needs to be very clear exactly what was done wrong and what could have been done differently for someone to end up in the situation of being banned from the usage of a platform, service, or business: the alternative where people get to apply secret and obscure reasons to refuse service is madness.)

I was ready to get behind this line of reasoning, but then I thought about e.g. comment sections. Anyone who has moderated a forum will know that trolls find every possible way to walk right up to the line without technically crossing it.

If you say "personal attacks aren't allowed", they'll say "I wasn't attacking him personally, I just suggested that anyone who says [the thing he said] should consider getting evaluated by a doctor for mental retardation."

At some point you just have to say "I know what you're doing, you're clearly not acting in good faith, and I have to ban you." Are all of those people now potential lawsuits?

Yeah, this is the issue. They're not going to catch every dirty thing extensions (perhaps unwittingly) do. If they are able to detect one or more problems with an extension, there's a good chance that there are other problems that were undetected. Forcing extension authors to go back and think about what they could be doing better rather than just telling them the minimal set of changes that are needed seems like it can only result in more well-behaved extensions across the board.

There should at least be some appeals process that involves a human then. In this case they fixed legitimate cases where they had too many permissions (including removing some functionality), and were still rejected. What if the reason they were flagged was because of a legitimate usage of a permission that the automated (or manual) system wasn't aware of? At the very least there should be a way for a flagged extension author to go through a process where they explain why they need each permission they request, and if that is they are still rejected, told why their explanation wasn't good enough.

This is especially important given google's market power in the browser industry. What if the extension is from Google's competitor? Google could ban the extension without a legitimate reason to hurt their competition.

> If you say "personal attacks aren't allowed", they'll say "but I wasn't attacking him personally, I just suggested that anyone who says [the thing he said] should consider getting evaluated by a doctor for mental retardation."

1) That pretty clearly qualifies as a personal attack

2) There is a significant difference between a forum and a business critical software distribution platform.

> That pretty clearly qualifies as a personal attack

Okay, let's take the obscurity up one level: I used to moderate a very small internet forum, where there was one user who had certain mannerisms in his writing. I also had reason to suspect this user was mildly suicidal.

A troll began to mimic these mannerisms as a way to make fun of that user, and talk about how horrible their life was. The troll wasn't overt about it, but if you knew the history between these two, you could tell what was going on. The troll of course feigned innocence, but as he'd been given warnings before, I made the call to ban him.

I firmly believe I was acting within the forum's stated policies—but I am not 100% confident a judge would agree. If these types of actions could lead to lawsuits, I don't know what I would have done.

---

> There is a significant difference between a forum and a business critical software distribution platform.

Great—but where's the line? Is Youtube critical, for example?

It sounds like this case would comply with the proposed rules to me. You informed the bad actor of what he was doing wrong, the behaviour was correctable, and he continued to exhibit said bad behaviour, so you banned him.

It would be common enough to have laws that say only businesses of a certain size are fully obliged to comply. GDPR has this for some clauses, for example. I guess the idea is along the lines of the market determines whether you're critical by letting you become a huge company?

Don't fixate on particular examples when someone is clearly trying to talk about a broad trend.

> I can only assume that this isn't the result of a human flagging Pushbullet like this; I expect it's an automated system

There isn't. Take a quick look at stupid popular apps on the Play Store, almost all of them require completely unnecessary permissions literally in conflict with the bullet points that Google listed in their email to PushBullet.

Also note that almost all of them have 10-50x the amount of downloads compared to PushBullet.

The odds just spell bullshit.

Their "automated scanning process" didn't just happen to pick the popular app that gives users more control and platform independence from Google.

Because really that's what it's about, PushBullet is dangerous to a closed ecosystem like Google wants.

(also, I believe that given the permission system, and if an automated scanning system was in fact in place, it should be super easy to exactly point out what is wrong, like compiler errors. and being granular permissions, there's no "gaming the system" about it, you either get the permission or you don't)

> If you're going to run a platform like this and capriciously threaten to kick people off of it, it's pretty antisocial to hide the reasons why.

I don't believe Google gets to be "antisocial", they're not a person. They exist since ~20 years and are made of shifting people, none of whom really control it. As far as it can be "antisocial", I wonder if it even vaguely "realises" that its interchangeable parts are the same entities as its consumables.

That's a bit extreme. Google sends out emails every few months about new policy that may get checks. They definitely aren't very good at doing automatic checks that are relevant or have much relationship with the policies they publish.. But that is a bit different than strategically attacking specific apps beyond choosing the X thousand most popular apps to get the most users affected per presumed threat.

(I get various compliance spam and ignore it and among things with small numbers of users and/or owners who lost interest a small percentage a year fall out of the stores. The only differences are that we don't care enough to forward the threats Google sends us and no one would bother to amplify them.)

> And if that automated system can make a decision to flag the extension, it could also include in the email specifically what caused that flagging to happen

Unless it's a black box machine learning model that just says yes/no but can't tell you why

I think it's more likely that their flagging is heuristic based and could easily defeated by a malicious extension author and giving detailed feedback makes reverse engineering the rules trivial.

Law enforcement has a solution for that: Do tell people what the rules are and what they are doing wrong. Don‘t tell them how they got caught and what methods were used to catch them.

Telling people that they are going to be severely punished within days by a very powerful global entity, without telling them which specific rules they are in breach of or what they have to change is a dystopian nightmare.

Law enforcement also has to tell people how they were caught, perhaps not the entire story, but at least most of it. That, too, is a good thing.

Yes, to a degree. But for instance, tax authorities don't have to reveal the algorithm they use to determine who gets audited.

Being audited isn't an explicit accusation that you've violated anything either; at best, it's an implicit suspicion to which the tax authority is fully entitled to probe and we as taxpayers are given reasonable opportunity to corroborate claims.

Absolutely. My point is simply that you can tell people what the rules are and which ones they may have broken without revealing absolutely everything about how you detect any malicious behaviour.

> flagging is heuristic based and could easily defeated by a malicious extension author

Well, if the reason is the "https://*/*" permission or something like that, and it is flagged and reported automatically, and everyone learns not to use it, it's a win-win. Benign extensions will have smaller targets on their backs, and malicious extensions will become less malicious OR will have to jump through hoops to achieve what they're trying to achieve (and will become easier to detect).

This isn't something vague or hard like OCR, voice-recognition or image classification.

It's a finite list of app permissions.

Unless it's a random number generator that just says yes/no but can't tell you why -- is just as valid of a non-excuse for using the entirely wrong solution to a problem, and surprise the solution doesn't work very well.

> Unless it's a black box machine learning model that just says yes/no but can't tell you why

Recently there's been some interest in Explainable Artificial Intelligence (XAI) which addresses the problem of "black box" machines. There are techniques such as SHAP and LIME that can help the ML designers provide explanations for why any specific observation has been "rejected". That said, I'm not sure if that's what Google uses here.

Sure, it's anti-social. But is it more or less anti-social than running a huge software platform that advertises the ways to exploit itself?

I think it's a very interesting question. Who is more deserving of a good experience - uninformed users who might get taken advantage of by malicious extensions, or developers for the platform, who definitely do get taken advantage of by virtue of running afoul of rules they can't read?

Even if the average damage to a user is very low, the ratio of users to developers is massive.

> But is it more or less anti-social than running a huge software platform that advertises the ways to exploit itself?

Security through obscurity is no security at all. Google is not doing its user a favor by hiding the criteria it uses to determine whether an extension is malicious or not. Also, just because Google won't publish the criteria, it does not mean that it can't be discovered by someone with enough determination. I think that a malicious actor is more likely to spend the necessary time and effort to do so than a developer who is more interested in creating an extension that will serve its users well. If anything, these types of policies drive away developers who act in good faith and leave the marketplace with more developers who are looking to exploit the users.

It seems likely to me that it's impossible to have an extension system that allows useful extensions that users want while also being completely secure against malicious actors. Security by obscurity is an important tool in the abuse fighting toolbox, because it allows you to have cheap heuristics while increasing the costs for malicious actors.

Not really, because malicious actors don't care about their reputations, accounts, etc.

A reputable developer has a reputation to maintain (by definition), which makes Google's threat to permaban them a threat indeed.

A disreputable developer doesn't care about their reputation (again, by definition). They can create a new throwaway account every day and apply using the same (or slightly, easily, altered) code with different permissions every hour until they get permabanned, and start again tomorrow.

So the "obscurity" can be discovered easily through experimentation by the bad guys, but is still obscure for the good guys. This is not a good outcome.

The point is making it more expensive for malicious actors. You can't make it impossible to get malicious extensions in, but you can make it harder. It's the same as captchas. You can also defeat captchas, but adding them reduces abuse a lot. As with captchas you're making it harder for good actors too. The difficult part is finding a good balance.

Reputation can be bought too btw.

> The point is making it more expensive for malicious actors.

Malicious actors don't care about expenses nearly as much as benign ones do. So the point ought to be, if you want me to fix something, tell me what's broken.

Except that in this case, it's misplaced, and causing benign actors far more pain than malicious actors. If they want to hurt malicious developers, they need to flag extensions as untrustworthy for:

1. age < 180 days

2. dau < 1000

3. some rule around user reports of malice on uninstall?

and this gets a bright warning banner on the top of the page, and it can't be discovered through the chrome store until crossing these thresholds.

But this is not email, where literally everybody can send things. Raise up the barrier at submission level - where you must be authenticated - to get rid of automated trial/error attempts, I bet they can. But when an extension/developer is there, when it has a long history in the store and used by 1M users, hey, give some human feedback. It won't break your automated anti-spam/scam rules.

Getting kicked off the platform happens on HN itself a lot/ I get flagged on HN on almost regular basis (and my ability to reply/talk back is taken away) for speaking my mind (no curse words, just opinions that are others strongly disagree with) about Google, HN itself, Amazon et al. If I ascribe a n ulterior motive to someone in power, I get kicked off. Worse, sometimes I get attacked personally. So I think it's pretty ironic to complain about being kicked off on HN. This is probably my 50th username.

Who is referring to getting kicked off HN? I believe the post you're responding to, which said "If you're going to run a platform like this and capriciously threaten to kick people off of it," -- the "it" referred to Google.

Reply on news.ycombinator.com