Attacks against machine learning – an overview
elie.net
elie.net
I've done similar, and afterwards nearly all advertising categorizations of me eventually dropped off my profile (after a period where they were schizophrenic and contradictory). I can't be certain I caused that, because this was contemporaneous with Zuck's congressional testimony and the run up to the GDPR (both of which probably motivated many changes).
But it would make sense that mountains of bad data would make it hard for them to confidently place me in advertising demographic and interest categories, do to all the contradictions.
I don't know how you an deceive Facebook without also deceiving your friends and contacts though.
FB is really good at hiding old activity and being utterly worthless at searching your feed, so if you can just put all the fake stuff in the past you'd be fine with your real friends.
> FB is really good at hiding old activity and being utterly worthless at searching your feed, so if you can just put all the fake stuff in the past you'd be fine with your real friends.
They allow you to back date, but if you're goal is to avoid annoying your friends, you could use the privacy settings for a similar effect. Just post your garbage as visible to "only me," let it age for a week or two until the algorithm will ignore it, then make it "public," "friends only," or whatever you want.
It would become like any other signal jamming arms race, whether it’s radar or social behaviors, and your model of generating random noise has to get more sophisticated as the other party’s anti-jamming techniques get more sophisticated.
I took a class with Scott Aaronson once where he mentioned the idea that the natural enemy of machine learning is cryptography.
So if you know the anti-jammers are using ever greater machine learning techniques, rather than trying to one-up them with adversarial learning, I suspect the best jamming would be cryptography.
Like, extensions to Facebook that essential encode text with PGP or something, send via Messenger, and allow decoding on the other side.
Then an interesting idea for machine learning would be how to make an autoencoder that accepts encrypted text, transforms it into human understandable text that would fool a machine learning algorithm designed to flag encrypted text, and can decode from natural language back to the encrypted data on the other end.
He cites Rivest for that one: https://people.csail.mit.edu/rivest/pubs/Riv91.pdf
Your question led me to this NPR article that briefly talks about the legality of lying on the Internet that's worth mentioning (https://www.npr.org/sections/thetwo-way/2011/11/15/142356399...). It seems that when you agree to the Terms of Service with services like Facebook, you agree to not spread misinformation or misrepresent yourself (https://www.facebook.com/communitystandards/integrity_authen...).
https://www.digitaltrends.com/apple/apple-gets-privacy-prote...
https://www.theverge.com/2018/4/11/17225482/facebook-shadow-...
* Political problem: It is legal and acceptable to track people on the internet to an extreme degree. Political solution: Call your Congressperson, donate to the EFF, reframe the issue as corporate stalking, etc.
* Technical problem: It is possible to track people on the internet to an extreme degree. Technical solution: Restrict ability to collect data by using adblock, poison existing databases with reasonable but false data.
Think of how many people are being involuntarily "tracked" by Dropbox because others are backing up photos in which they appear, or emails they sent, without their consent. For better headlines, we could call this information "Dropbox Shadow Dossiers".
"It visits and navigates around websites, from within your browser, leaving misleading digital footprints around the internet. Noiszy only visits a list of sites that you approve, and only works when you turn it on. Run Noiszy in the background while you're working, or start Noiszy when you're not using your browser, and it sends meaningless data to these sites for as long as you let it run."
Anyone know how to get, or compile, a list of everything likable on Facebook?
My own (approximate) translation of parts of the text:
"Sur Facebook, le trentenaire avait apposé un «J’aime» sur une image d’un combattant de Daesh brandissant la tête décapitée d’une femme. Il a été condamné à trois mois de prison avec sursis." --> "On Facebook, the man in his thirties had clicked "like" on a picture of an ISIS fighter holding the head of a beheaded woman. He was given a 3-month suspended prison sentence".
"«Quand on met J’aime, c’est que l’on considère que ce n’est pas choquant ou que l’on adhère», considère pour sa part Jean-Baptiste Bougerol, le substitut du procureur de la République." --> ""When you click "like" on something, you consider it's not shocking or you agree with it"", said the prosecutor".
But if you start to like random things, or if you say "my birthdate is the 2nd of March" while it's not, you become an unknown unknown, and the algorithm must start to reason with your wrong data.
I see two ways to address this (there are probably more, this is just me thinking out loud):
1. Increase the size of the pool of total reviewers so a 51%-attack becomes infeasible. Incentives can be offered to the rest of the community to get them to participate. (This is similar to what bitcoin tries to do, with the added obstacle of actor anonymity. In an anonymous system, 1 bad actor can trivially simulate an arbitrary number of actors. Bitcoin tries to solve this by increasing the operating cost for each perceived actor. Counter Strike can be seen as having a fixed lump operating cost: purchase price of the game + time investment to accrue enough XP to qualify for the cheater jury. )
2. Create an additional set of people you trust unconditionally. (These can be people you train and pay a wage.) This means you can spot-check anyone, and a consensus between bad actors is an investigative clue (to find more bad actors) rather than a hindrance.
With powerful machine learning systems, we need to think about security a little differently. See especially the section 4.8 about function approximation:
> Given a task for which no discrete algorithm is known to solve, there is a good chance a neural network can at least approximate it. The extreme value of neural networks are their ability, in many cases, to act as an unknown function that can map inputs to outputs with good enough generalization almost as if the actual function was known. This makes any system that relies on the difficulty of implementing an unknown function vulnerable to the malignant use of neural networks
Welcome to high frequency trading. You’re a bit late to the party though (around 15 years).
“Trivially true in the 'of course does behaviour of others matter' sense and in the 'could my actions influence others'. Not necessarily operational though.
Fine line from there to 'spoofing' (== placing trades solely with intent of engaging others to trade at price level) -- with is VERY EXPLICITLY not allowed and for which you can get fined and go to jail. Recall the case of that poor SOB out of London who was made a poster boy for the flash crash?”
It seems there is regulation against this.
I would like to hear stories about such attacks on stock market models.
Copying models is a problem for cloud-hosted pay-per-prediction image classification, not for constantly retrained stock market models that don't take external input.
What you are referring to is possible, but is not "copying" per se, just trying to infer what the system is doing (inverse RL), and then exploit that/make it do mistakes. If you are not HFT it is very difficult to distinguish bots from humans, so you'd have a hard time even finding a target.
Please don't title your article one thing (ML) and then in the first sentence set the context to something else (AI).
Please lead with a short paragraph stating what you did, in what context, and for what purpose, instead of trying to grab the whole pie and implying that your experience and worldview are commonly shared by everyone else.