Facebook Plans on Backdooring WhatsApp?
schneier.com
schneier.com
To be crystal clear, we have not done this, have zero plans to do so, and if we ever did it would be quite obvious and detectable that we had done it. We understand the serious concerns this type of approach would raise which is why we are opposed to it.
If this is done client side, it doesn't boils down to that. You can easily decompile and see for yourself what it does. You will gain quite a bit of notoriety if you are the first one to catch them too.
As he said:
> if we ever did it would be quite obvious and detectable that we had done it.
Assuming your device allows you to get the binary. Apple is already in a position to disallow this if they choose to in the future.
Theses kinds of thing never stopped anyone. Being the first to share a hash of a system file in a console is always an achievement that many hackers tend race to do when a new one is released.
For sure the harder it is, the less person will do it, thus the more theses things will be able to go under the radar, but for now it's not so much an issue.
Can you tell us a bit more about the circumstances? Is it something you are exploring to better understand the approach of a competitor (WeChat)? Are you receiving pressures to implement this?
See also:
>Respect for your privacy is coded into our DNA, and we built WhatsApp around the goal of knowing as little about you as possible ... If partnering with Facebook meant that we had to change our values, we wouldn’t have done it.
I'm sure you personally are a nice, honest and well-intentioned person. Unfortunately WhatsApp's corporate messaging has zero trustworthiness and should be looked at with suspicion. Even when the person saying it happens to believe it.
While I cannot speak for the Forbes' author, Schneiner is widely reputed as a trustworthy source, especially on matters related to information security. This article calls into question his professional reputation as a information security journalist or yours as an executive at WhatsApp.
As such, in order to help the general community decide for themselves, please shed some light on the following:
1. Does Facebook/WhatsApp have any specific plans for moderating content, via any mechanism, on the client? If so, please enumerate the kind/type of client-based content moderation currently in discussion.
2. Has Facebook/WhatsApp previously looked at doing content moderation on the client? If so, please enumerate the kind/type of client-based content moderation that was previously discussed.
3. What will you do, if Facebook/WhatsApp decides to implement content moderation and/or a content "backdoor" on the client sometime in the next 3 years? Will you continue to work for Facebook/WhatsApp?
4. Should Facebook/WhatsApp decides to implement content moderation on the client, what forewarning will Facebook/WhatsApp give us. What will you personally give?
5. You say that this is easy to detect. Can you please provide technical guidance (or pointers to such) on how to go about detecting this, so that the community at large may better learn how to detect this in any instant messaging app, WhatsApp or otherwise?
I ask the above, in all sincerity, as Facebook's previous poor handling of data requires these kinds of inquiries -- especially when in opposition to reporting by Schneier, who's reputation as a information security journalist is bar-none.
I looked at what WhatsApp promised to do against fake news (something where they had reason to promise harsh measures, since they were basically blamed for murders due to their forwarding features). I'm aware of restrictions and warnings on forwarding, but not some sort of 'fake news detector'.
> 5. You say that this is easy to detect. Can you please provide technical guidance (or pointers to such) on how to go about detecting this, so that the community at large may better learn how to detect this in any instant messaging app, WhatsApp or otherwise?
Reverse engineering their app. Doing it yourself is probably beyond the time you want to invest, paying someone to do it just for you is probably beyond the money you want to invest, but I'd really love if there was a group/entity that consistently checks (through reverse engineering and similar analysis) whether privacy promises given by apps are true, and most importantly, remain true over time.
Right now, at least on Android, it seems impossible to add a new contact without adding it to your phone's address book, then giving WhatsApp full access to it. If you revoke the access, you can keep talking to existing contacts, but their names disappear. I would expect that this is just a side effect of nobody caring/testing for the case, but it attracts less charitable interpretations (assumptions that it is intentional to force users to give access).
I genuinely believe that both from a software usability and network effect aspect, WhatsApp is the sweet spot among the secure messengers, and the trade-offs they made (e.g. key escrow for backups and encouragement to do cloud backups) were made in good faith considering the average user's needs.
Granted, it's not ideal, and not even feasible if you already use the work profile fully (with contacts you don't want to share with WhatsApp).
https://developers.google.com/android/work/requirements?api=...
because yor denial, only covers the moderation sugar on top. the damage is already done. a long time ago.
We are moving ever more towards 1984.
Software which people's civil liberties depend upon should be developed in the open, auditable by the public, as a minimum. Even then, developers need to be vigilant about tainting by state actors.
What do you want? An official announcement from Facebook?
Could be FB Messenger or Instagram DMs for all we know.
https://developers.facebook.com/videos/2019/applying-ai-to-k...
The relevant stuff seems to start at about 23:00.
Original message follows for continuity:
Thank you for this!
The 4-layers-deep source is an F8 talk on using AI to filter for FB TOS-violating content. The relevant portion of the talk is discussing hyperlink website previews of the kind that show up next to URLs posted in Messenger/WhatsApp that show the title and feature image next to the link. In order for this functionality to work at all, the app must crawl the link to pull the metadata for display. This AI team is attempting to pre-emptively block URL-masking attempts that fool the Preview into showing benign content before redirecting users to bad content.
To do this, FB must "pierce the veil" of E2E encryption in order to see what the URL is and test the content. This opens the possibility that FB can further expose supposedly encrypted content in the chat/message. The actual talk refers to putting this AI on the device itself, preserving user privacy (a difficult technical challenge).
If this leaking of URL data is status quo on FB messaging systems, is it reasonable to assume that the supposed encryption is more or less invalid? Are there any ways that this could be a secure system with this kind of content validation going on?
If everything happens on the device and noting is sent to servers, how does it break encryption?
In case I am wrong, and they actually suggest that they will really send data back to its servers, where it is suggested? I really failed to find it.
...So far, we have been keeping this fight [against bad actors and harmful content] on familiar grounds. And that is, we have been training our AI models on the server and making inferences on the server when all the data are flooding into our data centers.
While this works for most scenarios, it is not the ideal setup for some unique integrity challenges. URL masking is one such problem which is very hard to do. We have the traditional way of server-side inference. What is URL masking? Let us imagine that a user sees a link on the app and decides to click on it. When they click on it, Facebook actually logs the URL to crawl it at a later date. But... the publisher can dynamically change the content of the webpage to make it look more legitimate [to Facebook]. But then our users click on the same link, they see something completely different - oftentimes it is disturbing; oftentimes it violates our policy standards. Of course, this creates a bad experience for our community that we would like to avoid. This and similar integrity problems are best solved with AI on the device.
Linking to things is a normal thing to do on the Internet, but Facebook wants to control that, because links often also mean the user moves their attention away from FB, onto the linked thing. This is what FB doesn't want. But it is a very useful thing for the user, to be able to smoothly navigate between apps and websites depending on their needs.
I do wonder why they chose to go with the danger of URL masking that links can be "disturbing" instead of "scams". Because I don't believe these techniques are (widely) being used to trick people into watching animal torture or other disturbing things. They'll be tricked to click on scams and phishing sites.
The difference between "disturbing" and "scam" is literally the difference between subjectively not-okay and objectively not-okay.
Facebook just wants the power to prohibit you from accessing what it deems "disturbing".
Also just from a technical perspective, there's some very weird assumptions: FB's server would see something different than the user's browser. But FB's local AI spy process would see the same thing? Are they seriously pretending that you shouldn't be able to securely log onto another website in a browser? Yes the user's browser sees something different, because it is logged in, and no FB's local AI spy process is not invited ...
At most they could warn that the site looks different than what the server side FB scanner saw. Then the user could decide if they feel more secure because this is expected as they are logged in and FB can't watch along, or if this is unexpected and they should be cautious.
if the process runs on the device, scans information on the device, and then triggers to notify FB when it matches something, that's breaking the encryption.
it's leaking of encrypted information (the fact that it's there, which rules triggered), but more importantly what's the use of encryption if you got a process that's basically acting as a mole, even if it's running locally?
unless I am wrong how this works, but at some point it'll trigger and notify FB or perhaps a government? if it doesn't do anything, then it might as well no be there.
and if this system gets off, there will be false positives, and there will be trigger rules matching much broader things than they strictly need, "just in case". we kind of know how these stories go by now and which bits we can safely give no benefit of the doubt because a state or a corporation wants them so strongly. I'm gonna guess it will be a black box "algorithm" without clear trigger rules, because it's AI/ML it's hard/impossible to explain exactly how it reaches its conclusions and therefore nobody knows really (and much NDA unto those who do). that's gonna be a hard sell though, even to the general public.
Watched through the whole presentation. Not a single mention of WhatsApp.
It could be Messenger exclusive for all we know right now.
https://news.ycombinator.com/item?id=20549610
He has hyperlinks to further articles.
This will be true for a lot of people, and it's a huge annoyance. The only things that break this kind of network affect is a market changer - Discord is a good example here.
It doesn't help that, last I checked, the good (open) alternatives were all much jankier or missing one or more major features we use. Makes them hard to sell. :-/
It's better to cultivate that flexibility, than to convert people to flock to the next latest one true DM app.
/s
References:
[1] https://www.schneier.com/blog/archives/2019/08/facebook_plan...
[2] https://www.reddit.com/r/privacy/comments/ckrmjg/facebook_pl...
If you look closely at how the founders exited facebook, and what was said at the time, it seemed (to me at least) that they left over exactly this.
I mean, that aside, can you really come up with many other operational arguments that would result in people walking a way from hundreds of millions of dollars? (or by some estimates leaving over a billion dollars on the table between the two of them)
You haven't bothered to read the source, haven't bothered to understand what the context is, and haven't even bothered to see that there is no mention of WhatsApp in the talk.
Ducking tired of click bait and people who ducking reward it.
:face-palm:
It took my brain a few tries to parse the word duck in this context.
There's plenty of arguments:
- Wanting to do something new without the constraints of Facebook owning all the IP by default
- Not particularly liking the work environment and being independently wealthy enough not to have to put up with it
- Health problems from stress - money doesn't compensate for bad health
People walk away from huge piles of unvested equity every day of the week.
"Content moderation" is just another kind of surveillance. You have to have been watching -- surveilling -- in order to moderate.
AT&T already has capital and people invested in surveillance. https://en.wikipedia.org/wiki/Room_641A
https://www.theregister.co.uk/2019/07/31/home_sec_priti_pate...
So I am not surprised.
Unless the government plans to go back to the 80s, in which cryptography is heavily restricted from export, this doesn't accomplish anything.
It is a bizarre paradox where designing a secure system is a hard problem, where zero-days are commonly cheaper than brute forcing a 64-bit key, while cryptography is chronically lambasted.
And they can use it to converse only with themselves. Getting everyone else to stop using their favourite panopticon is the hard part.
All it takes is a usable interface, general availability, and a reason to change. If WhatsApp starts doing content moderation, of any form on the client, I am of the strong opinion that users will find this to be "creepy" and will immediately start shifting. As it is, all of my technical contacts have already left WhatsApp for Telegram. Since that group/audience has been the bellwether for all of the previous instant messaging client shifts over the last 2 decades, I imagine it's only time before "everyone else" also shifts off WhatsApp. I don't necessarily think it will be Telegram, but they definitely have the most momentum today when compared to, eg, Line, Zalo, Viber, Kik, et al.
That wouldn't accomplish anything now, anyway. All the current best crypto we have is out in the world, and plenty of crypto research happens outside the US. If the US revived the export restrictions, it wouldn't have much effect on the rest of the world.
We have to TRUST WhatsApp, Telegram, Signal, et al because there are no good OPEN SOURCE alternatives. Otherwise we could just run our own.
We have to TRUST the current PKI and DNS because the alternatives are still immature.
But EVEN IF we develop robust and mature alternatives, and wide adoption, we have to TRUST the Device, OS and Browser makers.
The Trusted Computing Base is made by a handful or companies. Open source in hardware is still a nascent field.
It’s interesting if one can ever be truly sure that some part or chip hasn’t been interdicted. Apple tries to scan components as they arrive vs their hardware designs.
But hardware will always be the weak link. Keyloggers. Cameras watching your fingers from the ceiling.
The future of surveillance is in this kind of stuff. We will ALMOST have secure communications, but not really. The only thing you can be sure of is sending quantum entangled particles from airgapped rooms.
Yes, there are. There has been for many years. Matrix or XMPP with OMEMO.
... hopefully
Fuck them hard.
Does anyone else think it's weird that it requires access to your camera roll to view images, access to your camera to use the web version, and access to your contacts to start group chats? (on android anyway)
Aside from the webapp QR-code thing, none of that is technically necessary.
"Please don't bother, Alan. Every time you turn that thing on it just gets printed out in my office. And yes, yes, yes, yes, yes, yes, yes' yes, that includes your little computer"
I hope people begin to take notice.
The value of WhatsApp is not that its technically better than Signal or Matrix, the value is that there are 1B+ users.
Give me Matrix with Signal's UX, and I'd be all over it :-)
Google is in the middle of rolling RCS out and that fact that E2E encryption isn't part of the basic spec is appalling. I don't know how a developer could work on something like this at Google and feel good about themselves.
https://www.theverge.com/2019/6/17/18681573/google-rcs-chat-...
[...]
After all, if either user’s device is compromised, unbreakable encryption is of little relevance.
[...]
The problem is that if Facebook’s model succeeds, it will only be a matter of time before device manufacturers and mobile operating system developers embed similar tools directly into devices themselves, making them impossible to escape. . . . effectively ending the era of encrypted communications.
I guess FB is one of the more visibly monstrous instances of surveillance capitalism so it works well as an example, but I don't understand why the author is acting as if what it allegedly aims to do is unprecedented. As if similar phenomena hadn't already been happening surreptitiously throughout various other hardware/software for quite a long time.
Wire is fine (not as good as or fast as Telegram), but the company seems to be focused on paying customers to increase revenues. So free customers hardly get any support or responses from support (the auto-replies from Wire support state this priority). I like that Wire is by default E2E and syncs conversations across devices and platforms!
I've been waiting to recommend Signal to others for at least three or four years now. But it's 2019, and Signal is still bad in UX and messaging reliability (not to mention one still can't migrate conversations to another device on iOS).
PRICE: Jack. Look at me. I am not the problem here. The problem is that hard cash is fading. Rapidly. That’s just the way of the world right now. And Bitcoin is spreading — and if Bitcoin takes over we are all in a world of hell. It is unregulated. It has already reached its transaction volume maximum and it is partly controlled by Chinese miners.
With E-Coin we control the ledger and the mining servers — we are the authority. I will make sure that you will have visibility into every single wallet that’s opened: every loan, every transaction. Which means we can start making new assets. Which means we can start rebuilding the banking sector without you having to inject even more politically unpalatable federal funds into it.
JACK: The President will laugh in my face.
PRICE: But he will know that this is the right thing to do. This is going to be controlled by a good old-fashioned American company. You want to regulate it? Be my guest. Regulate the shit out of it. I’ll give you backdoors, side-doors, trace — whatever you want. Just don’t. Shut it. Down.
Facebook has recently talked about merging Messenger, WhatsApp, and Instagram chat into one while "also keeping end-to-end encryption."
Why would Facebook cuts its own access to all the data ming it was getting with the Messenger and Instagram chat? It wouldn't, unless it uses "faux" e2e encryption. Besides, Facebook currently allows advertisers to send ads Messenger users. How would those ads be managed by FB's servers, if the encryption is truly end-to-end between the sender and the receiver?
Also, this "new encryption system" is exactly what the founders of WhatsApp left:
https://www.nytimes.com/2018/04/30/technology/whatsapp-faceb...
Fortunately the employees who work on security stuff tend to be anti-authoritarian so it will leak out of FB pretty fast unless it was some secret court order.