Signal's Cellebrite Hack Is Already Causing Grief for the Law
gizmodo.com
gizmodo.com
General security/crime prevention concerns were being provided as reasons for these searches. I had voiced my concern to my managers but they said as long as we had a paper trail which clearly tied each request to the info provided, they didn't really care. I left the place shortly afterwards, but it highlights, I think, how quickly these things spiral out of control. How specific turns general, how true belief turns into mere suspicion or blanket statements (ex - everyone in THAT area is a criminal so we need all the info).
What approximate time did you work there? Did you perhaps get to experience the difference between the situation before and after the Patriot Act, or before and after the emergence of ISIS? Or was this just the usual scope creep?
I'm old enough to have witnessed 9/11 live on TV and as I see it, there has been a downward spiral in the areas of privacy, the barrier for probable cause and the presumption of innocence - particularly in the US but almost equally egregiously here in Europe. Honestly, COVID isn't helping either, because it's just enough of a threat to be considered a security concern.
A representative from Italy might ask for one specific element of information, another one from Germany might look for associative information (who a person is and who they were with) and one from the UK might ask for things as precise as the hour and minute at which a particular action would occur (this is due to the high amount of cameras that cover London for example, which could then be used to match movement around a region with surveillance to the data I would provide).
I would say most of the requests were reasonable from my point of view given the background they would provide - I guess it's also down to the person requesting the information. But as time progressed they figured out that I was able to get information or that we had information on a wide variety of elements... As I said initially, it goes from "Find info on John Doe, with an email account jdoe@email.com, with a date of birth of 33/33/2033 who did this activity at 11:30 PM on 01/01/2053" to "We have this email which is either jdoe@email or jdoe@gmail but we're not sure. Can you check all such occurences?".
I think the point is also that Europe isn't THAT much better and while people might wave privacy laws and GDPR as proof of Europe's "superiority", I think a lot of those things look good on the surface, but they still don't protect you as much as you'd think.
For instance, take someone in your role or even a team of someone from enforcement and someone in your role, if 1-2 individuals get in that situation that are corrupt, it can turn into a blackmail or potential business espionage scenario. Since requests/warrants are being suppressed to keep a low profile, even finding out what and how information was looked up would be easy to pass for some scammers or organized criminals. Someone could sell access to information and start targeting many people. Scams could be run on wealth or small competitors suppressed.
The other side to this is, I wonder if you worked with anyone, or if people in your role, ever looked up information on people they know or potential competitors/enemies to use against them even if they did nothing wrong? What kind of oversight would be there to watch that, especially in the more underground operations in law enforcement or national security? Cops now in most places can look up anything on your phone. Are they doing it to family/friends/business people that they want to have some blackmail or upper hand on? Are they spying on people for their benefit, not for a crime? Probabilistically yes.
The gaping hole of security is the human, a few bad people with access to that data and you have some authoritarian powers that have never been seen in history to be abused. Giving individuals that much power will end badly, there is no way someone could control themselves if there is little oversight. If a law enforcement officer or security/intel professional has to provide no trail of that, it will be abused. It will be abused even with the trail.
IIRC Marlinspike didn't disclose one vulnerability, but a massive class of them: Cellebrite must process media in every possible format and thus relies on components from outside sources, FFmpeg being just one example. Cellebrite inherits security vulnerabilities from all those components, and doesn't appear to be diligent in keeping them patched (not that a perfect record of patching would protect them - the attack surface is enormous and probably most component developers don't provide sufficient security).
Integrating insecure code is likely a necessary decision for Cellebrite: Writing, maintaining, and updating their own code to process all types of media, and keep up-to-date with all changes to all those types of the media, seems impossible and certainly uneconomical. What could they do? Maybe they could sandbox each component but then they'd need a lot of horsepower in their device.
Googling "cellebrite forensic extraction device" shows a rugged laptop-like device in a case the size of a desktop PC. I'm not sure if that's the real thing or not, but in any case it doesn't look like space is a concern. There's plenty of room in that suitcase for a modern CPU, lots and lots RAM, and as much battery power as you need.
For reference look at panasonic's "fully rugged" toughbooks. A baseline 31 will cost you $3700 for a 7300U, 16GB RAM, a 256GB SSD, and a 13.1 1024x768 display. On base spec it'd be a $800 laptop if you could find one with a screen that bad, and that outdated a CPU.
Except you can drop the 31 from 6ft, you can literally hose it down, you can rub it in a dirt pit, you can drive on it with a truck, and it's expected to survive and work fine. It also has a baseline 20h battery. That resistance, that ruggedness, comes with limitations. And it's not very useful to have an indestructible laptop if a gentle shake kills the electronics either.
Hence "thermal and reliability concerns". You look for parts you can cool with a relatively low thermal budget — which limits your performances — and you look for parts which have been designed and manufactured for industrial resistances, which also limits your performances. No matter how you slice it, a GTK Boxer won't outrace a Bugatti Chiron.
[0]: https://blog.cloudflare.com/sandboxing-in-linux-with-zero-li...
Sandboxing _can_ be without overhead, but in the case of firejail it's definitely not the case.
Could you please name which application, and how do you notice it?
I don't know if it also makes a difference during execution.
EDIT: it's kinda amusing to get downvoted on this.
a) There's no performance issue. Just use linux sandboxes
b) There are these possible performance hits
c) I use those sandboxes and I have observe X performance issues.
a) Your performance issues are not valid performance issues, because I say so.
Ok, gotcha
Now, there is some overhead in forking a child process to have different capabilities than the parent, and some overhead in using a pipe/shared memory to communicate between the sandboxed child and the parent, but that overhead should be tiny compared to the work of decoding compressed video, etc.
seccomp has very little impact but it's not 0:
https://wiki.tizen.org/Security:Seccomp#Performance_Analysis
firejail or systemd isolation techniques (using a variety of methods such as mounting a private version of directories or initiating ACL's at application start) are extremely lightweight but not 0. Most processes (regardless of desktop or server) usually live long enough (more than a few seconds anyway) to justify any theoretic impact. Even firejail on an ancient desktop system will not change things so that it's noticeable by a user imo.
Essentially, I'm imagining a bunch of isolated VMs. How is that wrong in practice?
Out of boredom, I started 400 containers with podman running only bash in Ubuntu. It used less than 1 GB of RAM in total. podman is likely overkill if you just want sandboxing, however.
I'm sure there are still tradeoffs to containerization but at least the overhead is minimal.
Your OS already has a layer that checks, when a process tries to open a file or access a bit of hardware, whether you have permission to access it. Your CPU has a layer that checks, when you try to read or write a given memory address, whether you have the right permissions. And so on.
A lot of sandboxing amounts to making those existing checks more restrictive - telling the OS "If this program running as John Doe asks to open a file, instead of treating it like John Doe treat it like someone with no files at all" - which is no slower than what the OS was doing anyway.
The sandboxing that comes with performance penalties is generally doing more than just sandboxing - for example, Ubuntu's 'snaps' have sandboxing, but they also duplicate a bunch of libraries for compatibility reasons.
There is a middle ground of "force update software on the device more than once a decade." Yes, FFmpeg will always have vulnerabilities, and recreating their work would cost an enormous amount and be less secure. Properly distributing security patches is an affordable alternative.
At least that's what I would do.
But since he advertised that one, anyone wanting to compromise Cellebrite has an easy target. So Cellebrite has to fix it.
Sandbox prevents malicious code from escaping, but not from lying to you.
Edit: it seems like they downloaded the file from Apple's site. The order says using the IPSW file was fair use, and that doesn't appear to depend on them downloading it vs shipping it. I don't think that makes a different. The cellebrite usage is also fair use for the same reasons (doesn't compete with Apple, transformative usage, etc)
...What? Are cops frequently scanning people’s entire phones as a first move now? I had heard that this occasionally happened at borders or with serious crimes, but I had no idea it was this widespread.
Cops do whatever they want
> I had heard that this occasionally happened at borders or with serious crimes, but I had no idea it was this widespread.
See the first point
Some people I know who were arrested during the series of Portland protests last year, whether or not they were even active participants... (you may have heard of some people being snatched from the streets into vans, for example)
...mentioned that their phones were missing their MicroSD cards when they got their items back.
Let that sink in.
This works surprisingly effectively even with a family, purely by running a voip client on my desktop, and ensuring my mobile forwards to my voip line, meaning I can still receive calls as necessary from schools, daycares, etc.
It's an experiment I intend to only push further and further. Clearly also pull back, but a fun one.
Give a general a jet plane, and he will sooner or later bomb some people with it.
Give a politician the power to classify information, and sooner or later he will use it to cover up some shady stuff he did.
Give a cop the power to scan a phone, and sooner or later he will scan every phone he encounters.
This is exactly why the government should not get a "backdoor" into E2E encrypted chat: today it will be used to prevent a terrorist attack, tomorrow to send your mom to a gulag for thoughtcrime.
A turned off iPhone is completely unusable for most of their intentions, still will be taken and only given back after x months.
On a similar note, nation state hackers already have false flag tools in their arsenal, where they “plant” evidence of other nations doing an attack.
The point of the Merkle tree in this case is just to make tampering much more difficult/ easier to suspect; independent entities could hash he result and compare for evidence of tampering.
This isn’t a case of absolute perfection, this is bringing things up to the current standard (aka “state of he art”) WRT courts and police procedure.
Sure, one could design all sorts of additional mechanisms (error correcting codes in the trees etc) but realistically, it’s tamper detection that matters, and it only needs to be as good as paper, candlesticks, fingerprints (sigh) or whatever else is already customary in the evidence room.
This is false. The chain of custody and integrity of digital evidence is pretty much a solved problem. Digital forensics is a fairly mature field.
This is not to say there aren't issues with some of the tools, such as Cellebrite's clown show.
Trust and Computers don't go hand in hand [2].
[1] https://softwareengineering.stackexchange.com/questions/1947...
[2] https://pluralistic.net/2020/12/05/trusting-trust/#thompsons...
Sounds like someone took code from their homework assignment and added a few fun extra credit features to screw with their psychology professor.
I can imagine how it would be pretty confusing to stumble across if you hadn't read the paper though :-)
The mere possibility that a "clown show" of that magnitude can exist is evidence for the lack of a solution to the problem of digital forensics.
This has not stopped courts from accepting digital evidence and the creation of a "digital forensics" industry. Perhaps the only persons who could successfuly call this into question are the same people, so-called "experts", who are supporting its continued existence.
Imagine a scenario like FBI Bob and EFF Alice show up to image a suspect's hard drive. Both of them image it, Alice hashes it and throws away the image, Bob keeps the image, then if there's a dispute, Alice is called in and provides her hash and if Bob's image doesn't match it, the tampering alarms go off.
Q: How could you verify that a write blocker was actually used, and that it functioned as described?
If course, if they already trust each other (including all the used tools), then the whole excercise is moot.
They would need to trust that the bridge is not malicious. And that is a whole other rabbit hole. But I think it is possible for them to attest the firmware/gateware running on it, through some convoluted cryptographic ceremony.
They'd get a live notification of tampering, and could have independent signal blocksers that could physically block the command from actually arriving at the drive (assuming they use a few hundred meters of coiled fiber as a delay line).
The receiver on the drive side could even be a single phtotodiode, which could be made to allow easy verification with, say, an electron microscope if you're really paranoid. There are probably ways to use field-suitable technology if you only need to ensure the photodiode has the same structure as what you expect.
Cryptography won't help you with trusting hardware. Delays and intervention-ability would help, though.
But in any case, this is not really the level of concern here. It's equipment that tampers with the device. The only way to be sure is to roll your own. Which holds for both sides. So the perfect systems needs to be created by two adverse parties, which means it's impossible to do. Qed.
(In the real world with physical proof this is different since tampering is much harder and it's a problem worked on for centuries. It's not bullet proof either but much more mature.)
The hash verification you describe happens (without independent oversight but enugh to protect against a single bribed/malicious officer) and it does protect from post-factum altering of any digital evidence; you try to do the early (physical) parts quickly, the majority of analysis work comes after you have the images and their hashes - but everything from that analysis can reproduced from the verified images if it's disputed.
At least in Sweden (where I can follow the discussions easily) the defence are raising questions about the legality and to some extent, the validity of the evidence from EncroChat. So far it seems seems the courts are accepting the evidence.
Also, it is quite amusing to read the EncroChat logs from some of these trials. The user names selected and the messages sent sometimes shows very bad opsec.
That's interesting, where can I find those?
Did you see "The Last Dance?" Popular lockdown fare made the Michael Jordan hagiography more prominent that it would otherwise have been.
Scotty Pippen v Patrick Ewing [1] suddenly became more prominent in the minds of many. We were lacking excitement and drama in our lives. Social media was full of this. Lazy writing follows the zeitgeist. Brutal, disrespectful dunking became more of a thing again...
It'd be like going in and saying that someone could reprogram the red light camera to superfluously give out tickets and record incorrect data. Almost certainly true (there's no way these systems are anywhere near secure), but try that argument in front of a judge and I believe you'll find it won't get you anywhere.
Over here a guy got a speeding camera fine thrown out because the authority running the cameras couldn't prove the MD5 hash they were using was adequate to verify the evidence.
I can't quickly find a detailed explanation, but here's a reference: https://www.schneier.com/blog/archives/2005/08/the_md5_defen...
If my memory serves, the actual argument was that the MD5 hash was only covering the photograph, and that the speed/date/time stamps weren't included in the hashed data and therefore _could_ have been tampered with.
Who does the better job of framing the opponent?
Does X start off getting busted, as though framed by Y, only to have the truth be X framing Y to appear as though framing X?
/s
And if they wrongly testify that a genuine video is fake, it's much harder to claim that they submitted false evidence.
For how long?
Currently. Currently, experts only can determine if something is a deep fake.
Considering that we're still in the infancy of deepfakes, this is not a reliable defence against deepfakes being used as evidence.
We have to first graduate to "sender can write anything in From: header of an e-mail" levels of understanding of technology before tackling deep learning.
--
[0] - I'm very much going by anecdotes here, so if they aren't representative, my whole comment is irrelevant.
> So what you're claiming, Mr Defendant, is that these Signal messages found on your phone were not in fact written by you, but were placed there by a virus that someone else had put onto police equipment when they were previously arrested? And this virus also knew your exact writing style and knew to place incriminating messages to your childhood schoolfriends? And you didn't mention this possibility at your original trial?
Furthermore, the attacker would _want_ to mimic the suspect's exact writing style.
The real problem I would assume is: The software used to produce the incriminating material has been alleged to produce incriminating material out of thin air at will. This certainly should change your perception of the material produced by this software (even if in this case you find it does not change your conclusions).
https://www.schneier.com/blog/archives/2021/04/security-vuln...
Budget Android phone containing basic payload against an open source ecosystem and boom... you got a research paper, baby!
You will need a very good lawyer to explain that you didn't hack them but they fucked up by trying to steal your data and "by accident" ran into a special file you had.
Then make sure you cannot legally be compelled to give them the decryption key. But I guess you should aways be able to "forget" the key, can't be forced to remember something.
And now I'm wondering - since the exploit only requires that the bad file be parsed, couldn't you put an appropriate exploit in place just by sending the file via email, sms, or airdrop? As long as it's someplace Cellbrite will see and process it, you're good.
It sure seems like the overwhelming majority of law-abiding citizens with an opinion disagree with the extent of electronic surveillance (maybe it’s just this echo chamber). So, is democracy working? Are we all grossly misinformed about the terrible things that this prevents? How can democracy work if we are so uninformed?
People talk about abolishing police, and I wonder what would come of that. I believe that the popular will toward some form of community protective services is so strong that it would materialize out of necessity. But in what form? Even the cartels deliver world-class public safety for their territories, but only if you tolerate their own atrocities.
The justification of electronic surveillance is mostly stated in terms of physical crime prevention. This suggests that the forces on the ground are basically incompetent. Any sort of real-world crime that would require large-scale electronic communication, involves a lot of people, and leaves a lot of physical evidence. It just can’t go on within a community that trusts and invites law enforcement.
Likewise in the electronic world, certain levels of surveillance are welcomed. Most people would probably rather use a platform that recovered funds from hacks and scams. But this doesn’t seem to be its purpose. Actually we have no idea what is being watched or why, and the only thing we observe from officials is political shoe-banging against whatever group of citizens are the objects of today’s moral outrage. So it’s pretty easy to become extremely paranoid, and reject all forms of surveillance in favor of accepting the risk of all those terrible things that it may or may not have prevented.
So if this is the wrong idea, it seems pretty easy to fix with transparency. What do we really gain by operating in the shadows? I see this as somewhat similar to the comparison between proprietary and open source models. The Britannica is dead. Transparency allows trust and collaboration from sources that never would have been able to contribute. And what if criminals knew what evidence is being collected on them? My best guess is that 99% of criminals would give up at the first sign of trouble. Transparency is the ultimate force multiplier. The greatest victory is to win without a fight. We forget this. It feels good to fight and win and get a medal at the award ceremony. Transparent power is boring, safe, democratic.
I find the problems of integrity in that data to be worse. Tying right into that narrative is "Coded Bias", a documentary investigating the bias in facial recognition algorithms [0]. Available on Netflix [1].
Cellebrite has told all customers to stop using the physical device on iPhones. Could be related to the stolen Apple DLLs in their product. Any competent legal department would tell them to remove those immediately, because you don't want to get in a lawsuit with Apple over such a clear case.
Traditionally a non volatile memory chip is forensically read by being removed from the board and read out directly. Each sector is read from its memory registers. This is a robust forensic method since 1) its passive 2) its repeatable 3) you can fit the chip into another surrogate device and obtain the same results or use a different piece of software to obtain the results
If what cellebrite is doing is altering the memory when it interrogates it, this breaks the chain of custody. The process cannot be repeated.
[1] https://news.ycombinator.com/item?id=14864197 (from the top comment):
> Linked below is an example to restrict military use. Note though, it's so broad reaching that it might scare away even non-military organizations for using your software. And it still doesn't address how you enforce such license. So there's lots of questions about the applicable of this example license. http://web.cs.ucdavis.edu/~rogaway/ocb/license2.pdf
--------------------------
Also LEA could very well now be reminded to not break the law themselves and put pressure on vendors. Somebody in charge of procurement doesn't usually care much about the finer details of whether it works or is the right tool (they have other people to decide upon which tools they might find useful to procure). But procurement does take not over licensing cost and its terms.
Also licensing cost has always been a issue with justifying use of Cellebrite. Any excuse to switch to a different product would just be icing on the cake. E.g. 1 seat for Cellebrite costs about as much as a Grayshift license for the whole team.
[1] whether that will improve things for the person that finds themselves as the target of these tools is another matter.
Hahaha. I can't wait till HN finds out about how rubbish the other vendors are.