Apple Bloomberg Congressional Letter
documentcloud.org
documentcloud.org
One guy named as a source in the original article does not accept the story as written.
When asked what, exactly, he found strange about Bloomberg's claims, Fitzpatrick said, "It was surprising to me that in a scenario where I would describe these things and then he would go and confirm these and 100% of what I described was confirmed by sources."
You could take that to mean "Bloomberg took a scary hypothetical and pretended it was real," but if they did have other sources telling them "There are these backdoor chips in servers" this is exactly what you'd expect Bloomberg to do: go to a security expert and ask "Hey, does this really work?"
It doesn't particularly read like evidence one way or another to me.
Technically possible doesn't mean it happened. You could be punched by your wife, does that mean she did punch you? Sorry for the inappropriate analogy, but you get the idea.
To this point Bloomberg has presented 0 evidence of any of this. You have to understand how the burden of proof works, you can't throw a rock in the pond and expect others to jump in and retrieve it. Bloomberg made the claim, Bloomberg has to support it. Until they do I will take everything they write/claim with a tablespoon of salt. From my perspective it may very well be an attempt to hit Supermicro and nothing more.
The fact that Apple or Amazon are expected to show evidence that a yet unsubstantiated claim from Bloomberg is not real shows how low people set the bar for what reliable information means.
"You have to understand how the burden of proof works, [...]"
I don't HAVE to do anything you say, thank god.
But in all honesty not understanding how something works makes your opinion on the topic about as valuble as a turd in the rain.
Understanding how things work before attempting to pass judgement is “just” a matter of common sense and decency. But since no superior ordered them and you don’t have to have them, here we are.
I'm not a CFP, but you should probably sell your Supermicro holdings now; they aren't gonna recover.
That a company's share price is already bruised or disreputable has no material impact here. It doesn't matter if a company has pre-existing and systemic financial problems; that doesn't mean we can be cavalier about publishing news that will wipe out half its value in a day. This kind of cavalier approach to company valuation isn't suitable for considering the consequences of erroneous stories.
Moreover I think you'll find that regardless of how much you believe the stock is worthless, there is actually an objectively quantifiable loss that was suffered here. If you still believe the stock is worthless even after this reduction in share price then by all means short it. But you can't speak for the entire market, which clearly demonstrates that many parties disagree with you on Supermicro's valuation. Presumably some of those parties will be a little miffed if the Bloomberg story turns out to be largely incorrect - you can't just shrug and tell them, "Hey that stock is worthless, you shouldn't have owned any of it anyway."
If you think there's a "fair market value" of a company that can't meet basic financial regulatory and accounting practices that is anywhere above the price of toilet paper, then I suggest you don't put any money in the stock market.
Nobody really thinks it's worth anything but the large holders don't have to write down yet and probably are diversified across the industry. The market cap is like half a bil -- that's _tiny_ for a company as big as SuperMicro...even at the bil it was before the article.
The stock is garbage.
Hard facts? Which ones? An independent review of all server hardware by an external auditor? Call me stupid but I don't think companies can audit themselves if a lot of money is at stake. Not saying Apple is wrong, but for now we still have "This isn't the case because we say so, believe us." nothing more.
It will be interesting if Bloomberg has a followup or if they fold.
I wouldn't take anything as certainty at this point, not even close.
That's the key argument.
This isn't just a "your word against mine" type of thing. Were this issue true, then it could represent a significant legal risk to Apple.
Categorically denying something that can easily be proven by an external audit (a scenario that they cannot rule out might happen), as strongly worded as they did (excerpt below), would open them up to a huge legal liability in addition to the security issue itself, and for no real upside.
> In the end, our internal investigations directly contradict every consequential assertion made in the article—some of which, we note, were based on a single anonymous source. Apple has never found malicious chips, “hardware manipulations” or vulnerabilities purposely planted in any server.
After seeing what VW did with emissions, I wouldn't say any huge corporate cover-up is out of the question. Especially considering this situation would be very bad even if did happen and they were 100% truthful about it (it would create huge problems with China for them and their supply chain).
We just can't reasonably make conclusions either way yet.
The Apple situation is basically a reporter with no named sources saying that Apple itself was fooled by outside sources. There's a HUGE difference between willful fraud (VW) and corporate embarrassment (Apple IF it was true).
I also think this goes far beyond "embarrassment" — this is something that could potentially destroy Apple's supply chain.
Correct, but this liability would (most probably) be based on showing some kind of negligence.
> so lying about it isn't really the worst option if there's even a small chance it avoids the issue
Deliberately lying about it would be fraud. Getting caught with that would dramatically increase the liability.
Since you used the VW example: do you really think it would have cost VW something around the order of $30bn if some foreign power had manipulated their emissions test without their knowledge? Or that executives would have been arrested?
> After seeing what VW did with emissions, I wouldn't say any huge corporate cover-up
This isn't a cover-up, though. This is Apple expressly addressing the issue, and categorically denying every part of it. To everyone, up to and including Congress, no less.
Let's say this was the first they're hearing about it, and it turns out to be true.
Can they publicly state the truth, point the finger at China in the process, and risk having to immediately uproot their entire supply chain?
Is there a course of action that's better than outright public denial?
Sure. "We are currently investigating this issue and cannot comment until the investigation has concluded and we have determined the attackers, their motives, and possible mitigations".
> Can they publicly state the truth, point the finger at China in the process, and risk having to immediately uproot their entire supply chain?
Note that there is no reason to point the finger at China yet, only suspicion, but even if this were the case: I think China would stand to lose more from any uprooting.
This would affect Apple short-term, but Apple has, for all practical purposes and intents, unlimited amounts of cash, and they could build up a new supply chain. Weren't they keen to move production back to the US anyway?
China, on the other hand, would permanently lose this production line, and access to all the IP that comes with it.
Yes, they can state that they detected a hardware based security intrusion and have mitigated it.
Bear in mind according to the Bloomberg article the government and several other companies know all about this already as well. If the Bloomberg article is true there are thousands of these compromised motherboards out in the wild at multiple companies. There is no way on earth Apple could ever get away with a denial, so why do it?
Then where are they buying their hardware? Every company that I've talked to who purchases electronics from China has had issues. Stuff isn't as ordered. Strange boards are added. You have to rigorously inspect everything. Nobody has told me about rice-sized stealth chips, but malware, old firmware (ie with known/public vulnerabilities) and under-spec parts are a norm in the industry.
What seems odd about this Bloomberg story is the difference in intent. Rather than a supplier trying to save a few bucks, or a wayward employee inserting something to harvest CC details, they ascribe this to government conspiracy, to a cold war battle between secret squirrels.
Those are brilliant hardware guys. I'm having a hard time seeing how you sneak something by them. I think maybe in the firmware? (But probably not. They've also got the most brilliant firmware security guys on the planet.)
And on the off chance that this thing did slip by, a picture was provided of the purported part. It would have been a simple matter to go back through all of your audits looking for that part. Because if I'm smart enough to catalog and record my hardware audits, that means that Apple was doing so a looong time before I was.
I don't know man?
I'm getting more and more skeptical of this story by the day.
Apple orders in bulk. They aren't x-raying every server bound for their latest datacenter. I've worked with companies that do examine for every tiny imperfection. It costs thousands per-server and means significant and often random delays (defense industry). If apple, a publicly-traded company, were so rigorous they wouldn't be able to hide the costs. So we know they aren't.
I'm sure that apple does inspect systems that it deems vital to its internal systems, but servers meant for customer use are too numerous.
On the other side, it doesn't take much of a lapse in QA to let a single bad part through, when you are dealing with billions of components like Apple is.
If they're saying something inconvenient to their government or employer, that's neutral or even positive for their credibility. If they're saying something convenient but classified or otherwise not-for-release, that's generally neutral; 'authorized leaking' is an established practice. But if they're saying something that won't cause them problems and isn't a secret, then it's strange. It raises the possibility that they're anonymous because the claim isn't true and they don't want to be embarrassed, or even that the story writer encouraged anonymity to hide the weakness of the source.
If Bloomberg is saying "we used 17 sources including intelligence officials and an anonymous source who confirmed the hack", well, easy money says the key anonymous source doesn't measure up to the other 16.
To put my tin foil hat on, will we be seeing increased pressure for physical inspections of FAANG hardware?
Basically one expert described how a hack could be done and the journalist re-wrote it as here's how China tried to hack.
>In September when he asked me like, “Okay, hey, we think it looks like a signal amplifier or a coupler. What’s a coupler? What does it look like?” […] I sent him a link to Mouser, a catalog where you can buy a 0.006 x 0.003 inch coupler. Turns out that’s the exact coupler in all the images in the story.
That's not true, and fundamentally misunderstands what journalists' job is. They're reporters not researchers: they report on information in testimony and documents they acquire, they don't do the research to to create that information themselves. They cross-check testimony and documents between multiple sources to verify the information, they don't replicate research.
Bloomberg doesn't have a bugged motherboard to X-ray. Per their story, all of those were owned by other entities. There's very little reason to expect that they have an example, since they weren't a target and the people they talked to were likely not authorized to hand over their employees property. Given that, it's unlikely they even asked for one since it would have been a foolish request.
They certainly can't compel them to make such clear, specific, long-winded, and persistent denials of claims like these. That kind of thing's the stuff of conspiracy theories and Hollywood plots.
> On Saturday night, the U.S. Department of Homeland Security joined the U.K.'s National Cyber Security Centre in saying they have no reason to doubt the statements we've made.
> Which part of all statements made so far would be a lie if an issue was discovered by some unit under DHS or a 3d party entity?
If the DHS says "they have no reason to doubt the statements [Apple] have made", it means that no issue was discovered by a "unit under DHS". Otherwise, the DHS would have reason to doubt Apple's statements about being compromised.
For example, why place a detectable piece of hardware onto the MoBo, instead of just installing malicious firmware? The hardware piece is going to be limited in capabilities and much easier to detect. Given the level of access required for this "hack," it makes more sense to just write the bad firmware to do all the tricks. Especially since, as far as I understand, the malicious chip modifies the firmware anyway.
There are far too many reasons to be skeptical of the article, and the parties involved have motives that are very easy to trust.
Because the supply-chain may be verifying the integrity of the firmware? Firmware is "detectable" too. It's detection is probably easier because so much of computing is software focused nowadays.
Hardware implants have certain advantages: no one may be looking for them and they're extremely resistant to removal attempts.
I don't personally know if the Bloomberg story is false. But I do know that, so far, it's essentially unsubstantiated. So let's assume for the sake of argument that the Bloomberg story is incorrect. What would you expect a company to do other than issue a strongly worded denial and explain their side of the story? What would you have them do, publish their entire network logs and supply chain security assessment notes just because an unsubstantiated story claims they were compromised?
In essence: it sounds to me like you're demanding evidence to disprove an accusation that doesn't have any evidence. That doesn't strike me as a fair or rational way to ascertain truth.
Now I have a few responses to your specific points:
> I am not aware of a single f500 company that was not breached at some point
Most Fortune 500 companies have not suffered a breach. Every single company in the world has critical/high severity vulnerabilities in any given software product at any given time; this is the essential nature of software in 2018. A breach is something categorically different: it consists of a technical security vulnerability which has been demonstrably exploited to achieve tangible data compromise and exfiltration. Unless you have access to information I don't, no, most Fortune 500 companies have not experienced this. Consider that tech journalism has begun to conflate security vulnerabilities and actual data breaches in recent years.
> Scanning outbound connections is not a meaningful defense in this case there are legitimate outbound connections going to say China and given China's ability to capture any inbound traffic there are def. ways to ex-filtrate data without raising any flags.
This kind of claim borders on fantasy and conspiracy theory. Apple employs more security engineers than most companies in the world employ people period. If your threat model provides your adversary with vague omnipotence such that they can exfiltrate data undetected across countries under the nose of one of the most capable information security teams in the world for months at a time, then the entire discussion is moot. If you honestly believe that's a reasonable thing to believe then our disagreement is fundamentally insoluble.
Do you honestly believe it's reasonable to say no material details were provided in this letter when you're talking about data exfiltration from Apple with no material details of how China would do that? You can't just give fantastical powers of exploitation to an adversary because they're a country.
If we bring this down to reality we have the following observations:
1. The attributes and capabilities you're assuming of the adversary in this story are not only unproven, they're unfalsifiable. We can talk about what could be possible all day long, whether it's the NSA, China, Russia, chemtrails, 9/11, etc. I can also claim there's an invisible 0day that can compromise our entire electrical grid sitting on my desktop. Prove me wrong?
2. Bloomberg's story is essentially unsubstantiated. The sources are anonymous and several claims rely on a single source. At least one source has come forward to disagree with the story as written.
3. Every company involved in Bloomberg's story has issued vehement denials. If your response to a denial is, "But there could be a gag order!" then I'm frankly curious what level of evidence you think is required to refute an accusation that has no evidence.
"This kind of claim borders on fantasy and conspiracy theory" this pretty far from my domain of expertise but you have to deal with multiple limitation 1st you are not scanning 100% of everything 2nd if you flag aggressively you will not have enough man power to investigate each flag Apple or not. By being able to install equipment and telecoms and exchange points China can easily devise an exfiltration scheme that would be practically impossible to detect (e.g. the only thing you see 1 in say 100K packets from a legit connection being dropped somewhere along the route ).
"Do you honestly believe it's reasonable to say no material details were provided in this letter" Yes
Can specify what material details a company could provide, in theory, in face of such a claim, to prove the negative?
I'm interested by the Bloomberg follow up or if they fold.
Couple this with my own personal experience with Apple sharing personal data it had no right to share, and the fact that only a single person signed this letter, and Bloomberg still not backing down, Apple has a way to go. They could start by having every person would would remotely be involved with something like this (from top to bottom) signing a legally binding letter attesting that Bloomberg's story is not true. Until you have this, it only takes one individual who hid something from Seniors to make this true.
Expect letters to Congress from Amazon and the 30 other hacked companies.
It's the only way to kill this story, after they made the unbelivable mistake of forgetting to NSL Bloomberg despite having requests from them to comment on this story for months. Somebody is getting Guantanamoed over this slip.
Given what was reported in the Lavabit case, how sure are you that there is no secret caselaw which does, in fact, require companies to lie?
There are a lot of possible explanations. As someone who finds both Bloomberg and Apple credible, this is potentially far more interesting than “one of them is totally lying.”