The $1.5B Bybit Hack
blog.trailofbits.com
blog.trailofbits.com
- The multisig smart contract is owned
- The computer you're signing on is owned
- The hardware wallet (ledger, trezor) you're using is owned
The multisig contract in question here (Gnosis Safe) has shown to be incredibly robust, and hardware wallets are very difficult to attack, so the current weak point is the computer.
Cryptocurrency companies need to start solving this by moving to a more locked-down, dedicated machine for signing, as well as actually verifying what is shown on the tiny hardware wallet screen instead of blindly clicking "yes".
For that matter, I know signatures are long and human-unfriendly, but isn’t it on the order of a couple hundred bytes? Surely $1.5 billion buys transcribing the putative signature request into an isolated machine in a known state, validating/interpreting/displaying the request’s meaning on that offline machine, performing your signing there offline, copying down the result, and carrying the attestation to your secret conclave lair to combine with the others’ or whatever?
(No, I won't suggest carrying the BIOS chip around)
They were attacked when they went online
If hackers can get remote access and 'manipulate what signers saw in their wallet interface' that doesn't sound like cold storage to me.
You can do the transaction on an airgapped devices and manually copy it over, but that's different from just cold storage. It also may not have actually helped here.
Using something like trezor, the keys don't leave the device. It gets sent something to sign, you sign it and the result goes back. But if what you think you are sending to the device isn't what's actually being sent it depends on you catching that.
If the thing is "transfer X native tokens" then it's more obvious what the impact of that is, but it doesn't have to be that. Perhaps it's adding a signer, changing a setting, altering an address. Worse, perhaps it's making some change to a contract that allows those things, but isn't as clear what it's actually doing. Worse still if the target address is close to what you expect - perhaps you think you're shifting tokens to another storage wallet, how many of the characters of the address do you check on the device itself?
Air gapping doesn't really change any of that it just makes it a bit slower.
The internet is adversarial, a cold wallet should only be reachable by a wrench attack.
Right?
it is possible to infect the offline computer by infecting a USB drive with stealth malware which then propagates to the offline one.
It could also be an inside job in exchange for an employee getting a kickback from N. Korea . it's not like this has not happened in the past. Imagine being a low-paid employee at an exchange and being enticed by an offer for tens of millions by North Korea to pretend to be hacked and infect one's own computers with the malware supplied by North Korea. This would be easy for an employee to do, who has access to the computers, and then pass it off as a hack.
"Cold storage" has come to mean that the keys are stored in some offline location. It doesn't necessarily mean that the keys are hard to access or that the money being moved is otherwise hard to get to. That is used to be what it means, but practically, a wallet on a hardware keychain is called "cold" exactly the way a wallet whose keys are split up on slips of paper between 5 different physical vaults is "cold."
Coins are on the blockchain… what it held offline is the private key giving access to move those coins
The emphasis is on running the correct software. If you have to input cryptographic data every time you boot that's okay because you're offline and should be in a secure room (no internet connected devices).
But yeah, malware attack is still possible if you don't have a secure chain and that's a long one.
But online, it's the wild wild west. The North Koreans can throw anything they want at your systems and the main response you get is "lol get good noob, should have built more secure systems" despite the opposing side literally having quite literally hundreds of people specifically trained to take on organisations like yours.
Not saying the Bybit people couldn't have been more careful of whatever, but let's appreciate how wild the online environment actually is sometimes.
Factories are not designed to withstand sustained aerial bombardment because the chance of sustained aerial bombardment is small to non-existent due to effective (geopolitical) mitigations.
But, if you are in a active war and being actively bombed, then you absolutely design your factories to be resistant to sustained aerial bombardment. You do not just throw your hands up in the air and say: “Who could have expected this totally routine and expected situation in our operational environment? We can not be blamed for not adequately mitigating known risks and intentionally mischaracterizing our risk mitigations as adequate for commonplace risks we know we can not adequately mitigate.”
If there were effective geopolitical mitigations that made the chances of a attack minimal, then your argument holds weight. But, that is not the case. Failure to accommodate for known, standard, commonplace failure modes is incompetence. Deceptively implying you do mitigate risks while lying or with a disregard for the truth is fraud and maliciousness.
There is also a second problem with your argument which is the relative accessibility of executing these attacks being trivial compared to military operations; being easily within the reach of lone individuals, let alone groups, organized crime, or entire governments. They require 10,000% security improvements to actually stop commonplace and routine attacks. But that is a longer argument I am not going to get into right now since the qualitative argument I made above applies regardless of the quantitative difficulty.
That's not really a viable strategy. It has been tried a few times - Mittelwerk and Kőbánya spring to mind - but you can't really build a self-contained factory. If your enemy can't bomb the factory, they'll bomb the roads and railways serving your factory, they'll bomb the worker housing, they'll bomb the less-sensitive factories that supply your factory with raw materials and components. You very quickly run into the diseconomies of operating under siege conditions.
At least during WWII, it was generally far more effective to rely on camouflage, secrecy and redundancy. Rather than having a super-fortified factory that shouts "this is vital national infrastructure", spread your capacity out into lots of mundane-looking facilities and plan for a certain level of attrition. Compartmentalise information to prevent your enemy from mapping out your supply chain and identifying bottlenecks. Your overall system can be highly resilient, even if the individual parts of that system are fragile.
If nobody knows where your factory is, it looks like a parking lot from the air and you have multiple smaller factories instead of one big factory to mitigate the impact of a damage event you are resistant to aerial bombardment, even if your ceiling isn't any sturdier than a normal factory roof. Same if the factory is out in the open but everybody thinks your drone factory produces windshield wipers
But thank you for elaborating when I was too lazy to. It helps further reinforce my point that the key is mitigating the risk however you can, not specific risk mitigations somehow absolving responsibility.
All true, and German WW2 production kept increasing despite the bombing.
A normal bank being robbed for 1.5 billion, ESPECIALLY by a pariah country like North Korea, would absolutely not be met with "oh that was definitely your own fault" as many of the sibling comments seem to imply.
Routine harmful cyberattacks is a problem. You do not get to abdicate responsibility because it is too hard. If you can not handle the operational environment, then do not operate in it.
Maybe the solution is “go to war due to cyberattacks”, but that is not happening right now so their systems are inadequate for the expected operational environment (i.e. incompetent). And everybody knows this is the operational environment, everybody knows they can not deal with expected problems, and everybody does not adequately inform their customers because it would be detrimental to their bottom line.
The Korean War ended with an armistice signed on July 27, 1953, which stopped active fighting but did not establish a formal peace treaty.
https://en.m.wikipedia.org/wiki/Korean_conflict
I know that soldiers stationed in South Korea get paid at the wartime rate.
Maybe the US and North Korea will sign a peace treaty in the 24th century. Captain Picard can mediate.
The chief US adversaries have the advantage of national firewalls, and less of their crucial infrastructure is online, so it is perhaps less effective against them. Or for all I know they are subject to equivalent thefts every day and just keep it out of the news.
It’s like a building occasionally gets hit by a shell and we dont get on a war footing.
The closest analogy I can come up with is England in the 1600s and early 1700s. Fairly regularly ships would be attacked by pirates from North Africa, and sometimes an actual land raid woukd occur- pirates from North Africa would take slaves from small seaside towns.
It was not till Englands navy grew strong enough that the threat was eliminated - and perhaps that’s the real issue here - we know it’s happening, we cannot turn the Wild West into urban peace, so we just have to keep taking the licks and keep building more secure and stronger
I like your point, but that it a hell of an analogy. 1600 is when they formed the East India company, which was basically a state sponsored bunch of pirates, looting the wider world with its hundreds of thousands of soldiers. https://en.m.wikipedia.org/wiki/East_India_Company
It is silly to point to situations where the risks were mitigated as evidence that you do not need to mitigate the risks as the person I was responding to did. You can do that to argue that we need to mitigate the risks in a different manner, but not to argue that you can not be blamed for not mitigating the risks.
And to examples from history, we could look to Israel’s anti-rocket defenses as an example handling occasional shelling. Ancient castles and walls as an example to handle stray bandits, mercenaries, and armies. Private merchant naval vessels of the 1600s who routinely had their own cannons. Armored compounds and communities in areas with high crime. Armored trains and trucks. This is standard practice. We just figured out more effective and cheaper collective mitigations. But until that happens, you need to handle it yourself or you are incompetent.
What has changed is that there is an digital (as opposed to gold) international form of money whose transactions cannot be reversed or stopped. Bybit and those holders of large crypto are operating with a fundamentally different threat model where its worthwhile for an attacker to invest millions of dollars of effort (for the Bybit payout even tens or hundreds of millions) attacking them. Everyone else just needs to worry about getting ransomed for a much smaller amount.
The members are state sponsored and young/bright. Top 0.1℅ academic sorts. At one point, the BBC got access to a conversation with one of the hackers, and their only question was "how much do you get paid?" (the context was that the hacker thought they were talking to Someone else in the tech space)
Apparently they aren't paid very well at all. Far less than the average Western IT worker. Their lives are not luxurious either. They're in barracks style living quarters with strict schedules and travel. Presumably, the anonymous Lazarus hacker was putting out a probing question because they must have been ruminating about what life on the other side would be like, what they are really worth, etc.
That's part of the power of Lazarus, the ability to dedicate resources far in excess of what most expect due to their indentured servant hackers (the opportunity to join is presented as a gift, Which to some extent it is because it does come with the extremely rare opportunity to travel. Many of them are in China.)
It's the mob attacking a casino and making off with chips. That people keep valuing those chips is one of the mysteries of our days.
I think it really boils down to plausible deniability, and the fact that it’s convenient for the governments on the receiving end to ignore the damages done to private citizens when there’s no physical harm and clear responsibility.
No president is going to bomb NK because they attacked a crypro exchange. Maybe they should, but it’s not something the public will support. So it’s easy to say “oh well we don’t really know for sure who did it” and call it a day. It’s our own fault.
I also agree that private citizens have a responsibility to secure ourselves, but where do you draw the line? If I don’t have an AA gun on my roof, am I responsible for enemy warplanes bombing my business? Isn’t this partially why I pay taxes?
Russia kills people in the West with nerve-gasses or Plutonium, cuts electrical and Internet cables, blows up ammunition factories or puts incendiary devices on cargo airplanes and there are no repercussions.
But yeah, it's quite baffling how in a couple years we seemingly went from stealing email addresses to credit cards to straight up billions of dollars.
If we can expect that everything shifts online eventually, where will this end? Clicked on the wrong link? Guess your house is gone... tough luck.
Cryptocurrencies avoid a central authority, but by doing that, they also avoid any possibility of human discretion, oversight, or recourse. There is no institution to appeal to, no customer service to call, and no regulator to enforce fairness.
Which I guess the idealists would say is part of the point: “first they came for the DPRK extortionists, and I said nothing,” etc.
Could conceivably, under different circumstances, say no. And are uncomfortable with that state of affairs.
Or to be snarky. Doesn't it seem that the crowd that gets up in arms about unlawful search and seizure are the sort of actors who correctly anticipate that the legitimate authorities would take issue with their behavior?
Right now, if I want to avoid my dollars being among those billions stolen, I can (and do) keep them someplace far away from instant digital currency. With firms that, while they could move large sums of their money somewhere else, build in a whole lot of friction in proportion to the amount being moved—by their customers, their staff, and their counterparties. Limited and well-understood modes of potential malfeasance, and strong structural discouragement for each of them.
There is nothing that I need to do that needs to move fast. But I’d hate for the firms servicing my slow, boring needs to be tempted by the new shiny.
If you did this for non crypto there would be lawyers, bankers, etc involved in the transaction.
Root certificate authorities have already solved this problem with signing rituals which take place in person in an air gapped vault on specialized hardware and multiple parties as witness.
The hackers, having gained control of the smart contract, proceeded to empty it of funds.
Splitting funds over 100 wallets would’ve helped. A 100x lower amount would be lost.
And/Or having separate hardened devices used only for signing.
Does anyone know how many signers there were/are?
Have the signing devices been iOS or Android phones?
Shouldn't a signing device be (in such a case) used for signing only, e.g. a QR-Code signer with no OS that specifically shows the inputs and outputs to the transaction?
If Bybit doesn’t collapse (can handle all the on-going withdrawals), then Bybit lost money that they’ll need to recoup through operations.
Currently it’s trending towards the second scenario.
The main takeaway I have is that their “cold” wallet wasn’t very cold and they’d messed up a lot of their diligence, so I’d also read any statements from them as the products of damage control similar to how companies talk about “nation-state threat actors” trying to make it sound like you have to be the Mossad to exploit a Citrix patch which wasn’t installed for most of a year.
The article is also pretty clear about the method that was used to compromise ByBit and how it has evolved from previous hacks on cryptocurrency exchanges.
Sometimes it really is a nation state actor, and whilst it may be a stretch to blame a threat actor of this level if your user data was stolen, this is 1.5bn in fungible cryptocurrency, just the sort of thing a pariah state requires and can launder with minimal risk of arrest or any judicial action really.
This was a multisig - meaning M out of N signatures from different signing devices were needed to sign a transaction. The attacker infected enough signer devices to go unnoticed and the signers failed to verify what they were signing on air-gapped devices
This is the part that really surprises me given the amount of money involved.
It's a security domain issue. A highly secure system involves highly secure controls. Bypassing those controls for lower risk activities will typically reduce the security of the entire system. You need an entirely independent low or medium risk system.
The software development practices of banks are probably a good example here.
While it is nearly impossible to steal $100mn from one of the mega-banks, those <expletive> crypto bros, a bunch of failed morons (self-proven by all these hacks), manage to lose people's money. Now.. I am not defending the banking system (and its ethics/morals), but damn-it they do a f-a-r better job at IT Audit/IT Compliance/IT Sec (my bread and b utter for decades).
As far as I'm concerned, if you're parking money with a company based in an area that has lax regulation you're holding the gun that'll shoot your foot. I have a hard time seeing something like this happen at Bitpanda or Kraken, though you never know.
Unless you're one of the devs, and you just fork the chain, like Ethereum.
Bankers are a bunch of idiots, too. I know this to be true because that one investment bank collapsed a bunch of years ago.
In all seriousness though, ETH is just a commodity; a bearer instrument; a thing. It's similar to gold or cash in some ways. If you store it properly, you're fine. If you give it to someone untrustworthy who loses it, of course that's a problem.
Well-regulated banks can start holding crypto on behalf of customers as soon as they're given the regulatory go-ahead. They've stored gold in vaults for thousands of years; they can store crypto in digital vaults too.
They're already doing it. Most crypto or crypto-adjacent product you'll see traditional firms is relying on a provider white-labelling crypto exposure.
I have a hard time feeling sympathy here because I consider cryptocurrency to be fundamentally silly. Reversible transactions of fiat currency transactions is a feature not a bug.
I feel like securing something like this is practically impossible. There's always the risk of a bad actor who introduces malware for a small fee.
[1]: https://www.chainalysis.com/blog/2024-crypto-money-launderin...
Reversible transactions would generally be a bug regarding cash & hard assets of which cryptocurrency is trying to imitate.
It's not a bug if both parties give consent, which sounds like a wonderful way to transact, to me!
Thieves will not tend to consent to reversible transactions.
We all want:
"Reversing for me, but not for thee"
No, of course not.
Adjusting your comment for the situation: > Is $100.00 in cash silly? It has the same property (non-reversibility)
No, not silly if that's what I am comfortable to keep on me (wallet, mattress, etc) and I'm mugged/robbed most people will recover. (Especially if you're also able to afford the inherent risk of crypto.)
> Is $1,500,000,000.00 in cash silly? It has the same property (non-reversibility)
YES! And probably a challenge for most humans even if you're able to get that cash in the limited US $100,000.00 bill [1] - that's 15,000 green slips of paper. (I'm making a bold assumption that this link [2] is reasonably actually for the physical scale, though this apparently only shows 13,000 not the 15,000 needed.)
They effectively treated the $1.5B like a pile of cash in a fence with a few (easily pickable apparently) locks keeping it shut.
That SHOULD have been in a 100% offline, air gapped system with multiple levels of 2+ person approvals to access.
But this failure implies to me that even THEY didn't really consider the crypto assets they were holding as something with a real value either.
1- https://en.m.wikipedia.org/wiki/United_States_one-hundred-th...
- Height: 66.3mm
- Width: 156mm
- Thickness: 0.0043 inches = 0.11mm
- Weight: 1.0g
So the volume is 1138mm3. You need 15M notes so that's just over 17 cubic meters or approximately 603 cubic feet, which is a cube roughly 2.6 meters (8.5 feet) on each side, weighing in at 15 metric tons or 33,000 pounds. Put another way, that's over half the volume of a standard twenty foot shipping container (~1100 cubic feet).
But let's get it more compact. The current gold price seems to be about $2939 per Troy ounce, which is 31.1035g. You need 510,378 Troy ounces, which is actually heavier at 15.87 metric tons but way more compact. Given a density of 19.32g/cm3 that's 822,000cm3 or 0.822 cubic meters or 29 cubic feet.
Whatever the case, it's a lot less practical to steal.
[1]: https://en.wikipedia.org/wiki/United_States_one-hundred-doll...
Cryptocurrency was designed as a political statement rather than a serious banking system so you effectively have the same level of precaution for both large and small amounts, akin to a bank keeping a billion dollars in the teller’s tray.