Many Bitcoin wallets vulnerable to double-spending of confirmed transactions | Hacker News Reader

Many Bitcoin wallets vulnerable to double-spending of confirmed transactions

bitcoin.org

188 pointsaburan2844 comments

The bug here is interesting. There are three separate pieces of new code which was added. First, is code which implements a new ordering rule. The second is consensus code which actually requires the new ordering to be in place in order to consider a block valid. The third is a flag in the header.

The consensus rule was pre-programmed to automatically activate after 995 of the last 1000 blocks (in theory 99.5% of the network) included a flag in the block indicating that they would auto-activate the new consensus rule. So think of it like green and red lights. If there is no set of 1000 blocks where at least 995 of them are green, then the old ordering is still valid.

Important to note that the new ordering is a stricter subset of the old, so new ordering is perfectly fine under the old rules, just old ordering is not acceptable under the new rules. This is called a SOFT fork. As soon as there are any 1000 blocks where 995 are green, then every block from that point onward MUST be properly ordered. (BTW, if you change rules such that something new you are doing would be invalid under the old code, like say, increasing the block size limit, it is called a HARD fork. Hard forks are considered much more disruptive to the network, soft forks are supposed to be easier to orchestrate)

By "showing green" the miners were supposed to be advertising that they would enforce the rule once the cutoff point was met. The problem is enough miners showed green, the cutoff was hit, but then when an "incorrectly" ordered block came up sometime after the threshold was met, too many miners still accepted the block as VALID and so now we have a significant (majority?) hashpower running on blocks which are "supposed" to be invalid.

Of course consensus is, by definition, whatever the majority is doing. So now it's a battle of dev's trying to cajole enough hashpower into enforcing the rule, or somehow backing off the change. With every new block mined on each head, there's more money at stake for which way the decision ultimately falls. For Bitcoin to survive, "There Can Be Only One".

It will be interesting to see how exactly this came about. Were there bugs in the trigger code which was supposed to activate the new consensus rule? Did ignoramus miners simply turn on the flag without actually priming the new consensus code for activation? Did miners mistakenly think they just needed to show "green" and then their blocks would be valid, but then they published a "green" block with the older ordering?

Two mining pools (Discus Fish and Antpool) were engaged in what's called "SPV Mining", where they assume a block broadcast by another miner is valid based solely upon the block header (essentially, just checking that the block was "difficult enough"), without verifying (or downloading!) the transactions in that block. This strategy lowers their orphan rate at the cost of the risk that they'd waste work by mining on an invalid chain. This low risk was increased by the BIP66 enforcement (activation happened once only 75% of the blocks had the version 3 "flag").

What ordering? BIP 66 isn't about ordering. It's about the validity non-DER signature encoding.

Hmm... :-( Somehow I had the words "canonical ordering" in my head while I was writing that earlier today. Of course as you say it's nothing to do with ordering. Very sorry I can't edit it now. Not quite sure how I botched that one so badly, thanks for pointing it out.

Not only were some mining pools SPV mining, but they were advertising compliance with BIP66 at the same time; if they had not been advertising compliance, then it is unlikely we'd have switched over to enforcing BIP66.

So, don't lie about your SPV status in your coinbase is one of the lessons here for a pool operator.

When I ran a (small but noticeable) pool, the bitcoin core team would occasionally reach out with warnings when an impending network change was coming that we didn't look compliant with; often we'd get two or three e-mails independently with plenty of warning.

So, I would say this is chinese mining pools not playing by the rules, either because they didn't care to listen, or didn't care to have engineers do anything about it, and now they are eating it a bit, (and of course we all have some minor knock-on effects for a little while).

For the sake of quoting Greg Maxwell, I'd like to pick a nit. Bitcoin has no rules other than those you enforce yourself, and all these pools did was enforce fewer rules of their fellow players, than we assume they do.[4]

Optimize would be less judgmental than cheat and speaks more precisely to the motivations[1]... we do need to be careful about this and make sure we're managing the incentives. Cheating is a moral judgement[2]

Bonus quote, unearthed while digging up the above:

While the identifiable parties in question here were Chinese; the first miners I saw doing this in the pastwere not Chinese-- it's not a Chinese specific issue; its a response to orphan rate. (China just has lots of hash power and poor connectivity).[3]

[1] https://www.reddit.com/r/Bitcoin/comments/3c305f/if_you_are_...

[2] https://www.reddit.com/r/BitcoinMarkets/comments/3c2jci/dail...

[3] https://www.reddit.com/r/Bitcoin/comments/3c305f/if_you_are_...

[4] One could interpret the block version increment as a policy advertisement, but you know what they say about assumptions.

It's all good to quote Gmax, but I didn't say anyone was cheating, so I feel a bit straw-manned here.

And incrementing your version number is precisely a policy advertisement, full stop. It is saying you are in compliance with v1/2/3 of the spec. If you want to lie about that to other miners, there may well be consequences.

There are coinbase flags for neither SPV status, nor honesty.

They didn't cheat though, which in the end however means that they hurt (mostly) themselves (and their clients) and not others.

So I think it doesn't really apply here.

>Of course consensus is, by definition, whatever the majority is doing. So now it's a battle of dev's trying to cajole enough hashpower into enforcing the rule, or somehow backing off the change. With every new block mined on each head, there's more money at stake for which way the decision ultimately falls. For Bitcoin to survive, "There Can Be Only One".

Consensus is what everyone is doing. That is why the terminology used here would be "split consensus". The purpose of the devs trying to get the hash power back into the correct blockchain is so consensus can be re-established.

Needs better terms.

For instance, "split consensus" is dangerously close to being self evident nonsense. I think "untenable distributed state" captures the picture there.

The title is a little sensationalist. While the majority of wallets do only have SPV security, and such wallets were vulnerable to double spending during the recent fork event. The fork is over, so the wallets are no more vulnerable now than they were before (this vulnerability is not new).

I hope this incident has convinced other miners that the ~1% profit boost from "SPV Mining" is not worth the fork risk that it enables, although apparently one of the pools (Discus Fish aka F2Pool) has already been warned against SPV Mining in the past. Let's hope that coins lost through this fork event is the sterner warning they needed.

Doesn't seem sensationalist to me. As you say, the majority of wallets were vulnerable. The only problem is that the warning is far too late to protect anyone.

I doubt any potential F2Pool miner loss is enough to persuade them to not do SPV mining. Just look at how many blocks they've mined overall and consider how much extra they've earned over time thanks to this 1% boost...

It's sensationalist to suggest this vulnerability is anything new. The warning is late, indeed... I'd suggest putting it in the Terms of Service of every SPV wallet, but then even fewer people would read it. Maybe a better approach would be a popup each time the wallet blindly trusts a new merkle root?

The wallets are still vulnerable to future forks of the same nature.

It is not a 1% profit boost. It is a ~1% revenue boost, and mining is a very thin profit margin business. 1% revenue can be 10-20% profit. Furthermore, that gives an edge over the competition which doesn't use the patch which may be enough to shut them out.

This seems to be further confirmation that the ONLY safe way to use bitcoin is to use a full blockchain wallet such as Bitcoin Core. Any software which attempts to 'fake' it by not keeping the full blockchain has issues, exactly like this one being announced today.

This is a serious problem for Bitcoin, because right now it takes 2-3 days to download and verify the full block chain, and is fast approaching 10^H^H40 gigabytes in size.

There's going to need to be some sort of mitigation for this at some point but I don't see how that's possible.

It's 36GiB

If it was really that unsafe to use a SPV wallet, the announcement would have said so.

It did say that pools need to run a full node to not lose money, but the 36GiB burden should be a lot less of a problem for them.

The unpruned blockchain is about 30GB. Of course, the slowest way to download it is through the P2P network using a standard client. There are torrents which let you bootstrap the full blockchain up to some recent date, then let the P2P client continue from that point, so it doesn't have to take "days" to download, you can pull most of that 30GB at full line rate.

But more importantly, even to run as a "full node" all you need is the UTXO (unspent transactions) and you can throw away the rest. Bitcoin Core can be flagged to prune the blockchain as you go. Pruning just discards transactions after they are spent, keeping track of just enough headers to still provide "full node" validation strength. Basically all you need to know is what available "outputs" there are, aka the unspent outputs, because that is the universe of "coins" which can be used as inputs in new transactions.

I think the only thing that may be missing from all this is a way to download a proven pre-pruned blockchain to get started. That is possible, and it would bring startup costs of a full node down to about 1GB.

The trickier challenge is if you have a historical wallet and you want to download the full transaction history of the wallet, without any meta-data you are left to downloading the whole blockchain in order to discover them. But you would still be able to get your current wallet balance even with pruning.

https://www.reddit.com/r/Bitcoin/comments/33qv3a/pruning_sup...

Correct me if I'm wrong, but how would having just access to the UTXO helped here?

This fork could have still been prevented even if only UTXO were stored.

The problem behind the fork here is that 950 out of the last 1000 blocks had the block version set to 3, but didn't follow through with their promise. By setting the block version to 3, they were voting to transition to BIP66, meaning that once 950 of the last 1000 blocks vote for version 3, then no more nonstandard DER signatures will be allowed in blocks.

Even if you prune spent transactions (so only keep UTXO) you still keep the block version, so you can still measure if 950/1000 blocks vote for version 3. And you are still able to verify that new blocks don't have nonstandard DER signatures.

" As of Bitcoin Core version 0.10.0 or later, this torrent is slower than a direct download using the bitcoin P2P protocol & client. This torrent is now only maintained for research and development purposes as a secondary option. Please use Bitcoin Core directly to download the blockchain."

https://bitcointalk.org/index.php?topic=145386.0

Ever since headers-first sync was merged (https://github.com/bitcoin/bitcoin/pull/4468), the entire blockchain download & verification process can take as little as 3-4 hours.

In regards to the size of the blockchain, you might be interested in the recently merged Autoprune functionality (https://github.com/bitcoin/bitcoin/pull/5863), which deletes block and undo files to keep the total space used by those files below a certain threshold.

Last time I looked you could only use autoprune if you completely disabled the wallet functionality, so it was useless to anyone that wanted to use Bitcoin Core as a wallet (and it probably couldn't act as a backend to any of the lightweight wallets either). I think it's basically only of use to miners in its current state.

Wallet functionality has been re-enabled to work with autoprune, but it won't make it into the 0.11 release. https://github.com/bitcoin/bitcoin/pull/6057

If the bug I reported a couple of years ago hadn't been fixed, then even running a full node wouldn't have been safe. Basically, some older versions of Bitcoin Core didn't validate signatures in blocks older than 24 hours in order to speed up the initial blockchain sync, so they could be tricked into accepting an invalid chain where someone spent bitcoins that weren't theirs if it was far enough back. This wasn't considered a major issue at the time because you'd have to do a 51% attack for 24 hours to exploit it. If a large proportion of mining nodes are no longer validating blocks, that assumption is no longer true - you'd just need to mine one block, plus enough to push the non-validating nodes over the 51% line if they weren't already, and wait. (Oh, and I think Bitcoin Core may still skip full validation of older blocks during initial sync even with the fix.)

The idea that all these miners aren't validating blocks before building on top of them now is scary. Everything's designed around the assumption that they will unless they're trying to actively attack Bitcoin.

These are valid blocks under the old consensus rule. AFAIK, there is no mining of fully invalid blocks going on here.

That's probably because no-one's produced a fully invalid block with sufficient proof of work, but supposedly the pools involved would quite happily have built on top of one because they're skipping block validation almost entirely.

BIP 66 changed the definition of validity. These blocks are fully invalid.

No, any block building on a invalid block is automatically invalid.

There is an interesting graph on how the block chain size has progressed over the years since Jan 2009 [1].

[1] https://blockchain.info/charts/blocks-size?timespan=all&show...

> This seems to be further confirmation that the ONLY safe way to use bitcoin is to use a full blockchain wallet such as Bitcoin Core

This statement is false. Not all lightweight wallets (wallets that function without needing to download the blockchain) are SPV wallets. Some lightweight wallets use blockchain APIs (and therefore are immune to this problem)

Downloading the full blockhain is not the only way to secure bitcoin.

> blockchain APIs

What doest that even mean?

You mean blockchain.info API? Well Blockchain.info is not immune to this problem, they were running an old bitcoin core version.

It seems like there should be cheaper ways to detect and mitigate these kind of fork event (than doing full block chain verification). It shouldn't be too hard 'just' to detect that there are multiple chains being actively extended, and mitigate based on this fact (e.g. big warning to user, disallow transactions temporarily, or whatever)..

Also, Electrum wallet and friends don't do proof of work validation, they merely check which is the longest chain.

What do you mean by "don't do proof of work validation".

They don't verify the has in the header is the hash of the previous block's header? They don't that the hash is sufficiently small? They don't calculate the difficulty correctly?

> They don't calculate the difficulty correctly?

Yes. It's possible to create a long chain at low difficulty and the original bitcoin client protects against that by computing the total "work" that has gone in a chain. However, Electrum just assumes that the longest chain will be the one which has most "work" in it which is wrong.

That would be a pretty lame bug! It's the difference between "chainLength++;" and "chainLength+=pow;"

I don't think so. These wallets implement Simplified Payment Verification as described by Satoshi in section 8 of the original bitcoin paper[1]. They do validate the block headers, which contain the proof of work.

[1] https://bitcoin.org/bitcoin.pdf

The issue is not with SPV itself but with Electrum's implementation (by "and friends" I really meant Electrum forks, not all SPV clients).

Electrum does validate block headers but it doesn't handle re-orgs correctly. It just assumes the longest chain has the most PoW in it which is wrong and makes it vulnerable to attacks by malicious Electrum servers.

Block headers do not contain the cumulative work that has gone into the chain (only the difficulty of the current block). A chain's total work must be calculated and stored locally by clients.

https://github.com/spesmilo/electrum/blob/master/lib/blockch...

Surely they verify the PoW, otherwise it would be trivial to create a long chain and attack them.

Indeed. I was also surprised to find this out given how popular Electrum is. The code is here: https://github.com/spesmilo/electrum/blob/master/lib/blockch... and https://github.com/spesmilo/electrum/blob/master/lib/blockch...

One could run a modified Electrum server that sends a long chain at low difficulty and double spend Electrum users.

I read that generating that chain would be really cheap and simple too -- you just run a non-pool miner disconnected from the Internet and with an accelerated clock.

To clarify, the issue isn't SPV mining - the issue is not validating your mined blocks with the latest version when submitting. Even if the pools were SPV mining, their invalid chains would only be 1 block deep as the core client will reject any further blocks.

Reply on news.ycombinator.com