It's Time For a Hard Bitcoin Fork
hackingdistributed.com
hackingdistributed.com
It seems like this is a pretty big flaw in how Bitcoin is designed, as its security relies on miners remaining independent.
Really the more important point to note is that pooling for variance reduction has absolutely nothing to do with delegating control. Running a outbound only bitcoin full node, past initial syncup uses less than 20kbit/sec bandwith and a fraction of a percent of cpu... it's not costly to do, purposefully so.
It's perfectly possible to individually run your own consensus decisions but agree with others to, in a provable way, pool your payments. This is what P2Pool does.
Unfortunately many Bitcoin miners don't have a rigorous mathematical understanding of how mining works— they erroneously believe it to be a race where the fastest wins disproportionally— something entirely untrue (absent some proposed attacks which are not happening in practice)... just keeping yourself from getting scammed by the many scammy hardware companies is basically a full time job itself. Then you have various technically unsophisticated Bitcoin pundits claiming that hashpower consolidations in pools isn't something to worry about... not a great mix.
Fortunately, the reasons for the current behavior are mostly inertia— if P2Pool had been invented first the symmetry would have broken differently. It's still possible that there might be a massive swing (say if GHash.io decides to steal a bunch of coins from their miners and makes a runner).
I'd be interested to learn how mining delegation could (in theory) be avoided. It seems to me that as long as you have a proof-of-work designed around something that can be solved with distributed computing power, there may always be incentives for delegation and cooperation (especially to reduce variance). I'm genuinely curious to hear what other approaches can be used to "break delegation."
The intuition (which is described in the original article, by the way) is that in this puzzle, whoever actually finds the solution can take the entire reward for themselves.
We've expanded this idea into a rigorous research paper that's currently undergoing peer review, but we may release a preprint soon.
This destroys the ability of pool members to trust that their nonces, where valid, will actually be used to claim a reward for the pool.
I'm not familiar with the work, but I assume there are validated zero knowledge proof protocols that prove you know a partial hash inversion of a particular quality, like the Pinocchio one referred to.
Edit: with that said, I don't agree that eliminating mining pools is a good thing; that just rewards those who can enforce cooperation by some other means (say, having the capital to personally run the farm), which has a far more unequal distribution than mining pools.
In order for Bitcoin to work, in the sense that you don't have to trust anyone for it to be fair, then no group can ever have 51% of the hashing power of the network. It has now been proven that a mining pool can reach that threshold. It has NOT been proven that it is practically possible to buy that much hashing power. We should get rid of mining pools because they are the current cause of this crisis. We can cross the other bridge when we come to it (which it seems unlikely we ever will).
The non-outsourceable puzzle would have to be adopted at the same time as another change, which would allow solo-miners to enjoy the low variance they get in a pool.
Essentially you would need to allow miners to choose a lower difficulty, for proportionally lower block reward. This opens up DoS concerns if it can go arbitrarily low, and might generally require more bandwidth. This would essentially be Bitcoin internalizing p2pool.
Why should people [miners] be able to reduce variance at all? That's not a necessary feature of the system, and it doesn't seem like a goal worth pursuing. Security is paramount; mining variance is "first-world-problems".
>say if GHash.io decides to steal a bunch of coins from their miners
It seems far more likely that GHash would try to be sneaky about theft, rather than overt. This would allow them to keep getting away with it (that "long term revenue stream").
I know this sounds sort of unlikely, but consider e.g. GHash functionality to identify and subtly interfere with the operation of automated tools (e.g. ZeroCoin) that make lots of bitcoin transactions. The less likely that human eyes will see a transaction, the easier it will probably be to subvert.
100% agree. If people who can't tolerate variance don't mind, then those who can tolerate it will.
This was just before releasing some research that he thought would cause a confidence collapse. (That is, his prediction was almost self-consciously attempting market-manipulation.) In fact, the paper just formalized some concerns discussed in the mining community for years.
So then, rather than collapsing, Bitcoin went on an epic rally, and hasn't been below $339 since the same week of that doomsaying prediction.
There is certainly danger in one entity controlling 51% of the hashing power. But everyone's known this risk, as one of the design assumptions of the system, from the beginning... and also seen the tipping point approach/recede/approach repeatedly. And also, the "Bitcoin lunatic fringe", who this author mocks, has so far been right about the pool(s) attaining such power refraining from taking destructive (and self-bankrupting) next steps.
So: focused concern, yes. But @el33th4xor-style panic, no.
Further, any 'hard fork' (or forks) that were to remedy pool issues, using the "well-known" techniques referenced, would almost certainly retain some continuity with prior key balances. That is: imagine the most destructive transition possible. A total civil war between mining pools. Irreconcilable dissension in the core team (or offshoots thereof). Collapse of the Bitcoin price to values of 1-2 years ago. Still, at the end of that process, there are one or more "offshoot" chains, adopting the Bitcoin history as their own, patched and stronger than before, with pre-crisis Bitcoin balances intact.
(That is: a 51% cartel may not be actually "good" news... but it is survivable and perhaps even necessary.)
So if you like to try to trade in and out of predicted market panics, like @el33th4xor, maybe there are some trading plays here. But if you just like cryptocurrency for the long haul, keep your Bitcoin private keys (end eyes) safe & dry, and trust evolution. There are enough smart, well-funded, and relatively cool heads involved that @el33th4xor's predictions are just a car alarm going off in the night, whether the car is actually at risk or not.
>In fact, the paper just formalized some concerns discussed in the mining community for years.
This is false. Discussed here: http://hackingdistributed.com/2013/11/09/no-you-dint/
>the "Bitcoin lunatic fringe" this author mocks has been right about the pool(s) having such power refraining from destructive (and self-bankrupting) next steps.
No. The Bitcoin lunatic fringe was adamant that no pool would willingly cross the 50% boundary.
That just happened. Models and reasoning based on "no rational miner would do X" are clearly flawed, partly because the miners may not be rational, or partly because they are rational within a time-frame not modeled. In any case, people who reasoned like you have now been shown conclusively to have the model wrong.
This is an opportunity to fix the protocol, not shill for the price, and certainly not to engage in ad hominems.
Cheers.
I've addressed your continued "no-you-dint" willful-blindness about earlier analysis elsewhere... including on your own blog at (http://hackingdistributed.com/2013/11/14/response-to-feedbac...). You failed to discover (and thus footnote) prior community work, from years earlier, that did everything except for your more-rigorous boundary formalizations. So again, nice write-up, but exaggerated novelty. The interested can follow the links and decide for themselves.
I'm sure someone said no pool would ever even try to get 51%. Others simply said a pool in such a position wouldn't self-destruct the entire ecosystem, against their own interests. (Instead, they behave like the 'stationary bandit' of Mancur Olson's political-economy. Not ideal, and not what Bitcoin intended, and worthy of attempted-fixes... but also not an instant and unsurvivable crisis.) It's this latter prediction, of stability even in the presence of explicit (or secret) 51% cartels, that is still, so far, outperforming your own. For now they have the same claim to "I told you so!" as you do.
Perhaps you've not read our final paper. It doesn't just footnote, but actually cites the prior discussion.
And anyone who reads the previous discussion can see that our paper:
* shows a more extensive attack than the one described there, one that works,
* performs a full analysis of the revenue to be obtained from that attack, and characterizes that revenue as a function of attacking pool size and attacking pool's ability to control information flow in the network,
* shows that Bitcoin is not incentive-compatible,
* shows that, even under the best of circumstances (i.e. the attacker has terrible network connectivity, no Sybils, no control over information propagation and loses to the honest miners every single time), defending against the attacker requires at least 2/3rds of the network to be honest.
Perhaps the biggest giveaway that we did something differently is that THE BITCOIN TALK FORUMS CONCLUDED THAT THEIR ATTACK WOULD NOT WORK, WHEREAS WE SHOWED THAT OURS WOULD.
You're making things up when you imply that we're claiming that 51% is an "unsurvivable crisis." To the contrary, the article very clearly says that the Bitcoin economy remains unaffected, and that the Bitcoin price is also unaffected.
We have been trying to improve the Bitcoin system since day 1. I realize that you're part of the original brigade, and that also explains your ad hominems here. I urge you to elevate the discussion.
Every key aspect of selfish strategy is described there, from manipulating 'gamma' via network-tricks, to releasing the minimum number of 'secret' blocks, after each external-block, to maximize the cartel's expected return. ByteCoin's simulations show advantages, and breakeven thresholds with regard to 'override success' ('gamma'), very similar to your paper's calculations. That's why I credit your paper for rigorously describing the situation, under your specific assumptions, but not with the discovery of a previously-unknown less-than-51% attack.
Also, your final paper is simply lying when it says the thread "does not suggest a solution to the problem". It's almost as if your disdain of these 'fringe' Bitcoin fanatics has blinded you to the actual words of the thread.
Two commenters in the December 2010 thread (btchris and RHorning) suggest that preferencing accurate-seeming timestamps can disadvantage cartel-delayed blocks. That countermeasure is likely stronger than your paper's proposed random-choice-between-ties. (Randomization, by pushing gamma to 1/2, could make things worse if, on the real network, the effective gamma for late-releasers was already closer to 0. Preferring realistic timestamps, meanwhile, almost always helps 'honest' blocks, which don't have to guess a future time when they'll be released.)
Note that the last bullet of supposed novelty in your paper – "defending against the attacker requires at least 2/3rds of the network to be honest" – is the exact same best-case threshold as reported by ByteCoin in thread message #36, 2010-12-14. He states: "a cartel with no preferential network access can be profitable with 33% of the generating power"[5]. Same result, 3 years earlier. How can you allege ByteCoin was simulating some other strategy? Wouldn't the slightest difference in block-release-rules result in a different best-case threshold?
Finally, the Bitcoin Talk forums hadn't "CONCLUDED" anything. They're not a deliberative body. Some people were convinced, others weren't. The relevant actors – mining insiders – knew what they needed to know, to either try the attack, or detect it in orphan rates and weird timestamps... and to try countermeasures based on disadvantaging cartel blocks if ever necessary. Meni Rosenfeld also referred back to the matter as a known concern, in an answer on the Bitcoin StackExchange, in October 2011 [6]. So he knew it was an issue, and lots of people trust him about mining matters.
There's no "brigade" out to trash you led by some "failed academic" "Singaporean" "ringleader". Your critics are not the heads of some unified hydra, that you can disregard altogether as the "Bitcoin lunatic fringe" based on a few quotes from particular yahoos. You've made specific claims of novelty, or doom, that were either never true, or disproven by later events. These will be pointed out when you claim to enjoy a "we told you so" record of authoritative insights.
[1] http://arxiv.org/pdf/1311.0243v5.pdf
[2] http://www.cs.cornell.edu/~ie53/publications/btcProcArXiv.pd...
[3] http://fc14.ifca.ai/papers/fc14_submission_82.pdf
[4] https://bitcointalk.org/index.php?topic=2227.0;all
[5] https://bitcointalk.org/index.php?topic=2227.msg30138#msg301...
[6] http://bitcoin.stackexchange.com/questions/1475/can-someone-...
Yeah, the 'lunatic fringe' like Sam Altman.
Simply calling someone 'panic-prone' isn't in itself ad hominem. If the argument were that because the author is panic-prone he cannot possibly be right, then it would be ad hominem. But if there isn't an explicit causality implied (i.e. being panic-prone makes you wrong), it's more just name-calling. (name-calling != ad hominem)
There may be other fallacies that apply here, but there is actually an argument backing up the commenter's claim that the author is being too panicky, and that this is less of an issue than it is being made out to be.
He took a gamble: that the bold tweeted prediction would increase attention for his paper (which it did), and then come true, improving his credibility about such matters (which it didn't).
He's now saying "I told you so", literally, in a gambit for more credibility – but eliding mention of his prior bad predictions. (In addition to the "good time to sell" prediction, I would point out that (a) orphan rates since his paper was published have not shown the predicted wide self-interested adoption of "selfish mining"; (b) the pool that's achieved 51% does not appear to have used "selfish mining" to get there - just the same economies-of-scale and small-miner-superstitions that have been known threats since the beginning of pooled-mining. In other words, even if this is a disaster, it's not the same one, by the same path, as he predicted – but an older potential doom, prophesied by others long before his work.)
When facing such a claim of earned authority, it's entirely appropriate, and not at all ad hominem, to highlight the full record. To "shout from the rooftops" exactly the same early calls he himself would be bragging about – if he'd been right.
That's seriously rich coming from the dude who just wrote
> The main ringleader of this brigade was a failed academic from Singapore, someone who had a superficial knowledge of game theory and sufficient familiarity with Latex to create the look & feel of research papers, but someone whose own academic work never went beyond repackaging well-known results in game theory.
This principle pops the utopian fantasy of bitcoin that some cling to, but only a different approach to digital currencies will get you around the issue.
Money & power always consolidates down to the hands of a few that will possess dramatic influence compared to the rest of the market participants. See: JP Morgan, pre-Fed.
But should the rich have control in proportion to their richness or out of proportion to it? That's the key issue that I see behind this article (and the previous selfish mining work).
Edit: And more generally, for the same reason, pools of this sort should cease to exist.
How is that an accurate description of IPv4 at all? IPv6 has made monumental progress[0] in a relatively short time (yes, for what we're talking about, it's only been a short amount of time).
in that respect, wouldn't 51% be only marginally different to 49%. Both would be a bit of a concern, but neither would be the "position to exercise complete control over which transactions appear on the blockchain" that this article refers to.
Is there some mechanism I'm missing that makes 51% be vastly more powerful than 49%?
An attacker that controls more than 50% of the network's computing power can, for the time that he is in control, exclude and modify the ordering of transactions. This allows him to:
Reverse transactions that he sends while he's in control. This has the potential to double-spend transactions that previously had already been seen in the block chain.
Prevent some or all transactions from gaining any confirmations
Prevent some or all other miners from mining any valid blocks
Reverse other people's transactions
Prevent transactions from being sent at all (they'll show as 0/unconfirmed)
Change the number of coins generated per block
Create coins out of thin air
Send coins that never belonged to himAlso the "Some or all" doesn't seem very specific. What is the factor that makes some become all?
It would be more profitable to do something like extending the current blockchain but charge 1% transaction fees.
There is a hardcoded list of checkpoint blocks in the Bitcoin client. Any new chain starting before the last checkpoint will be rejected.
https://bitcoin.stackexchange.com/questions/1061/can-a-51-at... https://bitcoin.stackexchange.com/questions/3114/which-block...
If they lose their 50% control then they generate invalid blocks. Particularly the part regarding the number of coins generated per block. That one will obviously be wrong.
Essentially, as time progresses, with 49% you lose out, with 51%, you keep winning.
How detectable would such an action be? Wouldn't other systems be able look at the block and say "it's verified, but it don't look right to me"
At 51% you have more hashing power than the rest of the network combined, so you can start mining blocks on your own and create your own chain with the knowledge that eventually your chain will be longer than the 49% chain everyone else is working on. When that happens, the 49% will abandon their chain and start working on yours.
No. The 'winning chain' is determined by total difficulty, not block count.
In the previous crisis it was all based on trust ("it was just a bad player, trust will return to the market"). Now we've a doomsday scenario and what seems a serious flaw in Bitcoin.
Feels like a chapter out of The Foundation books.
Relatively speaking, this is a pretty small dip, though.
If you're perfectly fine trusting a GHASH to act honestly, you should be perfectly fine trusting them or some other entity to run a centralized, non-distributed currency.
If switching over to BTC means "trusting GHash.io"... then nothing has changed.
https://en.bitcoin.it/wiki/Weaknesses#Attacker_has_a_lot_of_... * Prevent some or all other miners from mining any valid blocks
All currencies suffer through booms and busts, even precious Gold and Silver.
However, BTC is unique in that wielding 51% hashing power grants you powers beyond what has ever been seen before in a currency. A properly wielded 51% can absolutely destroy an entire coin system (RIP Feathercoin).
Do NOT take the 51% attack lightly. This could mean the absolute ruin of BTC in its entirety.
There's more than a few entities with an incentive to seeing a decentralized network capable of replacing established methods of conducting financial transactions go away.
Let's say you implement a restriction like "5 blocks in a row max for a given pool". GHash can split into GhashA and GhashB, and keep going.
The only way you can truly prevent pools from taking over is create a system of authentication that'll destroy anonymity.
i.e: The pool would notice that certain participants contributions are conflicting with other discoveries, and ban such participants?
How? Or, you ban me, I sign up again under a different alias.
This kills the GHashA.
However, any miner in the long run would prefer to join a mining pool that does not require trusting some pool operator over one that does, all other things being equal.
Yes, the current situation is dangerous for the health of bitcoin, but I don't see any solution besides waiting for distributed, trustless pool technology to catch up in terms of usability with the centralized pools.
This problem isn't going to be solved by a hard fork, as any "fixes" done this way are untested, incomplete, and risky.
[1] I am not a bitcoin mining expert
Bandwidth is certainly a problem though... are there any good numbers on the bandwidth difference?
1. Assume that selfish mining doesn't work.
2. Because selfish mining doesn't work there will be fair weather miners who will only mine on whichever chain is furthest ahead, defaulting to the public chain in the case of a tie.
3. Since the selfish mining pool won't be ahead all the time nobody will mine for it.
4. Therefore selfish mining doesn't work.
It's not what I'd term a strong rebuttal.
It's certainly true that selfish mining strategy A which gave a 5% increase in results would lose out to another selfish mining strategy B which gave 10% increases in results. However that's not an argument that strategy A doesn't break the system, merely that the optimal selfish strategy breaks the system at least badly as strategy A.
Now it is possible that there isn't a stable strategy to use. If we supposed for the purposes of argument that the fair weather strategy was more profitable than the selfish strategy described in the first article you posted, then it would seem that the optimal strategy would oscillate. As more people participate in a selfish pool consistently the more profitable it is to be a fair weather miner. However, supposing that leads to a dissolution of the selfish pool all those fair weather miners turn honest. Against honest miners, however, the selfish strategy is proven to be more profitable. And so you'd see an oscillation where people constantly shift between being honest, selfish and fair weather.
Of course, if the long run gains keeping up on this strategy treadmill is less than simply sticking to the selfish strategy through thick and thin, then perhaps the fair weather strategy isn't better in reality.
Explained here: http://hackingdistributed.com/2013/11/08/fairweather-mining/
Also it's too bad that BTC is blinding everyone to a variety of other crypto-currencies that have improved features.
There will not be a way of "patching" the issue. The only thing that can be done is to set up a proper emergency handling procedure in the event a 51% attack is conducted, which involves directing as many clients as possible to work on a new fork.
Great accompaniment is from Peter Todd (Coinkite adviser, respected dev) who announced this AM he is selling 50% of his holdings in bitcoin until this is resolved
http://www.reddit.com/r/Bitcoin/comments/281ftd/why_i_just_s...
"as intended" reminds me of this koan from the codeless code[1]. [1]http://thecodelesscode.com/case/135
Anyway, stay tuned and don't miss the next iteration of 'We are all doomed!!1' by the two muppet academics. To be published shortly after a solution gets deployed. Or earlier.
http://www.coindesk.com/bitfury-pulls-power-ghash-community-... https://blockchain.info/pools
Is this official? Seems somewhat surreal...
Not really invested in this but at the very least I'd expect some posts along the lines of "Pool X is fast approaching 50%...BTC in danger". Not "51%...game over".
As someone who works in the information security industry, I can confirm from personal experience that a very disproportionate amount of organized and semi-organized cybercrime comes from Ukraine, Russia, and neighboring countries. I, however, do not have any sources for you at thistime.
Every major participant in the Bitcoin network, including GHash, has a vested interest in maintaining the network's integrity. Not only that: given Bitcoin's increased mainstream acceptance, it's in the best interest of every major participant to maintain a good reputation.
Thankfully in at least two other poker companies (hint: the largest one and its sister company) it was made impossible to see someone's hole cards before the hand is over, which is what allowed the cheaters at Ultimate Bet to perpetrate their con.
It's my pretty firm belief that when it seems like an entity is throwing away a pretty obvious economic self-interest, there's probably just a misunderstanding of where the economic self interest lies in the parties involved.
There's pretty much an infinite number of reasons people choose short-term profit with apparent long-term opportunity cost above what apparently stable long-term profit streams. Betting that an actor with the power to do so would never do so usually is equivalent to creating a greater incentive for them to do so.
But the meta-reason is: if someone thinks they can get away with it, why wouldn't they do it? With that presumption you could even argue it's the rational choice. If you do it subtly, there's plenty of room for doubt. Furthermore, there are enough people who're invested for ideological reasons that, in the absence of strong evidence, all most people will hear is a lot of he-said, she-said.
More to the point, factor in any monetary investment in the scheme -- the prospect of collapse should a critical mass reach the same conclusion, for instance -- and the people who've invested have an incentive to stay the course. That incentive to maintain a good reputation cuts both ways. Reputation is a matter of popular perception.
It's in Bitcoin community's, and GHash's, economic interest that no miner exceeds 50%. That's all you need to know to know that this is just another exasperated hand-wringer proclaiming the premature death of BitCoin.
BitCoin, RIP 2008 - 2009, 2010, 2011, 2012, 2013, 2014, ?
Are Bitcoin community's interests and individual miners' interests aligned? I understand why the bitcoin community wouldn't want a mining pool with more than 50% of the share, but why should "selfish" individual miners care?
For many people in the Bitcoin ecosystem, making the technology successful is about more than personal enrichment. A lot of folks see Bitcoin specifically (because of its present success) holding the potential for big changes in a shorter period of time.
The people I've spoken with don't mince words: in general serious miners compete against others to make money, not societal change.
Fine, but in their ignorance and greed, they're going to lose their ability to make money.
What happens when people lose faith in the system? The 2008 financial crisis, the great depression, etc....
Nobody wins if the entire system breaks down.
In other words, assuming that they don't know anything you don't. Considering we're talking about an entity that's managed to become a 51%er, that assumption sounds downright Pollyannaish to me. The same line of reasoning also supposedly implies that nobody should want to get even close to this point. Counterfactually, as it turns out.
That said, I can see the attraction of that line of reasoning, too. When you've got a tiger by the tail, it probably is best not to contemplate too carefully what's at the other end.
They can't even sell their future winnings as they don't have them yet, so they are really long!
Normally I'm not so inclined toward conspiracist thinking, but considering all the different capers that have put Bitcoin in the news over the past couple years, in this particular case I'm inclined to make an exception.
I don't really know what GHash knows, but I can tell you that GHash's hashrate has dipped well below 50% only a day later, as it did when this happened before.
The threat here is that a single group has the capability to produce 51% of the hashes, possibly reliably (we can't know whether scaling back was deliberate or happenstance). It doesn't matter whether they are persistently using that capability.
It's strange, this situation where in there are all these things that would be seriously problematic were they to happen, unless of course they actually happen, in which case they're not actually a problem at all.
You mean, "denominated in energy divided by global hashing power (of all miners collectively)".
But GHash does exceed 50%, right now.
We've long known this is a weakness of the system, but most early adopters assumed that our strong decentralized culture would prevail.
Clearly, we were wrong. So we either have to find a technological solution ASAP, or we may as well just let GHash operate servers -- it'd be much cheaper for them and easier on the environment, and the end result is the same.
Centralization.
They could have short-term profit in mind, or else crippling bitcoin itself. There is plenty of room for them to have interests that are out of line with the rest of the bitcoin community.
As I've been closely monitoring the ups and downs of Bitcoin price, what always stops Bitcoin free fall is the feeling that you're gonna miss a huge profit, it's not the reason, and dips become smaller, and smaller as speculators who missed the train last time don't wanna risk too much next time. When you have a bunch of greedy people with money, again, it's hard for this thing to die unless more profitable alternatives appear on the horizon. The one unique thing here is China and their limited options for high tech speculation. If China really kills Bitcoin over there, it will mark the final death of Bitcoin.
Miners join in pools to mine bitcoins to even out their earnings. It's a way to diversify their risk.
Unfortunately, bigger pools let you diversify the best, and this is currently undermining the core tenet of bitcoin, which is to avoid having any one person with central control over the network.
The linked article is suggesting modifying the core software for bitcoin in order to discourage this kind of centralization. Other people, however, think the core software is fine, and that the solution is to instead improve "decentralized mining pool" technology to get rid of the problem.
According to the article the big draw is that GHash don't have a fee.