Bitcoin's ASICBOOST Problem Explained [pdf]
rubin.io
rubin.io
It is possible that this is part of the reason some miners are blocking SegWit but there is more to it. Core (the main Bitcoin development team) promised to provide a block size scaling solution and then reneged. This pissed a lot of people off and led to the blocking of SegWit. I don't see ASCIBOOST as a real problem for Bitcoin.
Also, their 20% figure is unclear. The more collisions you can find, the more hash per second you should be able to gain.
But the chance of finding three blocks with the necessary 4 byte collision is much more difficult. It's the same problem as finding 3 people with the same birthday. You need 87 people to have a 50/50 chance of three people having the same birthday vs. 23 for two people who share a birthday. That's nearly a 4x increase in required computation for a small increase in effective hash rate.
It often incentivizes miners to reorder or drop transactions until they hash to a specific value. (A certain mining pool is publishing a significant number of empty blocks.) Miners exist to verify transactions, and Bitcoin was designed with transaction fees in order to incentivize miners to include transactions. A competing force pushing miners away from including transactions that can be trivially fixed is just a bug.
Imagine a mining "optimization" which involved miners making only empty blocks. That goes against why Bitcoin has mining in the first place, and would delay transactions eventually to a halt as more took advantage of the optimization. A trick that only often encourages miners to make empty blocks isn't categorically different, and it's something suboptimal about Bitcoin that can be fixed.
Without protocol upgrades that include commitments to other parts of the block (E.g. committed utxo, committed bloom filters, stxo, segwit, fraud proofs, etc) you can simply compute 4096 variations of left- side of the tree by altering the coinbase transaction and 4096 versions of the right side of the tree by just substituting any transaction on the right side with alternatives. Then combine the two in to the ~2^24-ish needed for a collision with only a single operation per try which is independent of the size of the block.
The lack of a commitment gets you another effective sqrt speedup and makes all the other costs negligible.
Now, as far as permutation goes, if you want to permute instead of replace you need only permute 7 transaction on the right side to get the variations you need. Virtually no blocks with more than a dozen transactions don't have at least 7 on the right side with no dependencies at all.
This is still a non-issue as a solid GPU farm can crunch the SegWit version trivially and there are ASICBOOST compatible optimizations for SegWit that have not been publicly discussed.
One should look at the R&D on ZCash's equihash to see some of the work that has gone into collision finding.
Transaction fees are going up and the block reward is dropping so long term this won't be a problem. Miners mining empty blocks will not make money any more.
And it's not true that a "significant" number of blocks are empty. Only ~2% of them are.
Furthermore the reason some blocks are empty is not due to Asicboost. It's because after a new block is found, there is a small delay until a miner can assemble the new set of txs, and during this period of time the hardware has to mine on something (empty block).
Above all, it would be in an Asicboost miner's interest to include as many tx as possible in order to gain a ~10% increase in revenues from tx fees!
So, no, I disagree that Asicboost is a "problem". It's not. It looks like Gregory Maxwell either misunderstands/misanalyzed the "risks" or is simply dramatizing the situation for his own personal (political) reasons which would be part of his plan (Blockstream's plan?) to attack and discredit large-scale miners in the overall debate of Segwit vs. Bitcoin Unlimited.
Asicboost could hypothetically become a problem only if (a) mining profit margins became very thin caused for example by a multi years period of BTC losing value or remaining flat, (b) if somehow the 3 Asicboost patents (Bitmain's, KnC's, Hanke's) were never licensed to anyone else, (c) if the patents owners actually did evil things by abusing their patent rights, and (d) while all this chaos is happening everyone else sits on their hands instead of ignoring the patent and adding the optimization to their ASICs for the benefit of Bitcoin's health or survival if it came to that...
It's very unlikely that (a) (b) (c) (d) all occur at the same time.
4 patents (applications) from 4 different holders! This provides enough competing interests that it is even less likely that asicboost will ever turn out to be a "problem".
If we have two competing successors (S_1 and S_2) to a Block B, an ASICBOOST miner can mine both simultaneously by creating Chunk 1's with different parent blocks.
I call it "The Oliver Twist Attack".
Increasing the Oprhan rate is incredibly costly to affected miners, this has the effect of making it also increase the hash rate of the attacker.
This is not true at all and highly deceptive. They originally were supportive of doubling the transaction count via an increase to 2MB blocks and then realized they could accomplish this without a hard fork via SegWit.
SegWit does in fact allow for 1.7x the number of single-signature transactions per block and 4x for multi-signature transactions.
Core absolutely could have handled the communications around this better. But to say they reneged on providing a scaling solution is wrong.
It's also important to keep in mind that what matters is not the block size but the number of transactions that can fit into a block. Block size doubling and cutting transaction sizes in half each double the throughput.
Regardless of your position in the debate, to say that many well-known Core developers reneged is absolutely defensible, given than they signed this: https://medium.com/@bitcoinroundtable/bitcoin-roundtable-con...
The agreement between all these parties was that they'd all support a SegWit soft-fork if Core developers also provided code and support for a ~2mb hard fork.
You can only cut transaction sizes in half once, but you can always double the block size again. Let's just double the block size?
Regardless, Core signed an agreement with a majority of miners that they would implement a block size increase, not an effective block size increase but a real one. They reneged on this promise and are instead telling everyone that they are going to do as they please because miners don't control BitCoin.
Not to add that a company called "Blockstream" have paid almost every developers of bitcoin core to push Segwit without a clear consensus from the community. Worse, this company actually support the huge censorship happening in bitcointalk and r/bitcoin. Blockstream received more than 75 millions from AXA, for obscure reasons.
The paper only focus on Segwit, without mentioning that Segwit has nothing to do with ASICBOOST. The community and a growing majority of miners is actually switching to bitcoin unlimited, a fork of bitcoin core.
Sorry for the little off topic comment but I love Bitcoin and I hate that a few dishonest people took control of it and try to destroy it.
Sigh
> 10 000 lines of code highly controversial because they require... A soft fork, and will change bitcoin in a fundamental ways
Most of those lines of code are tests. It would be nice for you to say why SegWit is controversial, beyond being proposed around the same time that fees got high.
Bitcoin has had soft forks before. This one will do great many great things for transaction throughput and privacy.
> "Blockstream" have paid almost every developers of bitcoin core to push Segwit without a clear consensus from the community
Blockstream employed 5(?) Core devs, some of them part-time.
> Worse, this company actually support the huge censorship happening in bitcointalk and r/bitcoin
Censorship is horrible, but do you have proof that Blockstream supports it?
> 75 millions from AXA
Yes, they received funds from a bank. But what matters about code is what the code does, not who paid for it. Bitcoin is open source, so you can inspect what's going on. If you have legitimate economic or technical objections to SegWit, I'd love to hear them. If you're spouting FUD, then go back to /r/btc.
Sigh
> Bitcoin has had soft forks before. This one will do great many great things for transaction throughput and privacy.
It's an inefficient hack. For example:
https://medium.com/the-publius-letters/segregated-witness-a-...
> Censorship is horrible, but do you have proof that Blockstream supports it?
All signs points towards it. Do the Blockstream people even speak out against the censorship on /r/bitcoin? No. Do they prefer to use channels censoring opposing opinions? Yes.
> If you're spouting FUD, then go back to /r/btc.
Exactly the narrative Core supporters push.
Sure. The creator of SegWit (Sipa) said in one of his first talks on the subject that if you could re-architect Bitcoin from scratch, then you would segregate the witness data. Nodes could potentially throw it away after witnesses were verified, to save disk space. The problem is that it would require a hard fork, which Core wants to avoid if at all possible.
SegWit hacks the coinbase and transaction structure in order to be implemented as a soft fork.
I'm not sure if it's inefficient or not, but it is a hack.
Does the fact that it's a hack mean that it's bad for Bitcoin?
That article talks about all sorts of developer troubles and losing coins without ever mentioning the testnet, which exists to allow you to lose worthless coins. If a company is pushing major updates to wallet software without conducting proper testing, then bad on them. Plenty of bitcoin has been lost to the current complexities of the transaction format. Remember those 100 BTC fee transactions?
The article's conclusion bemoans the plight of the miners without mentioning the actual users of Bitcoin, other than a hand-wave at "community consensus." I'm not convinced.
> Do the Blockstream people even speak out against the censorship on /r/bitcoin? No. Do they prefer to use channels censoring opposing opinions? Yes.
Ok, so let's assume the whole lot is rotten and loves censorship. Does that change the nature of the code that exists in the Bitcoin codebase? Censorship sucks, but Bitcoin is what really matters.
>> If you're spouting FUD, then go back to /r/btc.
>Exactly the narrative Core supporters push.
Anyone who tries the path of moderation and reason within /r/btc is downvoted into oblivion, resulting in de facto censorship of that sub.
Edit:
> Summary of /r/bitcoin, for those wondering.
You forgot the roller coaster price gifs :-)
Yes, gmax has, literally every time he is asked about it. His complaint is that r\btc as an organizational/policy difference is a worse place. That's by comparison. -- Which it is, since e.g. users like /u/pagex still haven't been banned after direct physical threats.
> Blockstream employed 5(?) Core devs, some of them part-time.
If you omit the arguable "blockstream have paid", the rest of the sentence is correct: almost every developer of bitcoin core pushes Segwit without clear consensus from the community.
They define "consensus" as a consensus between those who reside in (and who have not been ejected from) their tightly controlled forums.
> Censorship is horrible, but do you have proof that Blockstream supports it?
Adam Back and Greg Maxwell (CEO and CTO) regularly participate in the forums that support the censorship, i.e. /r/bitcoin, and rarely post on /r/btc (for a starting overview of the censorship, see here: https://medium.com/@johnblocke/a-brief-and-incomplete-histor... )
Is the CEO and CTO choosing to participate often and primarily in a heavily censored community not "support"?
-Keep bitcoin broken so fees go sky-high
-Push people towards their sketchy off-chain transactions
Bitcoin then becomes "bank coin", where large providers handle all the micro transactions and "settle up" with huge 1000+ btc chunks.
It's a scam and while I have my own reservations about bitcoin unlimited and making miners "too powerful" at least their plan is 100% on-chain.
Core dev's plan is suicide for bitcoin, major investors have been jumping ship to etherium driven by their insanity.
The Fork to a chain that does scale on its own seems inevitable and was fully expected by the original developer. It seems like an uphill battle for Core to convince people to NOT upgrade the network.
The "sketchy" off chain stuff is not sketchy either, it is just off chain. If you mean that they may be able to profit from it, I don't see why that is evil or even wrong, it is just their business plan.
If you do a tad bit of investigation, you will find that your fears are entirely unfounded.
Neither does Blockstream "employ" the developers in a bitcoin-development capacity. The number of people it employs to develop Bitcoin is something like. 1.5. That's out of hundreds of people.
The apparent consensus amongst the literally hundreds of individuals who have contributed to core should be demonstrative of the fact that a majority of technically-aware developers think SW is the safest and best way to increase on-chain capacity—this is a developer majority just by the numbers themselves.
Here is some very strong evidence for it: http://telegra.ph/Inside-the-Dragons-Den-Bitcoin-Cores-Troll...
Obviously, people could always say that there is no proof, because "supporting" is such a vague term. It is hard to come up with a mathematical proof. But I would say, it is already a proof that they participate mostly in /r/bitcoin and not in /r/btc. Reddit depends on participation. Therefore with their participation in censored forums in contrast to uncensored forums they at least indirectly support the censorship.
Just recently, someone in the ecosystem that I've known since the beginning of my time with Bitcoin came out in support of Blockstream and Segwit with a backing of other Canadian support.
https://medium.com/@francispouliot/canadian-bitcoin-economic...
I don't know what to make of it all. I spoke with him a bit and he seems to believe that those pushing against segwit are paid by ASICboosters in China because they stand to make $100,000,000/yearly from their (now not so secret) advantage.
In any case, I just wanted to add that I'm more confused than ever and things are very unclear.
If anyone is confused how the 30% advantage that ASICBOOST gives translates into this huge number, remember that the 30% boost applies to revenue, not profit.
The revenue of mining in the long run approaches the costs to mine (basically the cost of electricity), leaving extremely slim profit margins. Let's say the usual revenue from mining over some period of time is $1.02, and the cost is $1.00, giving a profit of 2 cents every time unit. A miner using ASICBOOST could have a revenue of $1.32 every time unit, giving a profit of 32 cents every time unit, which is at least an order of magnitude more profit than anyone else. This miner then has more resources to spend on buying hardware and scaling up.
(The above paragraph goes for any type of optimization and isn't necessarily nefarious. It can be a problem if one miner effectively keeps an optimization secret for too long, because they'll continue to grow in size and could get more than 50% of mining power, which is a specific point that causes huge problems with Bitcoin. But the real unique issues about ASICBOOST are about how it encourages empty blocks, and how it has been secretly incentivizing some groups to argue against any incompatible protocol change.)
Ignoring depreciation or hardware costs, the arithmetic is like this:
Pre-ASICBoost: Revenue: $1.02 Costs: $1.00 Profit: $0.02
Post-ASICBoost: Revenue: $1.02 Costs: $0.70 Profit: $0.32
The closer to costs that revenue for non-AB mining is, the closer to infinity the comparative profit margin is.
That is, AB doesn't increase hashrate. (That would be a very, very much more complicated equation since the hashrate reward pie is essentially static.) All it does is decrease the amount of power required to do the same work.
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017...
.. is to close off only the covert ASICBoost mining incentive which is directly incompatible with a host of high-quality Bitcoin improvement mechanisms. (An incomplete list of which is in the above link, near the bottom.)
Overt ASICBoost is completely wide-open, and remains untouched. In fact it's completely compatible with SegWit.
A lot of people are, in fact, keeping track, and can point to significant numbers of problems and inconsistencies in what people are saying and what they've supported and why. Factual assertions are particularly excellent indicators. For example, if someone says Gavin Andresen is a segwit opponent, you can literally just click one link to discover that he isn't:
https://www.reddit.com/r/btc/comments/60i4jl/miners_we_are_a...
- Segwit actualy block harmful form of ASICBOOST
- by "growing majority" you mean handful of chinese miners and 2% of nodes (http://luke.dashjr.org/programs/bitcoin/files/charts/softwar...)
It's - of course - not the only solution to this "problem", but somehow this paper doesn't propose anything else other than segwit, and his conclusion is a typical Bitcoin Core propaganda.
> by "growing majority" you mean handful of chinese miners and 2% of nodes (http://luke.dashjr.org/programs/bitcoin/files/charts/softwar...)
You aren't without knowing that in Bitcoin, miners establish consensus by voting with their hashrate. Nodes count doesn't prove anything because its really easy to host a node. The point is: there is more miners supporting BU than Segwit [1].
Literally there's another solution that has nothing to do with Segwit. gmax himself proposed it to block off covert ASICBoost. Here's a link:
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017...
Also, miners do not establish consensus. That is why if a miner mines an invalid block, even if other miners extend it, no nodes will accept it.
Just a single large Chinese miner trying to hold back progress, who likely has an economic incentive to do so due to his usage of ASICBOOST.
You are severely overestimating the amount of support BU has, and downplaying the support of segwit, which the majority of nodes and Bitcoin companies are supporting.
See here, almost nobody supports BU. https://coin.dance/poli
Bitcoin core also organize troll campaigns [2], push for a radical, unwanted change of Bitcoin with SegWit, coordinate attack on nodes when a vulnerability is discovered [3], and spread FUD everyday, sometimes technical nonsens like this very non-issue that is ASICBOOST. Those people work fulltime to stole bitcoin from the community, thanks to the huge bank account of Blockstream, the company behind all this shit show.
At least this proove that Bitcoin rely on a very solid basis, because despite having 75 million in bank, blockstream opted to corrupt core developpers and community managers instead of competing for hashrate. But still, the situation is pretty bad right now.
[1]: https://news.bitcoin.com/brief-history-censorship-bitcoin/
[2]: http://telegra.ph/Inside-the-Dragons-Den-Bitcoin-Cores-Troll...
[3]: https://www.reddit.com/r/btc/comments/5zgefe/this_was_an_orc...
A few facts on the core developers:
-- There are hundreds of developers who contribute to Bitcoin Core. Over 50 of them have 10+ commits to Bitcoin Core.
-- Blockstream employs 7 of these developers, including Pieter Wuille, Luke Dashjr, Greg Maxwell, Jorge Timon, Patrick Strateman, Warren Togami and Mark Friedenbach.
-- By number of commits, the developers Blockstream employs are ranked #2, #8, #13, #14, #19, #35, and #50, respectively.
-- Another company called ChainCode Labs employs 3 of the top 50 developers, including Alex Morcos, Suhas Daftuar, and Matt Corallo (formerly employed by Blockstream).
-- By number of commits, the developers ChainCode employs are ranked #5, #11 and #12, respectively.
As for SegWit, it is a multi-faceted gold-mine of an update with many, many benefits to scaling, security and efficiency:
1. It fixes the substantial transaction malleability problem once and for all.
2. It improves the efficiency of signature-hashing so it scales linearly rather than quadratically.
3. It 1.7x's the # of single-signature transactions per block and 4x's the # of multi-signature transactions per block.
4. It enables second-layer scaling solutions like Lightning.
5. It upgrades pay-to-script-hash transactions from 160-bit hashes to 256-bit hashes.
6. It makes it safer for hardware wallets to sign transactions by explicitly hashing input values.
7. It reduces the growth of the system's most burdensome resource: unspent transaction outputs, which are ideally kept in memory.
8. It introduces versioning for the scripting language to allow for more easy upgradeability.
Here are some facts on the developer consensus:
-- Out of the top 100 committers to Bitcoin Core, almost all of them believe that SegWit is the best way forward. There is near unanimous consensus on that front. Former project lead Gavin Andresen (no longer associated with the project) is perhaps the most notable opponent.
-- Out of all of the wallet providers, almost all of them are supportive of or at least OK with modest block increases. Almost all of the major wallet providers have prepared themselves to be ready for SegWit.
For full transparency, I [OP, author of the PDF] am currently contracting with (but am not employed by) Chaincode and am currently #39 by commits (#26 by additions though! ;)). I also think that SegWit is the best way forward; no one pays me to say that.
If Blockstream sent money to developers to endorse SegWit, they must have skipped me!
This is incorrect. Gavin Andresen supports SegWit. Your facts are incorrect:
https://www.reddit.com/r/btc/comments/60i4jl/miners_we_are_a...
I love bitcoin too. I might be underinformed but everything I've learned so far is that larger blocks and segwit aren't a dichotomy at all. I think segwit sounds good. And I think larger blocks sound good too (but I realize that they do come at a cost). If the overall transaction throughput increases but the fee income goes down for miners, I will be understanding but not too overly concerned with the miners welfare.
IMO the folks opposing segwit are not acting in bitcoin's best interest. The folks opposing big blocks are heavily optimizing against centralization. I think the latter are genuinely interested in doing the right thing but I wonder if they aren't being a little to zealous.
You aren't underinformed. You are completely correct with this comment. SegWit isn't incompatible with a blocksize increase, because it is a blocksize increase. :-)
Block sizes will supposedly be constrained by bandwidth / propagation times. SegWit will allow more transactions to occur off chain, reducing the need for larger blocks.
They seem complementary to me.
They explicitly want prices to be high/number of transactions to be limited (think, supply and demand) .
This is because transaction "price" is equivalent to "amount of money spent on securing the blockchain".
Unlimited block size? There are many reasons that's a very bad idea but here are a few:
1. It would put immense pressure on the network in terms of latency between miners, leading to less stable mining, a higher rate of reorganizations, and a massive advantage for the larger miners, resulting in increased mining centralization.
2. It would result in a substantial increase in the amount of bandwidth that each node has to handle. Modest increases are OK but unlimited blocks means the vast majority of nodes will die off. Remember that nodes have to check blocks as they come in, so this would be an insane DDoS vector. Imagine putting a funnel in someone's mouth instead of a straw and being able to force whatever you want in there.
3. It would result in a substantial increase in the amount of data that each node has to store. Remember that in Bitcoin 100% of data must be stored by 100% of nodes. Imagine "unlimited emails" with Gmail meant that you had to store all of the emails in the world for EVERYONE. Not a good idea.
I.e., if you were planning out how much disk space you needed years in advance, you would have to increase that figure non-negligibly if SegWit activates.
Propagation times have been addressed by multiple technologies, including FIBRE (http://bitcoinfibre.org/).
On-chain, actually. :-)
>2. If ASICBOOST was actually used, we'd see ample evidence on the blockchain.
The covert form of ASICBOOST (where they don't use the version field, the form that was only recently publicly discovered) would only show up as a higher than usual number of empty blocks or blocks with reordered or missing transactions (depending on how the attacker implemented it; it's not necessarily both).
>3. ... As you can also examine for yourself, the transactions are ordered essentially by fee-per-byte, which is not what we would see if they were shuffled to create collisions.
I find it extremely disingenuous that the author only focuses on looking for reordered transactions and ignores the often-noticed unusual number of empty blocks that Antpool mines[0], and the article reaches absurdity when the author claims that the lack of reordered transactions means there's absolutely nothing to worry about.
>4. Building more efficient mining chips is what miners are supposed to do, as it secures the blockchain. Framing an optimization as an attack is disingenuous. Going from 28nm to 16nm was not an attack on the network. Better mining algorithms have never been considered an attack.
If for example a mining "optimization" encouraged every miner to mine only empty blocks, then calling it an attack doesn't sound out of place. An optimization that only often encourages miners to make empty blocks isn't categorically different.
>7. Segwit is being offered as a solution to ASICBOOST. Yet Segwit has at best a tenuous connection as a response to ASICBOOST mining. There are countless other, non-controversial solutions that can disable ASICBOOST, should one choose (wrongly) to do so.
Greg Maxwell's very post announcing the issue introduces a fix for the covert form of ASICBOOST that does not involve implementing SegWit!
[0] https://bitcoin.stackexchange.com/questions/50184/why-does-a... http://www.livebitcoinnews.com/antpool-generating-large-amou...
The few sensible posts like yours are downvoted to hell and back while pro-SegWit rocket to the top. Every point you make is spot on and there is no sensible reply to any of them, just a bunch of garbage about no one supports BU and BU will never happen...
The hackingdistributed article I was criticizing was framing Greg Maxwell as making up the whole ASICBOOST attack as pro-SegWit propaganda. There seems to be a group that's extremely anti-SegWit that thinks that the ASICBOOST news is made up to push SegWit through, and the article is following that pattern.
I wouldn't necessarily call my criticisms "pro-SegWit", but it's definitely not on the mentioned anti-SegWit "side" if someone were to group things by side.
>or some well funded group is spending a lot of money to support SegWit and push Core's agenda.
The operator of Antpool has been one of the people speaking out against SegWit, and it's just been revealed that he most likely has had a huge secret monetary incentive against SegWit and not for the good of the network.
That said, I am surprised at how much this article seems to miss the point, given his otherwise stellar work. Gun, like many in the field, would agree that one of the defining characteristics of Bitcoin is its decentralization. Why, then, push the narrative that the current state of mining centralization is okay? Furthermore, how could it possibly be fine with him that these groups use a patented process - exclusive patents go against everything Bitcoin stands for - to cement their domination over other miners?
In my opinion, the fact that this was hidden invalidates much of the anti-SegWit rhetoric coming from the miners. We all can agree that the soft-fork is large and complex, but it is well-engineered, well-tested, and solves several hard problems while opening Bitcoin up to a variety of novel scaling situations. Now we learn that the opposition's absolute opposition was based on a hidden advantage, not any technical failing of SegWit itself. It is not possible to consider many of these criticisms honestly given this regressive, protectionist hidden motive.
People should read up of the Chinese miners disinformation campaign. Really crazy stuff. https://medium.com/@WhalePanda/the-extended-extension-block-...
1) The Bitcoin network began experiencing congestion due to rise in popularity driving large numbers of transactions. This resulted in slower transaction verification and higher transaction fees.
2) Users started asking for larger blocks, so that the network would have higher transaction bandwidth. The defacto Bitcoin developers at the time pushed back. Increasing block size is simple in terms of code change, but requires a risky hard fork. It's also only a temporary fix. Bitcoin will grow and require another increase, and thus another hard fork. They wanted a better, long term solution.
3) After a lot of discussion the developers came back with Segregated Witness. It's a soft modification to Bitcoin which meant no hard fork. It doesn't explicitly allow larger blocks. It does, however, upgrade Bitcoin to allow side chains. The argument is that faster, more abundant transactions can be implemented in a side chain system. A sort of second layer on top of Bitcoin. Much like how HTTP is implemented on top of the underlying TCP/IP protocol.
It should be noted that, while this enablement of side chains is the biggest feature of SegWit, there were a few smaller improvements. It _does_ increase effective block size slightly; ~2x. And it fixes a few minor annoyances in the Bitcoin protocol (e.g. malleability).
4) The community was disappointed by the lack of larger blocks. While developers were busy putting together their idea and specification for SegWit, the congestion issues on the network continued to grow worse.
5) Eventually the debate became "on-chain" versus "off-chain". Some of the community argued that they wanted to keep all transaction on the Bitcoin blockchain. In other words, they still wanted bigger and bigger blocks. I believe the root of this argument is that Bitcoin has served us well for many years, so why try to build another, potentially weaker system on top of it?
The other parts of the community were in favor of SegWit and off-chain transactions. Side chains would allow for faster transactions and significantly more of them, and they would consume far less disk space (a growing concern in the community is the disk space required to run a Bitcoin node). Side chains achieve this by compromising on some of the strengths of Bitcoin, either by reducing security or requiring more centralization.
6) Eventually the debate in the community shifted from just on-chain versus off-chain to SegWit versus bigger blocks. Now, to be clear, SegWit doesn't preclude the possibility of larger blocks. Obviously larger blocks can be added regardless of SegWit. But, for whatever reason, the dialog shifted.
7) When SegWit was finally released as part of the Bitcoin Core client (the defacto standard for Bitcoin clients/wallets) a sort of battle amongst the community began. Mining operations that ran the SegWit enabled clients started getting attacked; DDOS, etc. There was a lot of strife, hate, and fear. The way SegWit is implemented in the code, it requires a large majority of miners to support it before it officially activates and can be used. This is by design, to ensure the community agrees with the feature and to ensure almost all miners are able to understand the new blocks and not get left behind after activation. Because of this majority requirement, the battle in the community and between miners was very important. If half the miners didn't agree with SegWit, it would never activate.
8) This "battle" went on for a long time and is, currently, still on-going. Bitcoin continues to remain congested. Transaction fees have continued to rise. SegWit is still not activated. During this time a couple factions of the anti-SegWit community split off in attempts to fork Bitcoin. They modified versions of the Bitcoin client to enable larger blocks. So far, none of those forks have been successful, in terms of taking away any majority of users from the main Bitcoin network.
9) Recently it became public that a large mining operation may have been using a technology called ASICBOOST during this time. It allows mining chips to be more efficient, which means those chips make more Bitcoins for less money. Obviously an advantage. This technology has been known about for awhile now, but as far as the community knew no one was using it. It was covered by patents, and there's a sort of gentleman's agreement in the community not to use it as a result of the patent and potential ill effects on the network. (The patent means not all miners would be able to use it, so it presents an unfair playing field).
ASICBOOST, the way this mining operation has supposedly been using it, is not compatible with SegWit. That's explained in the OP.
The accused mining operation was also involved in a lot of the anti-SegWit activity in the community; promoting the alternative forks of Bitcoin. They had been accused in the past of using sock puppets to drive anti-SegWit agendas, drive character assassinations, etc.
They are in control of a large percentage of the total Bitcoin mining power.
The public revelation of them using ASICBOOST painted a dark, but enlightening picture.
As I mentioned before, SegWit ultimately has nothing to do with the on-chain versus off-chain debate. It merely _enables_ off-chain possibilities. It doesn't force them. So it was odd to see the discussion shift from on-chain versus off-chain to SegWit versus Bigger Blocks. The theory now, given the accusations against this large mining operation, is that they are responsible for the majority of the anti-SegWit movement. Because the activation of SegWit would force them to stop using ASICBOOST the way they've been using it, it was in their financial interest to prevent SegWit from activating.
That's basically my summary of events.
I think that theory, that this mining operation has been basically using anti-SegWit propaganda and other attacks to prevent SegWit from activating, so they can keep using their secret version of ASICBOOST and profit (to the tune of $100 million a year), makes a lot of sense. Why else would anyone oppose SegWit? I've read through the SegWit specs. It enables a lot of really cool tech for Bitcoin. Side chains are not just about trying to get more, faster transactions into the Bitcoin network. They're also about allowing other technologies like smart contracts to tie themselves in Bitcoin.
You know all those cool features that various alt coins are testing out? SegWit enables those features to become side chains so that you can gain all the security that Bitcoin offers. Side chain versus altcoin is like browser addon versus a whole new browser.
At the end of the day, people in favor of bigger blocks could still campaign for bigger blocks. SegWit doesn't prevent that. A mining operation being financially incentivized to block SegWit is the simplest explanation I've heard yet for why there would be significant opposition to it.
Then add on the possibility that the anti-SegWit movement might have been spawned or funded by people secretly using ASICBOOST who had no true philosophical positions about SegWit or the good of the Bitcoin network, and things are looking very... shady.
Segwit or not, these issues will have to be dealt with at some point.
Feels pointless trying to argue an opinion lately, all the arguments have been made ad nauseum, nothing new is being said, all that exists is a WW1 trench warfare stalemate.
If proven, having Asicboost hardware seriously undermines Wu et al, but fear the fact will change little. People have taken sides, it's very hard to undo such a thing.
Note there currently is no proof of these claims, it's legitimacy from authority, would be wonderful to see some proof, and who exactly did the testing on the chip? In such a highly politicised environment, evidence is far preferable to sledging your opponent for a week and then backtracking once the damage is done.
Regardless, Asicboost would explain recent actions by mining pools that looks like self-harm to outsiders.
### On a lighter note ###
>I was heavily involved in the Bitcoin community a few years ago, but have been on the side lines recently.
Apt username
Didn't Bitmain themselves confirm their chips support ASICBoost? I mean, they do hold a patent for it ... and tested it. So...
But, sure, that's not proof they've been _using_ covert ASICBOOST on mainnet.
We'll see what happens when the patch gets pushed through that fixes covert ASICBOOST. If Bitmain complains about it, or suddenly their economic strength weakens, then, well...
> Apt username
Yeah ... it's a shame I never pivoted our company into developing ASICs. Would have been nice to have another non-scam company in the mix, but ... such as it is. Nowadays I'd have go raise an order of magnitude more money to get started :/
>Our ASIC chips, like those of some other manufacturers, have a circuit design that supports ASICBOOST. However, the ASICBOOST method has not been used by us on the mainnet. We have not seen any evidence yet on the main net that anyone has used it in the patented way.
But yes, it is a somewhat important detail. It's not a long term fix, but it makes arguments for 2MB hardfork + SegWit a bit ... odd? And a more immediate increase in block size is nice, to hold us over until the proposed "better"* layer 2 solutions show up (Lightning network or otherwise).
* I put that in quotes because I haven't studied the proposed Lightning network or alternatives (if any exist?), so I can't say for sure if they are indeed better. And again, it's irrelevant whether they're better; SegWit is an improvement to Bitcoin regardless of whether those layer 2 solutions pan out for solving transaction congestion.
Many of the bitcoin developers , small blockers, and the blockstream people really, REALLY want segwit to activate.
They are so desperate to have it, that although they may not explicitly agree to a "compromise" 2MB HF + segwit proposal, they might at the very least not go freaking nuclear or something in their attempts to oppose it.
Segwit is nice, sure. But us big blockers know that if it gets activated now, then we are never going to see big blocks, regardless of how much support we get.
Don't believe me? Just check out some of the stuff that lukejr (the lead bitcoin developer) says.
They explicitly say that it could be decades before any more block size increases come around.
That said, good technology should _never_ be used as a bartering chip.
I think it's hypocritical of any Bitcoin user to be against SegWit for political reasons. From day 1 Bitcoin has suffered of accusations of being used for illegal and evil activities. And from day 1 we, as a community, have had to fend off those claims saying "Yes, it _can_ be used for illegal activities. But that's not what it's about. It's about a revolutionary technology that, for the most part, will be used for good."
To block SegWit is to go against that ethos, an ethos so embedded into Bitcoin. It's about technology; about changing the world. SegWit enables so many cool new uses for Bitcoin. Can it be used to create off-chain transactions; something some big blockers are against? Sure. But that's no reason to stop good technology. Just like it was no reason to stop Bitcoin itself, just because it _could_ be used for illegal activities.
> 2MB HF + segwit proposal
I don't understand that proposal at all. SegWit _is_ an increase in block size, by on average 2MB. So why have the additional hard fork if the block size is already increasing?
> Segwit is nice, sure. But us big blockers know that if it gets activated now, then we are never going to see big blocks, regardless of how much support we get.
The one good thing to come out of this years long battle in the community, is that ultimately users and miners are in charge. If big blockers wanted big blocks, all they have to do is offer them. Offer real specifications and real code with real tests and real study. To date, I've never seen that from the big blocker camp.
I'm not in either camp, really. Small or big. Personally, I think Bitcoin will need to increase block size eventually anyway, whether directly or indirectly. 1MB forever is silly. But nobody has offered a reasonable, long term solution to increasing block size, besides SegWit. Hard forking is very risky. Emergent consensus is risky.
All the big block supporting altcoins I've seen are cobbled together messes. And I'm not speaking as a user; I'm speaking as someone who writes code for a living, implemented the foundation of all modern mining ASICs, and have dealt with the Bitcoin Core code specifically as well.
If this whole debate really is because of soft fork is hardfork, that is easy to solve.
You can scale by soft forking using extension blocks, OR just by changing the segwit discount parameter that segwit does.
If segwit can do it, then so can 4MB segwit.
This is completely false. In fact, they're quite honest about the fact they don't mind if it doesn't. Here's an example:
https://www.reddit.com/r/Bitcoin/comments/617b7h/peter_rizun...
gmax says: "none of us are particular eager to see segwit activate-- we did our part, Bitcoin is just fine without it. -- bitcoin core doesn't even signal it by default, though I suppose the next version probably will."
> They explicitly say that it could be decades before any more block size increases come around.
Since nobody can predict the future by more than a few months, and most people know this, you should probably cite this. I'm pretty sure nobody has said that it will be decades before a hardfork. In fact, most people have been quite forthright in that a good hardfork is something they are looking forward to.
lol
> It doesn't explicitly allow larger blocks. It does, however, upgrade Bitcoin to allow side chains.
SegWit itself as implemented is explicitly a blocksize increase. More TX can fit on-chain. It upgrades Bitcoin so that a host of other, interesting upgrades become possible, but as part of SegWit there is a blocksize increase to about 2MB.
> I believe the root of this argument is that Bitcoin has served us well for many years, so why try to build another, potentially weaker system on top of it?
The interesting tangent here is that the exchanges themselves dwarf on-chain transactions in terms of economic volume. They absolutely dwarf it, and we don't see anyone demanding we put every exchange TX into the blockchain. :-) Instead we see people talking about how they should be able to put individual cups of coffee on-chain. It's odd, to say the least.
> But, for whatever reason, the dialog shifted.
This is a crucial point. The narrative change in large part is driven by totally opaque reasons that even an informed person like yourself can't remember or fathom. Why did it change? This is where speculation about the economic disincentive for it comes in, and with the evidence that covert ASICBoost is a direct option in all currently-deployed hardware from Bitmain, it becomes more clear why their constantly-shifting technical reasons might be.. constantly shifting.
> ASICBOOST, the way this mining operation has supposedly been using it, is not compatible with SegWit.
Nobody says they have more than circumstantial evidence for it being used. The main issue is that it could be used, literally tomorrow, and nobody would know it. And it would be something that would improve their efficiency even though none of their customers have it enabled.
This means their profit even in relation to their own customers would be a crushing advantage, and nobody would know, so they can just say e.g. what they did in their recent press release. "No, we're not using it."
The "problem" being that instead of being a proof of actual work, shortcuts in the work were found.
It seems then that a good solution would be to force many different types of proof of work. Different basis of hashes are obvious, but maybe some type of actual work every so often as well.
It's not clear that switching proof of works would eliminate this class of concern. It's also clear that eliminating this class of problem is not in the interest of the miners, hence the opposition to SegWit (which does exactly that).
Most miners have been supportive of SegWit for at least a year, see the HK agreement: https://medium.com/@bitcoinroundtable/bitcoin-roundtable-con... The agreement was that they'd support it if it came with a 2mb hardfork, and it never did. Sergio Lerner released segwit2mb recently, which the miners support, but many Core devs were vocal about their opposition to that.
> We understand that SegWit continues to be developed actively as a soft-fork and is likely to proceed towards release over the next two months, as originally scheduled.
The key part here being that SegWit as a soft-fork is what was agreed to.
When a hard-fork happens, it can do almost anything, including rewrite a soft fork to be cleaner. Luke Dashjr's hard-fork proposals, fulfilling his end of the HK agreement, did not modify SegWit's commitment, but did modify other annoying parts of the protocol (like version numbers -- breaking compatibility on headers) unrelated to block size. I can't speak for Luke, but if someone had told Luke about ASICBOOST's concern with SegWit, I'm sure he would have made his 3 BIPs address that too.
Which proposal are you talking about? Luke proposed a block size decrease, followed by a small increase in a decade... The HK agreement asks for a max block size increase, not a decrease and Luke never proposed such a hard fork. Again, I recommend reading the link I posted so we can discuss without false statements :)
To quote the intro: > The block size limit is reduced to a reasonably safe value (optional), and gradually increased over time, eventually expanding beyond the current limits.
It's not hard to see that this fails to meet the bar of a 2mb hard fork now. Luke is playing a game, basically saying: "you didn't specify exactly what year you wanted the block size increase in ;)"
The same "problem" occurred when people invented GPU mining, then FPGA mining, then ASIC mining. As long as SHA256 isn't broken, it doesn't matter what kind of speedup miners manage to get; whenever someone finds a big advantage, everyone else uses it soon enough.
> Different basis of hashes are obvious
"Use more hash functions" is sort of a reflexive response you get a lot from non-cryptographers, but the intuitive reason this is a bad idea is that if your hash function isn't broken, your cryptosystem shouldn't be either. It's better to fix the cryptosystem than to half-assedly patch it over by obfuscating any problems with more hash functions.
> but maybe some type of actual work every so often as well.
What do you mean by "actual work"? In all likelihood, whatever you're thinking of has none of the properties that make PoW a viable mechanism for sybil resistance.
Unfortunately in this case, ASICBOOST is both patented, and adds a fair bit of complexity to mining hardware. Both these problems have the undesirable effect of increasing the barriers to entry for new asic manufacturers - an undesirable thing from the point of view of Bitcoin users.
You're quite correct that shortcuts are irrelevant, but the reality of manufacturing makes the playing field a lot less level than we'd like.
Too late. :-)
just like any CPU or GPU solutions. just because they are more readily available you can ignore it for those cases?
China be like, lol "patented"
This isn't a break of SHA256 obviously but it is a break of the PoW for Bitcoin. The whole point is that you have to hash the same amount of data for each attempt to find a block. ASICBOOST is about not having to hash the same amount of data. This isn't a speedup of the hashing algorithm, this is about avoiding needing to hash the same amount of data for every attempt. Call it an optimization if you want, but Bitcoin was designed with the goal that this wasn't possible.
This would create many more opportunities for finding shortcuts like ASCIBOOST. Some of them might be much worse.
I get that randomly throwing stuff together isn't favoured (like TLS's PRF SHA1+MD5 construction). But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently.
> But is there any actual work where requiring 2 separate full checks to be worse than just one? That is, needing to verify MD5 plus SHA1 independently.
One could then use both SHA1 and MD5 shortcuts.
Even a crappy 256-bit hash (by using two 128-bit hashes) will probably be more secure than a (single) 128-bit hash, sure. But it won't be anywhere near as secure as a proper 256-bit hash. If you can afford 256 bits of hash, the best way to spend them is on a dedicated 256-bit hash.
Nope, the problem is that covert ASICBoost mining has created a strong, valuable incentive for miners to oppose protocol upgrades while supporting much more destructive ones like the recent extblocks, which is susceptible to instantaneous, irreversible full transaction invalidation on a reorg.
ASICBoost in its other forms is completely compatible with SegWit.