Segwit2x Bugs Explained
bitcointechtalk.com
bitcointechtalk.com
The planned fork was supported by only part of the community, which would not have been a problem if it weren't for the lack of replay protection, which would have lost many people money had the fork been activated. The rationale for not having replay protection was to take over the Bitcoin network as a whole, with the lack of differentiation from the 'legacy' Bitcoin network being seen as a feature. In other words, rich people with a lot of mining power were planning on forcing the network into accepting their own rules (with most people in the network, running SPV wallets, doing so unknowingly), and if people lose money in the process, well, that's just too bad.
And that's just what touched me the most personally. There were a lot of other things in the Segwit2x process to be appalled with, such as the anti-developer culture, the infamous NYA closed meeting and general lack of transparency, and more.
Because I was disgusted with that bully approach, I did not invest any serious time reviewing the Segwit2x code changes, but instead did spend time studying, reviewing and contributing (my very small bit) to the Bitcoin Core code. I guess that other developers similarly had no drive to contribute to the Segwit2x codebase. I also think it's probable some people have found the bug/s and did not report them so as not to help bullies get their way.
In conclusion, the lack of review and testing, leading to the bugs, is not just a technical issue. Doing serious review and testing of open source code relies on support from the community, which was minimal, because very few competent developers in the space who understood what was going on wanted to help.
I think that the survival and modest success of Bitcoin Cash eroded the initial overwhelming support for Segwit2X. Though if you mostly read /r/bitcoin I'd say you have a very warped view of what happened ;)
Yet, they weren't around when it came time to make it happen.
> Segwit2x had over 90% of the hashpower of the bitcoin world voting for it
Yet, it had a deeply flawed software development process.
> it is still the best measure of consensus and support the network has
Yet the miners stated that they would only mine the fork for 12 hours, do you know why? Because they would have been mining at a (relative) loss. Really undermines this measure of consensus. Maybe the best measure is the market price of the coin.
> I think that the survival and modest success of Bitcoin Cash eroded the initial overwhelming support for Segwit2X.
Basically, Bitcoin Cash is what Segwit2x should've been: an honest, replay-protected coin for people who think a money's value comes from its base layer payment system's transactions per second.
Bitcoin XT had a mechanism for larger blocks in mid 2015. There were multiple BIPs and implementations for larger blocks besides that in 2016. It didn't matter much because of the stickyness of the original/default bitcoin implementation which is controlled by developers employed by blockstream, which was always intended to profit off of "sidechains" enabled by segwit and necessitated by small block sizes. (For anyone less familiar, keep in mind the first versions of the original implementation were written by Satoshi and then maintained by Gavin Andresen, one of the developers now on the "outside".)
Segwit2X was a compromise, arrived at through much effort at the Hong Kong meeting in early 2016 and again at the New York meeting in early 2017. It does appear that any compromise was futile and hopeless.
I'm not convinced by this argument at all. Consensus in the network is derived by the traders who give Bitcoin its value, not the miners. If anything, the miners are on the opposite end of the equation as they have to sell their mined coin to pay for their mining expenses (since silicon and electricity are typically purchased in fiat currency).
Look at the exchanges for consensus, not the miners. The miners are effectively slaves to the system. I don't understand why it's so common to give them so much weight.
Second, if most miners stop mining a coin, it can kill it. At least the next few blocks will take a very long time to be confirmed. Just as importantly, the security of the system is reduced because a large miner could theoretically overwhelm the remaining hashpower and execute a "51% attack" for some short-term profit. It's unlikely, but the confidence in the coin is partly based on it being impossible.
This won't happen though. Miners are out to make a profit, full stop. They mine whatever currency is most profitable, so hash power follows price. Bitcoin has over half of the total cryptocurrency market cap. So long as Bitcoin itself is highly valuable, the miners will naturally follow along as well. And as some miners temporarily defect (thus incurring massive hourly losses in opportunity costs from mining less valuable currencies), transaction fees on Bitcoin simply go up higher, making each block worth significantly more reward, thus tempting the defectors with ever-increasing profits to defect back to mining Bitcoin. There is no mining cartel.
is it? another measure is futures trading. 2x futures traded at ~%15 the value of btc immediately before the cancelation.
The failure of the Segwit2X fork exemplifies this perfectly. If hashpower truly were as important if you say it were, then Segwit2X wouldn't gone ahead with >90% hashpower support (software bugs notwithstanding). Instead, it was called off for lack of consensus within the community, and the miners stuck around afterwards anyway. That 90% figure isn't nearly as important as you're making it out to be.
Amen to that. Doing a proper review for any software is so hard, so not fun and often misunderstood and unappreciated (by management).
And then when shtf you also get the blame for your "weak review".
One project I worked on some guy presented me with his masterpiece, a ridiculously large refactoring of the entire code base and clean-up of old code that was no longer used. He was really pissed off when I categorically rejected the request and asked him to break it up into manageable pieces.
Quite sad because obviously he put a lot of effort into it but he did not keep in mind that it's one thing to do this, it is an impossibility to actually review it.
When coding keep in mind the job of the people coming after you: review, QA, eventual long term maintainers and so on.
Anything else should be done incrementally, in small manageable commits.
You can take as much time as you need in order to give a thorough review and express your frustration so that it's clear next time he should strive for something more manageable, but catagorically rejecting a large code change is just silly and juvenile.
This is why I hate agile. It's conditioning people to believe the only thing you can do to a codebase is incrementally add features. Sometimes the most meaningful or beneficial work comes in the form of broader systemic changes to a codebase.
And it sounds like you are quick to rush to judgment.
You're essentially accusing me of doing exactly what you are doing. You could have instead asked more about the context within which this all happened (which was a pretty bad situation to begin with) and if I tried to work with the person submitting the pull request.
Instead, you make me out for lazy (which is something that I don't think anybody that knows even a little bit about me) and apathetic (which also doesn't fit the bill, rather the contrary).
You can take as much time as you need to do a review if you have that time. The middle of a house fire is not the right moment to push through your pet project, and it also doesn't mean that if I point blank refuse a pull like that that I had not thought it through.
One argument against gradual refractors is that they are often left half done when enthusiasm runs out. I’ve seem mloc sized code bases with multiple layers of half done refactors and migrations.
This was part of the problem for the Segwit2X code: inadequate testing. The original Bitcoin Core codebase is up to a very high standard, and that high standard seems to be the first thing to go anytime someone goes to fork it. This is part of the reason why the developers of Bitcoin Core have so much influence to set future direction of the currency within the Bitcoin community at large; there aren't that many highly-skilled cryptocurrency developers out there. It's unsafe to launch a fork without them.
I'm putting off a very sizable one at work (due to important functionality going in before FCS), and luckily, although it'll hit nearly every file, I can do them individually.
My concern, actually, is eyes glazing over on the n'th review that's vastly similar to the previous n-1.
Refactoring is a bottom-up design tool. You remove accidental complexity and uncover higher order organization in your code, occasionally uncovering or allowing new functionality that was intractable before.
A “large refactor” is a top-down change. Major surgery. You’ve used (maybe misappropriated) the tools of refactoring for a different purpose which is antagonistic to the school of thought that enumerated these tools.
I say this as someone who when I was starting out spent a holiday weekend rewriting hundreds of lines of repetitive code (a bad idiom) to move an O(n²) performance issue into a single function. I removed 500 lines of code in the process. At that point, and for years after, my most productive week.
Enter the Mikado method. With Mikado you start to do your large refactoring. You change A, which forces you to also change B. You change B, but now C has to change too. So you change C and, you guessed it, now D doesn’t work. Man this is getting out of control. D leads to E, and E leads to F.
Now you’re looking at making change F. You are worried, or rather, you should be. How are you going to verify all these changes are complete? How do you know here isn’t a G, H and I waiting for you after F?
Now the hardest thing for any developer to do (besides admitting they gave bad advice) needs to happen. You need to throw away code you just spent hours of time sweating over. Revert to master, and start over.
Just implement change F, by itself. Sure enough, F leads to G and H, but those three changes are still reasonably sized for a code review. Send it off and start working backward up the list.
The reason we do bottom up is that we uncover the difference between our intuition about the code and the situation on the ground. When you go to do E or D you might find you didn’t need to make the change quite as broadly as you thought. And you might be able to skip A and B entirely.
When I do Mikado, I find that only about 60-70% of the code I originally wrote survives the process. The changes are more contained, which means they impact fewer of your coworkers less often. They are easier to reason about, even self-explanatory. Of course you would do it this way, it just makes sense.
The interesting thing to me about Mikado is that some people independently discover the trick themselves. In fact these large refactors I had to do, I complained, reminded me of playing pick-up sticks. I can’t move this because of that. Can’t move that because of the other thing. I got overwhelmed by the size of a couple of these changes, and threw them out. Then started over either immediately or at a later date.
Because part of refactoring is realizing that certain problems can be fixed at any time you deem it necessary to do so, that means you don’t have to fix it just because you can. A lot of my code ends up prepped for a refactor I never do. Someone else does it for me, when they have a requirement that makes the change make sense. It’s my 90-10 compromise for the tension between YAGNI and broken windows. I don’t make the change, I just make the change easy.
But if safety does not matter, or its pure crap anyway. Then refactor away.
Proper review of core components that are used and proven is hard already, refactoring such components - don't do it if avoidable - there must be a very good reason. And convenience ain't one - you'll probably get some duplication of code and double maintainance during the transition period.
Making small PR is just a way to be safe and smart, one branch with individual gradual commits that could be used separately is also a way (if they could be reviewed separatley).
Maybe that would occur if you had to replace a central algorithm with a significantly different one (because the original one did not scale, for example). That is more of a rewrite than a refactor, and should be done as a separate product, developed incrementally, of course.
If 'refactoring' did only refer to simple, mechanical changes like renaming, I could see it, but you have just argued against that interpretation.
With lots of testing, yes, you can incrementally build a working system that you don't understand. But by myopically staring at diffs all day, you'll only get farther from the truth of how your system works.
The major refactors that I do, are because I've come to the realization that my data model was not correct. A realization, that often happens multiple times during a project. My utility functions are still useful, but I have to rewire everything. There is no meaningful way to do that incrementally. Why would you even try to do that incrementally? It's like, if you have a salamander and you want to turn it into a horse incrementally. Of course, most of the RNA is the same. But testing and code reviewing the steps between salamander and horse seems to me like a waste of time.
While "if I understand the steps which got me here, than I understand where I am" may not be guaranteed (especially if you don't know where you started from), it is almost always among the more effective ways to proceed (though it is not particularly helpful if you don't know where you are going.) It can help avoid getting lost, which is one of the reasons for doing development incrementally.
There is no way in which I can see that turning a salamander into a horse has anything to do with refactoring.
In static languages such as Java, renaming a variable is something your IDE can do for you with all the benefits of static analysis. There's almost no risk associated, and reviews can be a rubber-stamp.
/s
Now, after segwit2X has failed and proved that it isn't likely to succeed any time in the next X years, well us segwit2Xer moderates have joined the bitcoin cash people (as well as some have moved to ethereum).
That community is much stronger and we are able to survive because it was a hardfork.
Lite coin has zero community of note. It is mostly just speculators, and people who suck up to the bitcoin Core team.
There frankly isn't even a litecoin vision. Like what do litecoin supporters even believe? What even IS a litecoin supporter?
I go to a bunch in person crytocurrencies and talk to people. And I can't think of a single time someone has said "oh, I am really excited about LTC" or similar.
Nobody is going out to coffee shops and saying "hey, do you accept litecoin?". Nobody is hosting litecoin meetups or tech talks or anything at all.
My biggest problem with litecoin, though, is that it is an unthreatening, PC, coin that has no intention of shaking the status quo, pushing the needle, or doing anything groundbreaking that might upset the powers that be in the cryto space.
If you aren't making anyone mad, then you probably aren't doing anything interesting or important. And I can tell you that nobody is 'mad' at litecoin.
The block size is fundamentally too small to support the current volume of transactions in a meaningful way, and the weak (and suspicious!) arguments in favor of letting bitcoin’s transaction time spiral into hell don’t hold water.
"Use of uninitialised value of size 8"
"Conditional jump or move depends on uninitialised value(s)"
"Syscall param writev(vector[...]) points to uninitialised byte(s)"
Also It doesn't free some memory. Valgrind is not silver bullet, but it helps alot. Bitcoin Market capitalization is around $140 billions and these kind of bugs should not be there.
https://github.com/openssl/openssl/commit/d8ca44ba4158a9dafe...
One reason why testing wasn't thorough enough was that it was under specified. There was no public discussion on how this was supposed to work. The specification was whatever the maintainer decided to merge, which changed several times during the software's lifetime.
Most further discussion or counter proposals, including perhaps the most controversial one which was replay protection, were met with "that wasn't what the signees agreed to" and that anything going beyond that was off the table. But what the signees agreed to was always very unclear, as the agreement only described that a "fork" was to be activated and that something was supposed to be 2 MB in this fork. What was supposed to be 2 MB was not specified, let alone how deployment was supposed to work, and neither the protocol for signalling, fork activation and block weight. And the latter things are probably what reviewers would have concerned themselves with had this ever been a real proposal.
It was supposed to be twice as much as segwitz done via a base blocksize hardfork (thus, the name segwit2X).
The only people who were "confused" about what it was were the smaller block trolls who didn't even sign or support the document in the first place, therefore their opinion doesn't matter. The only opinions that mattered were of those who actually supported the NYA and segwit2X.
Some other details, such as replay protection, we're obvious sabotage efforts that were also proposed by the small blocker trolls. None of the NYA people were arguing for that, and once again, they were the only opinions that mattered.
The other details, such as deployment date, signaling method, ect were not specified, though, and I agree that this came back to bite everyone in the ass, as proven by the fact that it failed.
Note that the most common reason among the former undersigners for withdrawing support was the lack of replay protection (which only underscores the lack of consensus around some fundamental design decisions among the signers).
To anyone familiar with post segregated witness Bitcoin an increase in base block size could be done in a number of ways. The weights for data structures could be changed, but so could the weight limit. What would be reasonable if backwards compatibility is no concern is use the same multiplier for all types of signatures, thereby removing the so called segwit discount, but this turned out not to be the design chosen.
Like most people interested in Bitcoin, I only followed the NYA at a distance and have no more information than anyone else in this space, but I do know some people working with a company that took part of the agreement. I do not think there was some secret cabal behind the agreement, which some people seems to believe, but I also think that there were conflicting goals involved.
... and that's opetruzel's (almost) only contribution to anything in github in his/her two year history.
Isn't that odd?
We use a better pattern at work. On our project, we require approval from members within the team for a commit to go in. If there's one person with the best experience in any given area, or even just a strong opinion, then their approval is required as well. It's a big anti-pattern that I catch myself thinking about occasionally. It's temping to send out a commit for review by more junior/inexperienced code members, bypassing the more experienced reviewer who you know will have lots of input you might not necessarily want. That's what Jeff Garzik did here; by the end of it he didn't actually want a review, just a rubber stamp, and so any stamp would do. Thus the entire point of doing reviews was bypassed. It's particularly egregious that he never even bothered to write tests as requested by another reviewer.
“Reviewing and testing consensus changes is really, really hard. [...] Essentially, even one or two weak reviews in a chain of reviews can break the entire consensus system with a catastrophic bug.”
Use multiple reviewers for critical code (treat it like proof reading an email you are about to send to millions of customers).
Write automated tests if you can, including for regressions and edge cases, but also to assert your code works the way you’re saying it will.
Finally, check the code out locally if you’re a reviewer and verify the changes yourself, don’t just read the diffs.
At the current level of use, this issue is a side note. At the envisioned level of use by crypto promoters, it woukd be a catastrophic economy-crypaling disaster that would have impacts far beyond a single institution. As infrastructure, this sort of thing is must-work cannot-fail technology on the order of nuclear reactors.
I'm not saying crypto currencies aren't in our future at that scale. I don't know if they are or not. I'm saying we have a much, much longer to go, and the order of a decade, before they have time to mature and then prove that maturity. Until then, they're a speculator side show. Heck, with so much mining capacity in China, they exist in significant part on the suffrance of the Chinese government not feeling too threating by the level of social disruption they might cause. If that threshold is crossed, significant mining capacity an transaction speed that goes with it is a few great-firewall configs away from oblivion. Yeah, miming difficulty will kick in ti alleviate pressure, but only after disaster-level and life-savings-wipeout levels of financial fallout ensue.
It shows how bad things can be if you don't use development practices that are appropriate for the project, but it doesn't show anything at all about how actual bitcoin code is developed. This is about a fork (that never ended up happening).
no, there is the open market. 2x futures traded well below btc's value.
from https://lists.linuxfoundation.org/pipermail/bitcoin-segwit2x... :
"Unfortunately, it is clear that we have not built sufficient consensus for a clean blocksize upgrade at this time."
The real problem with segwit2x is that the consensus was dropping and was trending toward a very weak level. It had strong consensus for long enough to do a switchover, as far as I understand, but it didn't take the opportunity and then people eventually started to change their minds.
If 95% of miners wanted something, but nobody else did, then every single block they mined would be ignored as invalid by the rest of the users in the ecosystem. They'd be foregoing huge profits by the minute, and inevitably would start defecting in ever-increasing numbers.
The only way a hard-fork ever has a prayer of taking over the main chain is if it gets implemented in the most popular software programs used on the network (e.g. Bitcoin Core). This is what killed Segwit2X, despite seemingly having massive miner support. Note that the miners immediately capitulated; such easy capitulation does not lend much weight to the importance of the 90% figure. They're full of hot air.
There was no reason why one wouldn't run a testnet and see how fork will go after block XYZ.
I'd rather have a global ledger with consensus than allow a few companies to stop all banking at weekend so all banks arrive at a consensus.
This was never about scalability-- the actual on chain capacity more than quadrupled in August and Lightning ads a nearly infinite increase in theoretical capacity.
This was always about the self important CEOs wanting to take over the project and force the engineers into their will.
For me, as a young adult, the most important part of growing up has been learning to not engage in bike shedding. It is hard not to bike shed. When design questions come up, and I see one thing as being WRONG, it is really hard not to become myopic about it. But this is a great example of how bike shedding can be harmful.
https://www.youtube.com/watch?v=nSRoEeqYtJA
And I think this is what Greg's referring to:
There should be a law of bike shedding, where you don't realize you're bike shedding because omg it's the most important thing in the world.
The 2x project would have doubled this to a 2mb weight but a 7.4mb physical block theoretical max.
For those unaware, like me, this refers to the Law of Triviality.