The Bitcoin Blocksize: A Summary
rusty.ozlabs.org
rusty.ozlabs.org
There also seems to be a hint of power struggle in the backlash against Gavin's push to increase the limit to 8MB.
I personally believe that Bitcoin will need to evolve if it has any hope of surviving and maintaining its value. The fact that the core community appears to be having a tough time reaching consensus on this particular issue doesn't bode particularly well for the sort of changes that may be required in the future (e.g. changing the proof-of-work mechanism).
In 2011 he was proposing to Satoshi that he should take over the project[0], in 2013 he was trying to pitch the concept that development was stagnant and that a fork was needing to fix it[1][2], and now in 2015 it's again come about that he has found an excuse to attempt it (this time with some mild enthusiasm from part of the community). In the past Mike Hearn has pitched censorship features in Tor[3], attempting to subvert the inclusion of privacy fixes in Tor[4], proposed "redlists" of supposedly undesirable transactions in Bitcoin[5]. The current branch of Bitcoin XT already includes an alarmingly ill advised hardcoded blacklist of supposedly Tor exit nodes which are de-prioritized[6].
The measure of "consensus" has already slipped down to 75% when it become clear that 95% of the hashrate was never going to happen. The solution from Mike Hearn is that if miners don't want to get to 75%, he will simply hardcode his own centralized markers into Bitcoin wallets[7] to make sure it happens regardless. This is unbelievably toxic stuff, and spells certain death of Bitcoin if it goes ahead.
[0]: http://pastebin.com/raw.php?i=3kt5Reeh
[1]: https://www.reddit.com/r/Bitcoin/comments/29qafs/circle_ceo_...
[2]: http://www.coindesk.com/bitcoin-core-development-falling-beh...
[3]: https://lists.torproject.org/pipermail/tor-dev/2014-July/007...
[4]: https://lists.torproject.org/pipermail/tor-dev/2014-July/007...
[5]: https://bitcoinfoundation.org/forum/index.php?/topic/505-coi...
[6]: https://github.com/bitcoinxt/bitcoinxt/blob/73c9efe74c5cc8fa...
[7]: http://sourceforge.net/p/bitcoin/mailman/message/34162353/
This "attack" on someone who worked for google as a network engineer, who has contributed to bitcoin for years, who has given us SPV wallets and a million other things, in this forum out of all places, sounds desperate.
The hardcoded list of "bad" peers is effectively worthless (it's already hopelessly out of date), and the additional service which downloads a new blacklist from a centralized website is completely insane. The reality of the situation is that if anybody with criminal intent wants to attack the Bitcoin network you need exactly two IP addresses (one v4, one v6) to completely disable all new incoming connections on all nodes in the network. This patch doesn't change that fact, nor that criminals often have botnets with unlimited access to new IP addresses every second of the day.
It doesn't achieve the stated design goal, and introduces new vulnerabilities which aren't stated in the commit.
> All of his other proposals are just that, proposals and "lets talk about this and the problem". I hate to think if I proposed/discussed a bad solution to a problem and suddenly I'm part of some conspiracy theory.
Bitcoin isn't like any other software on earth, it can't exist in fragmentation or in consensus incompatible forks. Operating on a fork of the software with soft consensus changes or simply operational changes that do not affect consensus are completely fine, that happens today under the assumption that you are somewhat at risk if you run software that is even slightly different to anybody elses. Some nodes in the network for example support different P2P commands, and that's fine because the P2P network is not part of the consensus at all. The linked discussions are mostly harmless, it is extremely positive that all scenarios and eventualities are debated out in the open.
Prompting companies and individuals to attempt a hard fork of the network without consensus is another matter entirely. In the light of that, the previous discussions lose their innocence somewhat.
I have not wanted to fork Bitcoin for years. I still don't - I have many better things to do, like working on Lighthouse or oh .... really anything else. Screwing about with gitian and gcc all day is right at the bottom of my list of "fun things to do on a sunny day".
But the fact that Bitcoin Core was heading for disaster was obvious for a long time now. It has been progressively abandoning things that the user community finds important: SPV wallets, unconfirmed transactions, now even the notion of growing the platform at all have all become "controversial" and therefore untouchable. Anyone who suggests that maybe these things are useful is immediately branded an idiot. Combine with a maintainer who hides any time a decision is needed and you have a recipe for deadlock.
You seem to think I hate Tor. I am actually the maintainer of a full blown Tor implementation (Orchid). I've done a lot of work on integrating it into bitcoinj and I'm basically the only guy who can actually move the needle on Tor/Bitcoin usage, by enabling the use of it by default in consumer wallets that have hundreds of thousands of installs. We're not there yet (it's still too slow) but we're a lot closer than before.
This doesn't change the fact that Tor is heavily abused. It can be useful but it's a frequent source of attacks of all kinds. So finding ways to get the good without the bad involves some tricky coding.
Below, you say "anyone can jam the network with just two IP addresses". Yes, that's unfortunate isn't it. I've been sounding the alarm about Bitcoin Core's poor DoS protection for years. Nobody listened, that's why I have now written a new anti-DoS system that can handle this sort of thing. It starts by clustering and deprioritising Tor because we've seen actual jamming attacks that came through Tor, and because using it is a lot safer and more convenient for an attacker than using your own IP addresses or using a botnet. But it absolutely should be extended to have more advanced heuristics. Instead of whinging that (gasp) loading a file from a web server is "insane", maybe you should be writing code instead.
No one in core is saying that growth is a bad thing and must be avoided, the key point that has been repeated is that the approach you and Gavin took with XT was not a good one. It has been poorly thought out from the get go. The initial proposal contended that we had to go to 20MB or we were doomed, remember that?
Bitcoin needs a steady hand at the wheel. Wladimir and co are putting out a solid amount of code with remarkably little disruption.
Your statement re: orchid is a little misleading. You might be a maintainer, but https://github.com/subgraph/Orchid/commits/develop shows that your contributions are minor. https://github.com/subgraph/Orchid/graphs/contributors
Why do you continue to cause this degree of public spectacle and unnecessary drama?
Unconfirmed transactions are not at all a failure. The data is in - usage of them has been increasing over time, not decreasing. When someone attacked shapeshift.io (an exchange that uses them!) and double spent, their response was "We get significant value from fast payments, we can easily patch the exploit they used, and we're going to continue with our current path".
The initial proposal was not "20mb or we're doomed". It was "we need more space or we're doomed, 20mb seems to work OK based on <reasons>". But lowering it to make the Chinese pools happy is not a problem, as it's an upper limit.
I'm afraid the drama has all been created by others. The limit was always meant to be removed, remember? We're not the ones suddenly trying to change the plan.
Not all his ideas are good, but he's earned the right to be listened to in the bitcoin space.
I disagree. There's good reason to believe the ideal block size is now smaller than 1 MB because large pools have been caught red-handed not validating blocks. (Which is like their ONE job and the the thing they get paid the big bucks for!)
Presumably, the only reason they're doing this is because orphan rates are too high, which in turn suggests block sizes are too high.
There's two ways you can fix the incentives. One would be to kill the block reward. You don't process transactions, you don't get any fees. This would likely cause a massive increase in transaction fees if the network were to retain its current hashrate (and thus level of security).
The other way is to make a law, basically. You'd need a hardfork that refuses to recognize any block that isn't at least 75% full, but you keep the block reward and thus avoid blowing up transaction costs. This has the problem that the network could "stall out" under the right conditions (no incoming transactions, gigabyte blocksize, etc).
Or instead of messing with incentives you can change the protocol itself - if everyone works on the same transaction set then all you really need is a fixed-size structure to propagate the winning hash, and there are no empty blocks unless there's no incoming transactions. Again, you're talking a hardfork and this one would involve core changes to the Bitcoin protocol.
You're absolutely right on that, but you have the issue backwards.
Each individual miner has an incentive to build a block which pays the most in fees.
That means that as long as the transaction rate is less than the absolute limit, fees will tend towards zero. (Any fee is marginally better than nothing).
The size of the block and propagation time are no longer strongly correlated thanks to the hard work of Matt Corallo on the relay network ( http://bitcoinrelaynetwork.org/ ).
Everyone also benefits from high scalability of transaction processing, so ideally transaction fees should stay low or zero. I don't accept the idea of meta-currency (alt-currencies and sidechains) to rectify this situation - too much risk when you have a known quantity that works pretty OK. It also decreases security by splitting processing power. If the problem is that it doesn't scale well, fix it, don't replace it. Joel on Programming's #1 Thing You Should Never Do: start over from scratch.
It's impossible to maintain the massive amount of processing power spent on this solely through transaction fees. Block rewards effectively socialize this collective interest in network security though a very low level of inflation, which hits the right population (Bitcoin holders as a whole). To me transaction fees should be nothing more than spam prevention, to prevent a DOS attack from clogging up the transaction blocks.
Really there's no reason you couldn't have an arbitrarily-large transaction block as long as the bandwidth is reasonable. You still have the problem of the chain becoming too great in size, but that's a problem with some pretty obvious answers.
0.00625 BTC in fees is enough to match the current block reward.
That's to say the current security of the network would continue on if everybody paid $1.63 per transaction in fees.
> You still have the problem of the chain becoming too great in size, but that's a problem with some pretty obvious answers.
Uh... like what?
Such high transaction fees effectively make the currency indivisible to a substantial degree. Sure you can send someone a penny, but if it costs $1.63 most people won't. So that sets a very high minimum floor before people will consider using it in a transaction. Note that this is different from transaction service fees - yes, some places have minimums for CCs, but it doesn't cost me anything to hand you a penny, or even to send you a check for a penny.
The answer to blockchain size is - sidechains, actually. What I have a problem with is meta-currency - the idea that you must necessarily transfer to a separate currency or trust a third party to perform day-to-day transactions. That creates independent currencies with their own security risks, and divides the hashrate into individual pools which are more easily attacked.
Instead what I want are sidechains that compress a series of transactions. You never actually trade on a sidechain, but at some trigger event or time interval a series of transactions is brought onto a sidechain and reduced to a net-sum of inputs and outputs, then instantly reattached to the main blockchain as a single net transaction. You can call this checkpointing or whatever you like, but it doesn't need to be (and shouldn't be) all of the transactions on the blockchain, or even the most recent transactions that are sidechained. In fact I think it makes the most sense for it to be the oldest, say the oldest week of transactions (targeted) within a rolling 28 day window. Long enough to make attacks difficult, short enough it's not too large on disk.
You essentially rebase the blockchain onto a new squashed changeset. This transaction also includes the original hash of the transaction that used to be there, so the chain still validates properly, and the hash of the last confirmed regular block, so it can be strictly sequenced. Everyone validates the squashblock to be sure that it tallies properly and hashes correctly, and if it doesn't reach consensus it's not accepted. If accepted, the next regular/squashblock must include the hash of the squashblock to indicate acceptance.
Still doesn't solve the problem of dust taking up space, but it solves most of the problem. Or maybe sweeping dust is part of the reward for the squashblock somehow. I do like the idea that if you don't sweep your ha'pennies they fall into the couch cushion. Maybe you'd implement that like "any dust that hasn't moved in a block before the squashblock is confirmed may be arbitrarily moved", so dust older than a month (average) is swept.
Have you heard of Lightning (https://lightning.network)?
A simpler solution would be to require that blocks are always 1 MB (or whatever) in size, and that they must be padded out with 0xDEADBEEF or whatever if the block can't be filled with transactions. This way all blocks will take the same amount of time to transmit, and there's no incentive not to include a transaction of any fee if what it's replacing would just be bad data.
I'm not saying I'm in favor of this proposal necessarily, but it works better.
It's not possible to ensure that all miners are operating on the same transaction set.
[0]:https://gist.github.com/gavinandresen/e20c3b5a1d4b97f79ac2
An easy way to understand the proposal is to think about it in terms of forward error correction.
The design is an improvement over the existing relay network to compactly encode transactions which are not yet fully relayed.
In practice the improvement is very small.
Information on the existing relay network is available here http://bitcoinrelaynetwork.org/
The optimal current block size is without a doubt smaller than 1MB.
The two most important metrics for block size limit are node count and miner validation.
Node count has been continually dropping for years despite heroic efforts to improve performance in bitcoin core.
Miners have been found to be blindly mining blocks in an attempt to reduce their orphan rates.
... see what I did there?
Too far in either direction is bad.
The evidence strongly supports the argument that the current 1MB limit is too high.
Wasn't that the point of Bitcoin?
But if blocks are not consistently full and there's no significant backlog, there is room for cheap transactions. This has been the case up to now. If the block size limit were to be increased, the assumption goes, there would again be room for cheap transactions - until they fill up again.
In practice, we don't know if miners would actually include cheap transactions if it makes their blocks significantly bigger. Their incentives seem to be against it, since bigger blocks have higher orphan rates.
This is incorrect in fact. The Google search term you'll need is selfish mining. It's a well studied phenomenon that for sufficiently large miner they stand to benefit from having larger blocks.
Right now there's a default transaction inclusion policy in Bitcoin Core that only includes a small number of low to zero fee transactions. Under current conditions zero fee transactions have to wait a while to be confirmed.
There's no proposal to change the default allowance for below standard fee transactions if the block size increases, so assuming cheap means zero or very low fee the increase in size will come from standard fee transactions which means increasing capacity won't drive down existing fees.
The marginal cost of each transaction in the block is approximately zero.
The only effort is from propagation speed, however thanks to the relay network that effect is almost nothing.
Say it is $0.15. A miner can make extra revenue at ~0 cost by filling a block with $0.10 offered fee transactions, but if there are many of them, they may choose not to, from the belief that enough of them will turn into $0.16 offered fees in the future (the cost is clearly ~0 in the short term, it's harder to say what it costs them in the long term to propagate the impression that low fee offers will still eventually clear).
Another miner will pickup those transactions and mine them.
Blocks are either full and there are fees or blocks are not full and fees are approximately zero.
Unfortunately the block size cannot be limited by individual miners.
(Which I realize is for a long time in terms of there being a block reward, but if bitcoin falls by 50%, so does the reward...)
(In general you cannot get 10+ MW of power without agreeing to pay for installed capacity and energy separately)
> Ultimately miners decide which transactions to include in a block or reject so if miners feel they need to get a certain fee per transaction that is their prerogative regardless of the block size.
Again the issue is a bit more complex. A miner can of course enforce any fee policy they want, but that policy will only have a significant effect if it is enforced by a large pool. And suppose a large pool decided that they will accept all transactions, regardless of fees, up to the max block size, then all other miners have to deal with the consequences of this decision, because they have to validate the blocks produced by the large pool. This is what Rusty is calling large miners attacking smaller miners.
[1] http://bitcoin.stackexchange.com/questions/1863/why-was-the-...
This might change if Gavin's IBLT proposal [0] gets implemented. Basic idea is that the miners have most of the transactions already (otherwise they couldn't mine), so you don't have to transmit them again in the block itself. You just need a small fixed-size data structure that lets miners reconstruct the full block from the transactions they have already.
Another idea is GHOST [1], which shortens the block time by letting all the forks take part in the consensus process, and get rewarded even if they don't end up in the final linear chain. It was proposed as a Bitcoin modification but just got its first production launch with Ethereum.
[0] https://gist.github.com/gavinandresen/e20c3b5a1d4b97f79ac2
Such a thing already exists, blocksize has little to no effect on propagation time to other miners.
http://bitcoinrelaynetwork.org/
Edit: not that this is not IBLT but another actually implemented setup
Bitcoin will die the moment someone figures out how to build a decentralized crypto-currency that doesn't need a stupid idea like "mining" to be functional and secure.
Yup. But I wouldn't hold my breath. No one has ever created a decentralized consensus algorithm with the properties bitcoin has, before or since. And since bitcoin achieves its unique properties as a result of the economic costs of mining, asking for a mining-less bitcoin is kinda like wishing for a perpetual motion machine.
You might be able to make mining more efficient, or based around some other finite resource besides computation power (like storage), but even that's a long-shot.
Even altcoins that have tried this approach have seen custom hardware appearing if the coin grows popular enough.
The approach just seems to be a desperate hope that home users will be able to mine profitably, but ultimately it still fails because those with more resources will win out.
No altcoin has used a PoW where memory latency dominates computation.
Home users will never be able to mine profitably (except by claiming their electricity is "free"), but the hope is they can do so within an order of magnitude loss of efficiency.
I'm not holding my breath
Links? (seriously I'm interested)
https://groups.csail.mit.edu/mac/classes/6.805/articles/mone...
If the systems security is ultimately based on a reputation model then the system is just as secure as the existing banking system.
There's no way around this under Brands system, someone has to hold the keys to mint new currency, although it can be federated in the sense than every bank, store, or even person, could have their own issue.
* There is not the organic transaction growth for this to even be a problem, and there is no evidence there ever will be. Bitcoin's only real-world use case is illicit goods.
* Even 20MB blocks would be susceptible to a cheap spam attack like the DDOS "stress test" a few weeks ago.
This is the quintessential bikeshed: a fight to the death for insanely low stakes.
That's ridiculous, what about illicit services?
I use it for buy Joylent for Europe, because I don't get screwed by exchange rate the Visa/PayPal give me.
These are real everyday uses of Bitcoin that normal people could engage in that to my knowledge are better provided by the Bitcoin network than by traditional system.
You have a lot to learn; I recommend starting here: https://medium.com/@lopp/the-multifaceted-nature-of-bitcoin-...
A deeper dive here: https://www.youtube.com/watch?v=IgETC2JMUBI
* Bitcoin is not better for any of the consumer uses than existing financial systems, and is significantly worse for most.
* the only real-world use that we see happening is illicit goods (and, as noted in imglorp's comment here, the blockchain's utility as prosecution futures makes it not a great idea for that either).
* real-world consumers, who are at the powerless end of the financial power relationship, consider regulation a feature - the only people who consider a lack of regulation a feature are cranks and scammers.
* pretty much nobody actually wants smart contracts. Dr Strangelove is a movie about an unstoppable smart contract going wrong. Both consumers and businesses want the possibility of regulatory interrupt upon a bad deal. The only people who would want smart contracts are businesses looking to screw over customers with no choice (c.f. mandatory arbitration clauses).
Look, assume I know a bit about this stuff and have looked into the brochure version claims in detail. The possibility of a use case isn't sufficient; we've had six years of Bitcoin, and that's been enough time to see most of that blog post list has been tried and failed. You've had six years, it's time to show actual utility.
Also, this is a site for communication using words; expecting someone to watch a YouTube video when you could just write words yourself is probably not effective communication.
shrugs
I spent quite a few hours developing aforementioned blog post and the presentation to my alumni Computer Science department - specifically to save me the time of typing it all out to every individual who wants to learn about it.
So no, BTC is not much good for illicit goods.
It sounds like the full blocks happened spontaneously for several months before the stress test.
Please get up to date with things. Bitcoin is huge in the remittance world, and bitcoin-like technology is being tentatively explored in all realms of financial technology.
Really, who with? I recall rebit.ph saying the problem was they couldn't actually sell enough BTC for PHP at the .ph end, for example. Bitcoin makes the transmission bit cheap, but that bit was cheap already.
Actual remittances startup company guy talks about why Bitcoin doesn't work for remittances: https://www.regalii.com/blog/the-bitcoin-remittance-myth
I think the 400 Million[1] in venture capital alone can disqualify this statement.
It refers for example to an attack by Ghash, and then dismisses their explanation that it was an employee without providing much evidence and without pointing that his dismissal of their explanation was simply speculation. Moreover, it suggests that Ghash gained 50% of hashing power and nothing happened. In fact a lot happened, so much so that Ghash currently has around 2% of the hashing power.
It is hard to present an unbiased view when one is necessarily biased. But as an engineer, one would expect someone like Rusty to present a more complete view of the problems as both sides see them and the solutions as both sides present them. His failure to do so raises questions on his integrity pertinent to his responsibility to present to the community a software which can work and problems are not hidden under the carpet.
Specifically, the Lightning Network which wishes to change bitcoin from a payment system as stated by satoshi to some experimental, uncoded, and conceptually flawed settlement system needs to be openly presented with all its attack vectors laid out as a lot of money could be lost otherwise. I do not see how I can trust such code however when one is extremely biased and makes no attempt whatever to overcome it.
Good enough is not sufficient where money is concerned Rusty. And to suggest that Gavin, or even Satoshi himself, is myopic, seeing only the "user" ui point of view, is slightly idiotic and shows that you do not quite understand the debate or you wish to wilfully mislead.
I did not dismiss it, how did you read that?
> Moreover, it suggests that Ghash gained 50% of hashing power and nothing happened. In fact a lot happened, so much so that Ghash currently has around 2% of the hashing power.
That doesn't seem to be related: they were close to 50% 4 months after the theft incident, and again 10 months after. Someone else queried that, and I did some digging to ensure that my memory of event order was correct. See: https://www.reddit.com/r/Bitcoin/comments/3fxvbr/blocksize_a...
> And to suggest that Gavin, or even Satoshi himself, is myopic, seeing only the "user" ui point of view
"only"? In my final paragraph I tried to draw the distinction between the priorities of the different views.
You seem to have read things in my article which aren't intended :(