Bitcoin: Soft block size limit reached | Hacker News Reader

Bitcoin: Soft block size limit reached

bitcointalk.org

49 pointslignuist59 comments

I am not a developer for bitcoin, but I have been following along in #bitcoin-dev and reading some of the forum threads. (most interesting conversation by far is on freenode)

In pools/miners that are not blocking satoshi dice 70-95% of every block is filled with SD transactions. Further, SD has upped their fees to get higher priority.

Interestingly, the sentiment of the devs and the sentiment of the users is vastly different. It seems that the users are overwhelmingly pro SD, anti filtering out SD transactions, and think this is just a nice test for bitcoin and this type of transaction volume is inevitable anyway. (plus, they are of the libertarian leaning in general).

Sentiment amongst the devs varies, but it does seem they all agree that SD is abusing the blockchain. There is no reason SD can't remain provably fair and have the same flow without spamming the blockchain; and, since they are reusing addresses and acting in a such a way that their transactions can be trivailly tracked, there is no reason (in some of the devs' minds) to not treat them as spam and filter out their transactions.

For their own part, the owners of SD have reportedly taken a "fix your shit" attitude when asked to change their methods.

In any case, it has spurred a lot of discussion on "how" to raise the soft limit; which is completely arbitrary, and many pools have already raised it regardless. The real news is that the `hard` limit, which will create a fork is not even an order of magnitude away. No one wants to keep making hard forks, so there is a lot of debate on how to best determine future block limit size on a system wide scale.

For my own part, it is hard not to see SD as spamming/abusing the blockchain. If I ran a mining pool or a hasher (I don't) I would block SD as Luke-jr does by not including addresses into blocks that start with `1dice`. It's note worthy that it would be essentially impossible (rather than trivial) to censor SD transactions if they followed best practices.

It is also worth noting that there are whispers of SD transaction volume being something of a conspiracy. I have not done any analysis on this; but what I have heard is that the vast majority of SD transactions appear to be bots, playing losing bets 24 hours a day. SD has issued stock, which is publicly traded.

Interesting observation. Actually, I think most of the devs are against filtering SD, too. There is just one very vocal one against it. As far as I can tell, all of the long-time devs oppose any filtering, since SD is working within the limitations of the Bitcoin protocol as it is defined. Please correct me if I'm wrong. But I agree with you that SD could try to find some ways to reduce the quantity of spam. One simple but drastic solution would be to run their own block chain and use it as poker chips that players could easily buy with Bitcoins (once you're in the virtual currency system, exchanging currencies is trivial).

How can you have a viable worldwide currency subject to "spamming the blockchain"? The nice thing about gold and USD is they have no viable DOS attack vectors.

What implications does this have on the future of bitcoin? Is it even possible to have a reliable currency if anyone can create a crippling load on the system?

Well you determine what counts as spam and just don't propagate it. Fees are a part of the process of keeping spam out of the blockchain. Some people also count sending 'bitdust' as spam.. seemingly defined as anything less than 10k satoshis (or 1/10,000th of a bitcoin--obviously that definition would have to change over time if bitcoin continued to appreciate).

The real answer to your question though is 'off chain transactions' will likely become the way a lot of debts are settled. It's trivially easy to double spend coins that have been not confirmed into any blocks, so it's inevitable that this type of fraud will become more prevalent. As such, it makes things like 'buying coffee' with bitcoin really annoying, if I am going to have to wait 2-15 minutes for my transaction to get confirmed into a block it isn't really going to work. So B&M retailers will be forced to make their customers wait for transactions to confirm, or risk getting scammed (which breaks the whole "point" of bitcoin having lower fees than CCs).

Enter off-chain transactions. Retailers could either let you hold a balance there, or, more likely.. someone will pop up with a competitive off-chain transaction scheme; perhaps Ripple, perhaps something more like bitinstant; where balances are demoninated in bitcoin and retrievable in bitcoin, but day-to-day small transactions actually happen in a 'centralized' database and are only written to the blockchain upon cashing out. (note this is exactly what is happening with mtgox, where tons of coins exchange hands just within their system that don't get written to the blockchain until you cash out your coins).

Reading the thread, the most interesting thing seems to be that this is a problem partly caused by the website satoshidice.com, and several people are adding special filtering to ignore any transactions performed by this website.

This immediately raises a large number of very interesting questions. I was under the impression that the whole point of bitcoin was that it allowed anonymous transfer of funds with no central authority. If some group of large miner groups have the power to a) identify satoshidice, and then b) ignore it's transactions, then what is the point of bitcoin?

Based on my very elementary understanding of bitcoin, it seems to me it would undermine trust in the entire system, as the bitcoins lost on Satoshidice may still be spent elsewhere, and entire chains of transactions may be lost. Correct me if I'm wrong.

Seems like this fundamental issue of every node needing to replicate all transactions is turning into a real problem. I expect there will be a system of supernodes at some point, and most nodes will only store hashes. Operators of the supernodes could then become targets of regulation, undermining the decentralized nature of bitcoin and eliminating it's primary purpose.

I'd love to be wrong. Anyone?

See parallel comment for my (shaky) understanding. As an analogy.... I can't think of a good analogy!

Everyone still stores the hashes, some just don't use these transactions as hash input. May cause problems with satoshidice transaction processing and yes, could result in slow transactions and potential double spends if a large proportion of miners did this.

AFAIK Satoshidice has some clever behind-the-scenes cross validation that makes a double spend impossible even without confirmations. That's how they're able to issue payments so quickly.

Does that guard against a double spend with them or a double spend on the blockchain as a whole?

i.e. could I (theoretically, if this got a lot worse) spend BTC at Satoshidice, count on them not being validated very fast because Satoshidice transactions are ignored by a proportion of miners, and spend the same BTC somewhere else?

I'm just an interested observer at this point, I have no BTC and I'm not into online gambling.

Currently, rule-of-thumb is to wait until a transaction has been verified by 5 miners. But instead of counting, most people just wait an hour. Some merchants are more thorough than others, so if you find one that doesn't wait for a transaction to be verified properly, you could certainly defraud them within an increasingly-large time window.

However, eventually the network will include one of your transactions in the longest chain, and the other will be discarded. So the double-spend will not be permanent.

> eventually the network will include one of your transactions in the longest chain, and the other will be discarded. So the double-spend will not be permanent.

The longest chain? Why would length be the determining factor?

If someone double spends a bitcoin, it could saved by one recipient and quickly be spent many times by another. The most fair way is probably to take the first spend.

Regardless, anyone ignoring a block of a transactions seems unsustainable long term if bitcoin is going to have any credibility. Seems like those with an interest in bitcoin should avoid even considering this.

Since it's possible (and increasingly likely as more miners join) that two different miners will add different transactions to their chains, the clients are all programmed to ignore all but the longest block chain. This is important because it's harder to spoof a longer block chain than a shorter one, so longer block chains are more secure.

How is it hard to spoof a longer block chain? If you intend to do so, it's trivial. You can create an unlimited length block chain and screw the short one that is perhaps the honest one.

This is one of the more disturbing defaults I've heard of with bitcoin.

There is finite (about 25) number of accounts owned by satoshidice and every one of them starts with "1dice", so there is no dificulty in identyfing satoshidice

btw: bitcoins by default allow semi-anonymous (because all transactions are public; if you want to be anonymous you need to do some tricks) transfers.

AFAICT (not authoritative, please correct me if wrong on the way it works here, my understanding is a touch shaky)

The 'mining' operation is basically a race to produce a valid block for the blockchain and have other miners accept it. The hash input data for the block is transaction data. Per-protocol rules, any miner is allowed to ignore any transaction they want to. Partly this is so that they can (if they wish) concentrate on processing transactions with a decent fee attached. This is thought to become a more significant thing as the built in reward for block discovery shrinks over time.

Therefore there is no guarantee that any transaction will be processed at all, and certainly no guarantee that it will be done in a timely fashion. Though it probably will. This is part of the wait for confirmation.

If a large miner group ignores satoshidice it doesn't mean that they disregard satoshidice transactions, it just means that they won't use them as input to the blocks they're attempting to mine. Others will still use them as input, and if the successful block comes from one of these others then the whole ecosystem counts the transaction.

However, If everyone did this, then satoshidice would indeed find it close to impossible to get transactions into the system.

--edit-- obviously this doesn't address your anonymity concerns, but others have done that.

Bitcoins are not anonymous, they are pseudonymous. It works, by announcing a transaction from one of your accounts ( defined by public and private key) to another account. So if you can keep your Bitcoin address separate from yourself, then you are anonymous. But there is nothing in Bitcoins which protects the identity of your address. In fact the entire point about the Bitcoin protocol is, that there is a method to find a consensus of the contents of each individual address.

So long as there's someone who mines coins and who accepts satoshidice transactions, they will eventually be completed. Being discriminated against slows your transactions down, it does not prevent them.

The way the system is designed to deal with discriminating miners is to allow all discrimination (a miner can choose which txns to include based on any criteria he/she wishes), but diluting their choices by having enough miners.

SatoshiDice is actually very useful for the network, because it allows us to test how it functions in a saturated environment long enough in advance that we can make changes if they are needed.

Yet this action does seem like collusion which was the oringal point. I didn't know transactions could be excluded as well. Who's to say that in the future miners could collude against certain merchants?

Oh they could if they wanted, and if they knew what the addresses in use by the merchant were. This could be made difficult by the merchant continually generating new addresses. The merchant could also add a little sweetener (transaction fee), and it feels pretty likely that a proportion of the wider BTC community would overcome their (presumably) ethical concerns and process the transactions anyway.

But yes, in principle, colluding miners could make life difficult for merchants they dislike. This is one reason why a large and divers base of miners is important to the BTC system.

Goto http://satoshidice.com/ and look. They have pre-calculated addresses that all start with '1dice' and payments to these addresses are considered a bet.

The miners are not really filtering satoshidice, they are filtering all transactions that start with '1dice' this includes them and a tiny number of false positive matches.

This thread provides further detail and discussion on the SatoshiDice issue:

Should the bitcoin community ban the Satoshi Dice filter patch? https://bitcointalk.org/index.php?topic=150481

especially this comment from misterbigg:

https://bitcointalk.org/index.php?topic=150481.msg1599107#ms...

The main complaint with Satoshi Dice is that most of their transactions are not actually transferring any value (actually, they are transferring a tiny bit of value if I understand correctly; I've never used SD). They're using the Bitcoin system as a secure message system to notify everyone who lost a bet (i.e., most betters). It seems to me that Satoshi Dice could very easily create their own block chain to do this and just use it as poker chips, which would take all the load of the Bitcoin system. Once you are "in" the virtual currency system, it's quite easy to exchange currencies. The hard part is getting your dollars in.

But Bitcoin would have run up against the block size limit at some point anyway; the vast quantity of Satoshi Dice transactions have only accelerated the confrontation of this dilemma. I actually appreciate the fact that Satoshi Dice is effectively stress testing the system. Bitcoin block size is a serious impediment to scaling up.

I wonder if Bitcoin core dev (and Google employee) Mike Hearn has consulted with Jeffrey Dean and Sanjay Ghemawat about this issue. I know that Bitcoin recently switched from Berkeley DB to leveldb for most of the client's bookkeeping, which would have been a good opportunity for Mike to approach them about this dilemma. However, I know that all the core Bitcoin devs are already very capable, so I'm not sure if a technological solution to this issue even exists. The answer may be more political (i.e., convincing all the miners to go along with an idea).

<idlespeculation>On a related note, I wonder what the level of adoption of Bitcoin is among Google employees. As mentioned above, Mike Hearn is a Google employee and a long-term Bitcoin core dev. I would speculate that the level of adoption is relatively high, which makes me also speculate about when Google might start accepting Bitcoin. Anyone care to comment?</idlespeculation>

I doubt Google as an organisation would accept Bitcoin unless the adoption was overwhelming. Google Wallet is yet another data mining tool for them, and they won't give that up easily.

Single reason I don't have a Nexus 4 is that I have to use Google Wallet and I won't give Google my real name to link it to my account no matter how hard they try.

Unfortunately, I think you're correct here. I guess I'm hoping a cabal of Bitcoin insiders will somehow convince the powers-that-be to start using btc, but your interpretation is much more likely.

I'm not following this. Can someone dumb it down a bit?

There are too many bitcoin transactions, so the blocks are too big. The miners have to be run with different settings to account for that.

I don't know what bitcoin proponents think will happen in the bitcoin future when everyone will be payibg with it; the size of the block will quickly get to megabytes.

Yeah, bitcoin doesn't scale well.

But Moores law

Since the issue is disk storage, not transistor count, that should be Kryder's Law.

http://en.wikipedia.org/wiki/Mark_Kryder

Of course if bitcoin transaction volumes rise faster than that, there's a problem. Which, uh oh, they are. By a lot!

http://blockchain.info/charts/n-transactions

http://blockchain.info/charts/blocks-size

>bitcoin doesn't scale well

Meanwhile, average and available bandwidth and disk space goes up.

By the time the block chain breaks into the terabytes range, storage and network technology will have advanced to the point where it's not a problem anymore.

But what if Bitcoin - and therefore blocks and blockchain - will rise more quickly than the technology?

Yea. Everyone screaming Moore's law (a historical observation that isn't guaranteed to continue) must be also saying that the Bitcoin economy won't double from its current size every couple years.

Does every new user have to download the entire blockchain? That's a point I'm not really clear on.

When you first start up your client, yes. From then on it's just kept in sync with the network.

It was around a gigabyte back in January.

If you are hosting your own wallet and not using an online solution yes.

Do you know of an online wallet that seems to be reputable? I'm looking at Coinbase at the moment.

Blockchain.info is quite popular. That's really all I know.

If you run a full client, yes.

It is perfectly possible to not do this, but doing so requires that you trust someone (the one running the client for you).

Currently yes ( with the official client). But this is not a requirement of the system. Theoretically you can announce transactions without looking at the block chain, but then you do not know how much BTCs are in your account. There is another optimization, the block chain consists of an Merkle tree, such that you can verify the block chain without storing all the previous blocks. In this case you still need to download all blocks, but you only need to store ~ 80 Bytes per block. [1]

[1] http://bitcoin.org/bitcoin.pdf

Someone on Reddit recently dug up this email from the early days of bitcoin development, from Satoshi Nakamoto (the founder of bitcoin).

He anticipated this problem and foresaw that eventually only specialized hardware would run full nodes, and everyone else would run on a highly simplified system requiring a lot less bandwidth/storage.

http://www.mail-archive.com/cryptography@metzdowd.com/msg099...

Those people with high power nodes will be known as banks - see Dan Kaminsky's thoughts: http://www.slideshare.net/dakami/bitcoin-8776098

"Bank" carries a lot of unfortunate connotations with it that won't necessarily apply to the supernodes. It would be better to use a different term for those.

What's likely to closely resemble banks in the future are bitcoin clearing houses, where transactions just move around internally. These will be supernodes themselves, but not all supernodes need to be this kind of clearing house.

There is a hard limit on the size of the block. We are already 25% of the way there. This is going to become a problem in another couple of years, because the protocol cannot handle the rate of transactions.

I just tried to fire up a new client last night and the whole thing just ground to a halt with about 18,000 blocks left to process. Is this why?

Upon further reading, it sounds almost like a supply and demand issue that needs to be solved with an increase in transaction fees, no?

Yes, one of the options is to let the market take care of it.

But then you cannot advertise Bitcoin transactions as free any more, as free TXs will likely be lost at a non-negligible rate in the future.

Blocking Satoshidice is a temporary fix to give time to the developer community to come up with a better plan. So is increasing the soft limit (we are currently at 1/4 of the hard limit, doubling the soft limit will put as at 50% of it, and it will double the blockchain bloating rate).

What type of solutions could developers come up with? Would most of the solutions involve a "hard fork" that I've been hearing about?

There are a few things going on already, and some ideas floating.

Another one is the integration of hybrid systems. For instance Ripple + Bitcoin (bulk of small transactions would move out of the block chain, basically things that wouldn't need to be register and/or have extra anonymity requirements).

There are a couple ideas on alternative data structures to store the blockchain more efficiently. This would mitigate both bandwidth and storage requirements by potentially an order of magnitude. (Although not so much for miners or general users, but mostly nodes and users of the "standard client" which is still the most used one).

So far nothing I've said requires a hard fork.

Now, forking (some of these are far from definitive solutions, basically only fee requirement changes are a real solution from the following ones):

Enforcement of a minimal but non-0 fee in the very protocol instead of by miner consensus. The way to go IMO.

Higher hard limit for the block size (kicking the can, but this can be a very effective kick that would very likely allow for the previous things to happen over time)

Equivalently, reduce avg. time per block.

"Floating block size" - not a solution proper but a mitigation system to avoid translation volume instability. It would adjust in a similar way difficulty does.

I don't know what's going to happen medium or long term, but the best plausible and likely scenario in my opinion is 1) some fee enforcement scheme 2) the bulk of small and local transactions to happen offline - both solving the latency problem and minimising blockchain bloat by orders of magnitude.

I think it's fair that "free TXs will likely be lost". That's really just the market at work - If Satoshidice would pay proper fees, I bet those miners would be very happy to lift their block. In a sense making it "worth their effort".

Any P2P network dies when leeching of resources completely outweighs sharing of resources. I think Bitcoin with its transaction fees has a very clever mechanism to prevent that from happening.

Satoshi dice transactions pay relatively high fees and is probably the only reason transactions have any fees at all. Miners already limit the number of free transactions that are processed some (BTC guild) don't even process free transactions. The problem is luke-jr's pool explicitly blocks transactions including satoshi dice addresses. For whatever reason possibly to artificially lower transaction fees to increase adoption. If the miners acted economically this wouldn't be a problem.

Ah, interesting!

I would still be curious whether it's just a matter of price, though. Surely at some point the miners couldn't resist processing the transactions?

Nope, they don't pay relatively high fees for the strain they produce. Although "relatively" is a very loose term I guess.

Bitcoin lead dev addresses the issue- https://bitcoinfoundation.org/blog/?p=135

Another curiosity I had. Has anyone considered storing files in the block chain?

Obviously it wouldn't be practical but it might help with censored files.

You can store ASCII in the blockchain. Has been done before: http://news.ycombinator.com/item?id=2830084.

One can only imagine what ordinary folks who thought they'd be on the bleeding edge in accepting bitcoin think of all this.

All this talk of spamming block chains, miners, soft-hard limits - word that mean nothing to your average cowboy - can't be good for bitcoin's near future.

So could a miner mine more coins by settings their max block size lower?? Or am I misunderstanding completely?

No. If they lowered their max block size, they would get less fees from transactions.

No. Nearly all (99.9999% or thereabouts) of the work you need to do to mine bitcoins is independent of the size of the block you mine. You want to run large blocks to collect as much transaction fees as you want to.

In the case of a hard fork does an end user have to do anything to move their coins or is it transparent?

Reply on news.ycombinator.com