For many people it was clear that off-chain transactions are the only feasible way to solve this. Small purchases in particular do not need to be broadcast to the whole network and can be routed in payment channels. Lighting Network is the most promising solution at the moment.
Isn't that what "off-chain transactions" and the "Lightning Network" are?
Can it operate on it's own, or does it need to be a layer on top of a cryptocoin network?
What incentive do LN node operators have to operate?
https://cdecker.github.io/lightning-integration/
It runs on top of Bitcoin-like Blockchains that have segwit or something comparable.
If you run a LN node you can charge fees for routing payments.
The most expensive step would be opening a channel, but there is work on reducing that cost.
But right now this is still VERY early in development.
You don't need to store Terabytes per day for 1B daily transactions. BC does it differently, but you could get away with 4 byte time-stamp, 4 byte amount, 32 byte source public key, 32 byte destination public key = unique transaction + 32 byte signature demonstrating sender with correct private key signed that transaction. Now, 1 billion of those = 100GB per day = 36 TB per year which sounds like a lot but most nodes don't need the full amount just current state + verifiable hash + new transactions.
But even that's not needed, suppose every node includes a subset of the account amounts so every K nodes allows you to recover the state of all accounts. That way you don't need a full database just the most recent K nodes and they have complete state. Some people can keep a historic archive, but the system operates even if that information was lost. (You would want some GC as well so accounts must either be larger than A or have a transaction within the last Z steps.)
Granted, managing pending transactions increases bandwidth needs, but nodes can manage reputations based on only forwarding valid transactions. Some metrics so their is some statistical chance of conformation and you end up with ~2x overhead for unconfirmed but likely transactions.
PS: You can also have the protocol grow so if the 11 of the last 20 nodes where full the new node is 1% larger.
Satoshi made a similar argument in Nov 2008 in response to a claim about Bitcoin not being able to scale:
"The bandwidth might not be as prohibitive as you think. A typical transaction would be about 400 bytes (ECC is nicely compact). Each transaction has to be broadcast twice, so lets say 1KB per transaction. Visa processed 37 billion transactions in FY2008, or an average of 100 million transactions per day. That many transactions would take 100GB of bandwidth, or the size of 12 DVD or 2 HD quality movies, or about $18 worth of bandwidth at current prices."
"If the network were to get that big, it would take several years, and by then, sending 2 HD movies over the Internet would probably not seem like a big deal."
https://www.reddit.com/r/btc/comments/6navjt/very_first_disc...
Note: this was before the DoS attack in 2009 where hackers tried to freeze the network by creating blocks of uneven or massive size. Which is why in Oct 2009, satoshi imposed the 1MB block size limit (I presume was intended as a temporary stopgap measure) to prevent future similar attacks.
>>The current system where every user is a network node is not the intended configuration for large scale. That would be like every Usenet user runs their own NNTP server. The design supports letting users just be users.
-July, 2010
>>It would be nice to keep the [block chain] files small as long as we can.
>>The eventual solution will be to not care how big it gets.
>>But for now, while it’s still small, it’s nice to keep it small so new users can get going faster. When I eventually implement client-only mode, that won’t matter much anymore. (note to readers, "client-only mode" refers to SPV mode)
-August, 2010
and in an email to Mike Hearn:
>>The existing Visa credit card network processes about 15 million Internet purchases per day worldwide. Bitcoin can already scale much larger than that with existing hardware for a fraction of the cost. It never really hits a scale ceiling.
So your explanation for his reasons for putting in place the 1 MB limit are not consistent. Others have said that he was afraid that someone would soon come out with GPU miners and be able to DOS the network with large blocks, so put the limit there as a temporary measure. Now that mining hardware evolution has slowed, and the mining ecosystem matured, this kind of threat is non-existent.
Actually the reason I stated is consistent with what satoshi said:
"In 2010, a block size limit of 1 MB was introduced into Bitcoin by Satoshi Nakamoto. He added it hidden in two commits[1] in secret, when challenged publicly he said it is a safety measure to prevent miners from creating large spam blocks." [1]
[1] top of: https://en.bitcoin.it/wiki/Block_size_limit_controversy
Another thing to keep in mind is 100GB/day / (86400s/day) = 9.2MB/sec ~= 90Mbit/sec Internet connection. That's not a small potatoes connection...
Also, most people don't need to actually run a full node to be a miner. Assuming you trust someone running a full node and the block is signed by hash of hash of (node + random value ) < some value then most miners need only get that initial hash from the trusted party which is under 1kb. You can also send back the best hashes you find to the full node to demonstrate you actually have that hashing power.
It's appealing to have a lot of full nodes backing things up, but in practical terms I don't think it makes much difference if there is 1,000 or 100,000 full nodes. The important bit is for anyone to be able to trust the network not just to waste as much computing power as possible. And for that you can verify some random collection of transactions with less than full bandwidth which lets you verify the full node you're supporting.
PS: You could also set things up with shards, but again I don't feel it's nessisarily worth it.
That's a pretty big leap - that's one of the biggest problems that Satoshi solved in the original white paper.
We have yet to see Schrodinger's blockchain, and all of the altcoins claim they are it.
That is one way to view at bitcoin now, a very pessimistic one.
But an alternative positive view is that Bitcoin's network has proved its success at being able to process transaction up to the rate it was designed to handle (1 MB worth of transaction data per ~10min) without failing even in the face of sustained overcapacity barrage of transactions. No bugs since late 2010. No crashes. No successful 51% or other attacks. No double-spent coins. No invalidated transactions.
Those who have truly drunk the koolaid call this a feature not a bug.
By that logic, I could say: "Those who have truly drunk the koolaid call the high price of gold a feature not a bug."
Turns out much of the world revolves around drinking some kinda koolaid.
Not sure if this is a feature or a bug?