Robot icon pop answers level 4 tv and film part 2
23 comments
Hsgac bitcoin stock
I'll be talking about some recent advances in block propagation and why this stuff is important. I am going to talk about how the original bitcoin p2p protocol works, and it doesn't work that wy for the most part anymore. I will tlak about the history of this, including fast block relay protocol, block network coding, and then building up to where the state of the art is today. There are two things that people talk about when thy are talking about the resource usage of block propagation.
One is block propagation cost to the node operators. They use bandwidth and CPU power to propagate blocks. This is obviously a factor. The original bitcoin protocol was particularly inefficient with transmissions of blocks.
You would operate as a node on the network and receive transactions that come in and then when a block was found on the network you would receive the block which included the transactions that you already had received.
This was a doubling of the amount of data that needed to be spent. But not a doubling of the node's overall bandwidth usage, because it turns out that nodes do things other than just relay blocks, and these other things are even less efficient than block propagation. In particular, this process that bitcoin uses to relay transactions is really inefficient. The standard bitcoin protocol method of relaying a transaction is to announce to the peers hey I know bitcoin txid "deadbeef" and then the peers respond to me with "please send me transaction with txid deadbeef" and then there's 38 bytes of INV transactions on the wire, and then a transaction is maybe bytes.
The other thing that is a problem with resource usage is that the old bitcoin block propagation mode is really bursty. You would use a steady amount of bandwidth as transactions come in, but when a block came in wyou would use a megabyte of bandwidth as soon as the blocks came in. Back at Mozilla, I could tell which of my colleagues were bitcoin users because during video chat their video would stall out in time with blocks appearing on the bitcoin network. So they would have to turn off their bitcoin node..
That behavior was a real usability factor in how comfortable it was to run a node on residential broadband. A few years ago there was noise on the internet about buffer bloat on routers having excessive latency, which has still not been fixed.
Big blocks all sent at once is basically the perfect storm for buffer bloat. So it makes your VOIP mess up. The bigger concern for block propagation is what is the latency for distributing a block all around the network, in particular to all of the hashpower? This is a concern for two major reasons. One is that if it takes a long time relative to the interval between blocks, then there will be more forks in the network, and confirmations are less useful.
It becomes a possibility that confirmations get reorged out. So this has an impact on bitcoin's interblock interval, currently 10 minutes, which is a nice and safe number which is far away from convergence failures. In order to lower this number, the block interval needs to be up to this challenge, so that's one reason that it's important.
The other reason is that block propagation time creates "progress". When you introduce delays into block propagation, mining works more like a race instead of a fair lottery. In a race, the fastest runner will always win unless the next fastest is very closest in speed. Mining is supposed to be like a lottery, propagation delay makes it more like a race, and the reason for this is that if I don't know about the latest blocks then my blocks won't extend them and I'm starting behind.
Also, if other miners don't know about my blocks then they won't mine on this, either. So why is "progress" in mining bad? As I was saying, there are incentives such that every participant should find a proportion of blocks equal to their hashrate proportion. There is a centralization pressure: You are more centralized, you can buy more hashpower and get even bigger. Miners have the opportunity to choose, to collaborate into a single pool. I'll talk more about this.
This is often misunderstood, when propagation comes up on reddit, there's some vaguely nationalist sentiments like "screw the Chinese with their slow connections" things that people say. I think that's wrong, because block propagation issue isn't better or worse for people with faster or slower connections, instead it's better or worse for people with more hashrate.
So if it were the case that people in China had very limited bandwidth, then they would gain from this problem, so as long as they had a lot of hashpower, which in fact they do. In general, this is a problem that we need to overkill. When we talk about how we spend resources in the community, there's many cases where we can half-ass a solution and that works just fine. It's true for a lot of things. But other things we need to solve well.
Then there's a final set of things that we need to nuke from orbit. The reason why I think that block propagation is something that we should overkill is because we can't directly observe its effects.
If block propagation is too slow, then it's causing mining centralization, and we won't necessarily see that happening. The problem adversely effects miners, so why won't they solve it?
Well, one reason is because bad propagation is actually good for larger miners, who also happen to be the ones in an economic position to work on the problem in the first place. I don't think that any large miner today or in the recent part has been intentionally exploiting this.
But there's an effect where you might be benefiting from this, and it's profitable for you, then you might not notice that you're doing it wrong, you're not going to go "hmm why am I making too much money? I mean, it takes a kind of weird person to do that. I am one of those weird people.
But for the most part, sensible people don't work like that, and that's an effect worth keeping in mind. Also, miner speciality isn't protocol development. If you ask them to solve this problem, then they are going to solve it by doing it the miner way-- the miner's tool is to get lots of electricity, lots of hardware, contracts, etc. There are tools in the miner toolbelt, but they are not protocol design. In the past, we saw miners centralize to pools. For example, if their pool had too high of an orphan rate, they would move to another larger pool with a lower orphan rate.
This is absolutely something that we have seen happen in the past, and I'll talk about what we did to help stop those problems. Another thing that miners have been doing is they sometimes extend chains while completely blind, so instead of waiting for the block to be propagated, they learn enough from another pool to get the header and they just extend it without validating, and they can mine sooner.
The bip66 soft-fork activation showed us that likely a majority of hashrate on the network was mining without verifying. Mining without verifying is benign so as long as nothing goes wrong. Unfortunately, SPV clients make a very strong security assumption that miners are validating and are economically incentivized to validate.
Unfortunately ythe miners incentives don't work out like that, because they think that blocks aren't invalid too often. If you are using an SPV client and making millions of dollars of transactions, it's possibly bad for you right? Bitcoin is designed to have a system of incentives and we expect participants to follow their incentives. However, this isn't a moral judgement.
We need to make it so that breaking the system isn't the most profitable thing to do, and improving block propagation is one of the ways we can do that. Why does it take a while for a block to propagate? There are many sources of block latency to relay a block through the network. Creating a block template to handout is something that we can improve in Bitcoin Core through better software engineering, not protool chanes. In ealrier versions it would take a few seconds.
Today, it's a few milliseconds, and that's through clever software optimization. The miner dispatches a block template to their miners, and that might be entire seconds in some places, and it's not in my remit to control- pehrpas mining manufacturers to do this. When you are purchasing equipment from mining hardware, you should ask the manufacturer for these numbers.
These can be fixed wiht better firmware. You need to distribute your solution to peers outside of your network. You have to send the data over the wire. There are protocol roundtrips, like INV and getdata. There are also TCP roundtrips, which are invisible to people, and people don't realize how slow this is. One of the things that is very important to keep in mind is that if you're going to send data all around th world, there will be high latency hops like from here to China will be more than a few dozen milliseconds.
And there are also block cascades from peer to peer to peer. Obviously, in the original bitcoin protocol, every time you relayed a block you would first validate it.
These more advaned propagation techniques can work without fully validating the blocks though. This is a visualization of the original bitcoin block relay protocol. Block comes in, there's a chunk of validation, the node sends an INV message saying hey I have this block, the other node responds with please give me this block, and the other node gives a megabyte of data.
This is really simple, and it has some efficiency in that you don't send a 1 MB lbock to a node multiple times. A node knows As I mentioned before, in addition to the roundtrip in this INV getdata block sequence, the underlying TCP protocol adds additional roundtrips in many cases. And you have to wait for validation.
One of the earliest things that people worked on, is that Pieter and Matt observed that bip37 filterblock messages could be used to transmit a block but eliminate transactions that were already sent. So we would skip sending the transactions that were already sent.
In , testing suggested that this slowed down transmission due to various overheads, and blocks were much smaller and it was a different network than today.
The filtering assumed you would remember all transactions you had been sent, even those that you had discarded, which is obviously impossible. This was the start of an idea- we got it implemented, tried it out, and this inspired more work to refined ideas. Around this time, we started to see high orphan rates in the network that contributed to pool consolidation.
There was a bit of a panic here among us, we were worried that the bitcoin network would catch fire and die.