Two days ago "CoinWallet.eu" announced that they were going to do an "Ultimate" Bitcoin stress test, flooding the network with transactions. It looks like they are in fact going ahead with that.
Specifically they claimed they would spend 20BTC - about $5kUSD - to create transactions with a fee of 0.1mBTC/KB. That works out to 200MB worth of transactions, which will take some time to be mined.
Transaction fees are a supply and demand marketplace. On one side you have the supply - up to 1MB worth of transactions approximately every 10 minutes - and on the other side you have demand - what fee/KB people are willing to bid to get their transactions mined. With few exceptions essentially all miners use the profit-maximizing algorithm in Bitcoin Core to choose transactions: start with the highest fee/KB transactions, working down to the lowest, stopping when you're out of transactions or the block is full. Like any market place if you don't pay a competitive fee your transaction simply won't get mined until demand subsides. 20BTC of fees is a lot of demand, so that might take a few days, or even longer! Like any market place, if demand is sufficiently high prices may never go down, similar to how it might be years, if ever, before you'll be able to buy 1BTC for $1 again.
What's important is that by paying a competitive fee that outbids other transactions your transactions go to the head of the queue and will get mined immediately, much like how if you want to buy 1BTC right now, if you offer about $250 you'll get a buyer on an exchange pretty much immediately. Concretely, the fee rate CoinWallet.eu claims they will use is such that an average Bitcoin transaction - 250 to 1000 bytes - needs to spend at least 0.6 to 2.5 cents in fees to outbid the tx flood. For most transactions this isn't a big deal, although extremely low-value transactions will be uneconomical.
In a perfect world just like any other financial market getting a quote on how much a transaction will cost would be easy. Similar to when you buy bitcoins themselves data on prices would be readily available, such as estimated spot prices, as well as graphs of pending transactions and what fees they're bidding. Because blocks are found in a random Poisson process it's a little different from a standard market place - orders aren't filled instantly - but the basic principles are pretty similar.
When you send a transaction your wallet would have a threshold value for what level of fee you don't care about - "if the fee costs less than 10 cents, don't ask me for confirmation, just send it". In cases where getting into the next block would cost more than your threshold setting, your wallet would let you know and give you an estimate for how long you might have to wait at various fee levels; because blocks are created randomly, fees have a certain probability of going down due to a bunch of blocks getting created in a row. Equally, sometimes your original bid might be too low, perhaps because of an unexpected increase in demand, or miners having bad luck at finding blocks. In that case you'd have the option of resending your transaction with a higher fee to get back to the front of the queue.
Not very well. Even really basic things aren't done right. For instance even though the fee/KB is the basis by which all miners evaluate what transactions to mine first, I don't know of any block explorers that show the fee/KB for you - you have to calculate it yourself. Similarly, even though not all transactions are the same size most wallet software sets fixed fees, resulting in inconsistent fee/KB rates paid. Even worse, a lot of wallets give you few options with regard to what fee to pay, or even no options at all.
This is not unlike having a self-driving car that could fill up at the gas station all by itself, yet regardless of how empty the tank was or what the current state of middle east politics was, always tried to pay the same amount of money. If the gas station was unwilling to accept your price the car would just sit there for a few days or weeks until middle east politics improved, refusing to even let you get out your wallet and pay a few extra dollars.
As for the information side of things, even basic stuff like websites with some charts of the minimum and average fees paid by transactions in blocks pretty much don't exist right now. There's lots of pretty graphs available of things like the number of transactions per block, or total transaction fees paid, but none of this stuff tells you what you actually care about: What fee/KB do I need to pay to get my transaction mined?
Not only does our fancy self-driving car refuse to let us pay for gas, but no-one will tell you how much it actually costs anyway. Statistics for how many barrels of oil per day are consumed and how many cuddly marmots in wildlife refuges were killed extracting it are readily available, but the data you actually care about, how much did people pay for the stuff, just isn't available. I'd go on and talk about how the abiogenic petroleum crowd is going around trying to convince us that we're all floating on an endless sea of underground hydrocarbons and gas should be free to all... but that's enough for now.
As of v0.10.0 Bitcoin Core estimates fees for you based on the supply and demand observed on the network. By default it tries to pay a sufficiently large fee to get into the next block, so as demand increases it pays higher fees to compensate.
As of v0.93.2 released June 10th 2015 by default Armory uses Bitcoin Core's estimatefees RPC to estimate fees. As with Bitcoin Core, your transactions should go through with no issues.
Fees are set for you based on floating estimates, within a pre-defined sanity range. As with Bitcoin Core, your transactions should go through with no issues.
Fees are paid automatically for you based on floating estimates. They don't give any details on how this works, other than saying that "Coinkite will calculate the optimum fee for you and we even handle the payment of that fee, to ensure timely processing of all your transactions."
All these wallets, and many others, at least allow you to set fees in some way, e.g. via a slider with some pre-defined "economic/normal/priority" settings, or by entering exactly what fee you want. The simplest thing to do for now is set a fee 0.2mBTC/KB or higher to be sure to outbid the CoinWallet flood.
If your initial bid for blockchain space was too low and you're transaction isn't getting mined - in other words getting outbid - unfortunately there's no easy answers right now. No wallets currently support an explicit way to increase a transaction fee after the fact; you can try to make use of Luke-Jr's child-pays-for-parent patch - supported by a number of miners - to increase the transactions fee by respending the output, but that's usually tricky as most wallets don't give you that kind of control.
tl;dr: Until wallets improve you're probably going to just have to wait. :(
If you're feeling really adventurous and are using Bitcoin Core you can make use of first-seen-safe replace-by-fee, which F2Pool recently enabled. (credit goes to F2Pool themselves, as well as Marshall Long of FinalHash his big role in convincing them to adopt it) To do this you'll need to compile and run a Bitcoin Core node with the RBF patch. The patch automatically finds other RBF nodes and connects to them in addition to normal Bitcoin Core nodes; you can verify this by running the RPC command:
bitcoin-cli getpeerinfo | grep 0000000004000001
If this returns a bunch of lines like the following, you're connected to other RBF nodes:
"services" : "0000000004000001",
Next get my replace-by-fee-tools and use the bump-fee.py script to actually increase the fee on your transaction. It has two modes: full-RBF and first-seen-safe RBF. F2Pool supports the latter, so we'll be using the -s switch to enable the first-seen-safe rules:
$ ./bump-fee.py -vs cc056c60beee1a9bc05fcc0537cedf1755ce4b0cfb54c5a166aafc24a35ac3d8
DEBUG:root:First-seen-safe enabled: will not reduce change txout value below 0.01127058 BTC
DEBUG:root:Old size: 0.521 KB, Old fees: 0.00000522, 0.00001001 BTC/KB, Desired fees: 0.00010019 BTC/KB
DEBUG:root:Delta fee: 0.00004698
DEBUG:root:Adding new input 8e172a2f46a4ba99b957e523c398233a357ea997ff5880db89d2a7726584ff3a:0 with value 0.00301824 BTC
DEBUG:root:Delta fee: 0.00006171
DEBUG:root:New size: 0.668 KB, New fees: 0.00006693, 0.00010019 BTC/KB
DEBUG:root:Sending tx <snip>
da3ccefb7b9dd1cd9a157cd7e210ead76c5f0bd48fe175bc1be00937e4ae9ab8
Basically we took transaction cc056c60, added an extra input to it, and increased the value of the change address such that the new fee/KB was 10x greater, producing a second transaction da3ccefb. RBF supporting nodes see that tx2 pays more fees than tx1, so they relay it across the network. When it gets to RBF supporting miners they accept it into their mempools too, and with any luck the extra fees are enough to get bumped to the head of the queue.
That said, be warned: ./bump-fee.py hasn't been tested much, so don't use it on a wallet whose funds you can't afford to lose.
Transaction fees aren't going away, blocksize increase or not. CoinWallet.eu is only spending $5k flooding the network; even an 8MB blocksize increase can only raise the cost of that attack to $40k, which is still very affordable. For instance an attacker looking to manipulate the Bitcoin price could probably afford to spend $40k doing it with the right trading strategy; let alone governments, banks, big businesses, criminal enterprises, etc. to whom $40k is chump-change. Wallets need to become smarter about fees, as does the rest of the Bitcoin community.
What we need to do:
-
Add fee/KB displays to block explorers.
-
Change wallets to calculate and set fees in fee/KB rather than fixed fees regardless of tx size.
-
Make websites with easy to understand displays of what the current mempool backlog is, and what fee/KB is needed to get to the front of the queue. We've done a great job for Bitcoin price charts, let's extend that to transaction fees.
-
Add the ability to set any fee/KB to wallets, rather than be stuck with predefined options that may not be high enough.
-
Add support for fee-bumping via (FSS)-RBF to wallets and Bitcoin Core.
Capacity limits are just a fact of life in the design of the Bitcoin protocol, but that doesn't mean we can't give users the tools to deal with them intelligently.
This was /really/ useful.
Thank you very much!