Bitcoin Core vulnerability disclosure policy discussions report

report.md

Context

Back in september 2023, during the Coredev in the Azores, we (the group of Bitcoin Core contributors present) discussed we (as a project) should do a better job of tracking and disclosing security-critical bugs. A bunch of us volunteered to dig up old reports (disclosed or not), set up some private infra to track vulnerabilities and come up with a disclosure policy. The group is comprised of Ava Chow, Michael Ford (fanquake), Niklas Gögge, Antoine Poinsot, Anthony Towns and Pieter Wuille.

In march 2024, during the Coredev in Berlin, this group had another discussion regarding the disclosure policy involving other long-time contributors. It was decided to go through all the past reports which were dug up to inform our disclosure policy. "If bug X was reported today, how should we treat it?". Following this the group had a couple of calls to decide on a disclosure policy.

Disclosure Policy

When reported, a vulnerability would be assigned a severity category. We differentiate 4 classes of vulnerabilities:

• Low: bugs which are hard to exploit or have a low impact. For instance a wallet bug which requires access to the victim's machine.
• Medium: bugs with limited impact. For instance a local network remote crash.
• High: bugs with significant impact. For instance a remote crash, or a local network RCE.
• Critical: bugs which threaten the whole network's integrity. For instance an inflation or coin theft bug.

Critial bugs would most likely require an ad-hoc procedure. As they are very rare and it's hard to come up with a standard policy for those in advance, we do not consider them here. Instead we focus on a standard policy for the large majority of the reports, which range from low to high severity.

A bug may also not get into any category at all. A reported issue may be considered seriously yet not be classified as a vulnerability because it does not require secrecy.

Low severity bugs will be disclosed 2 weeks after a fix is released. A pre-announcement would be made at the same time as the release.

Medium and high severity bugs will be disclosed 2 weeks after the last affected release goes EOL. This is approximately a year and a half after it was first released. A per-announcement would be made 2 weeks earlier.

Plan moving forward

Announcement and request for feedback

1. Share with the wider groupe of Bitcoin Core contributors.
2. Announce the policy to the wider Bitcoin developer community through the bitcoin-dev mailing list. Request feedback. We are not asking people to tell us how they think we should handle this, we are asking if this would be harmful to anybody for objective reasons we may have overlooked.

• Example of valid feedback: "I'm in charge of mining infrastructure. We are occasionally running a couple versions of Bitcoin Core behind for reason X. Given our past experience i believe it's possible for a business like ours to not be capable of upgrading within 2 weeks of the version we are running going EOL for reasons Y and Z. This is unlikely, still i think a pre-announcement period of 1 month instead would make sure to err on the safe side."
• Example of invalid feedback: "I just Google'd 'CVE best practices' and here is how you should do things instead or else it's an attack on Bitcoin" (except passed through ChatGPT to get to 10k words).

3. Try to communicate it to a wider audience. Ping the Optech guys, etc..

Gradual phase-in

We want to publish historical reports. We also want to gradually start enforcing the policy, to let the chance to everyone who didn't to upgrade. To this effect we are going to incrementally disclose security bugs which were fixed in past releases, up to the point where the bugs for all EOL releases will have been disclosed.

We'll start this month (june) with historical bugs: all which were fixed in 0.21.x and below (we have undisclosed bugs for as old as 0.11). Barring any good reason to change course (through the feedback received from users) we would continue in July for bugs fixed in 22.x. In August for bugs fixed in 23.x. Etc..

With regards to the (pre)announcements: is there a view on how this would be done already? I think it's important for users to know 1) which channel(s) they can monitor, and 2) steps they can take to increase confidence that the announcements are actually coming from the security list? Sharing that information before doing any preannouncements seems useful.

Medium and high severity bugs will be disclosed 2 weeks after the last affected release goes EOL. This is approximately a year and a half after it was first released. A per-announcement would be made 2 weeks earlier.

This seems a bit aggressive. Obviously, for the majority of users this likely will not be a problem. However, a small tail of users may sometimes need more time to upgrade, for a variety of reasons. Giving more time after the pre-announcement before disclosure (and after the last affected release is EOL) seems like a good way to reduce the risk for a small tail of users with no major downside? I know that "high" bugs are only announced after EOL, but a honest mistake by someone to forget to do an upgrade shouldn't put them in a sudden rush to complete it in two week, or be exposed to malicious actors or "lulz" actors.

A two week pre-announcement could easily not make it to the responsible person in time. For example, the person could be on holiday, work travel, or otherwise be temporarily occupied with work or non-work tasks.

Moreover, an upgrade process may be more involved occasionally and could very well take more than two weeks to complete. For example, one could set -rpcdeprecated in a release to avoid a breaking change an then forget about fixing their code before the release goes EOL in a honest mistake. They receive the pre-announcement, but they realize it is infeasible to fix their code (or external dependency) in two weeks, review it, and upgrade to it. Another reason could be that they are compiling from source on an old Linux distro, but a suitable compiler is lacking to compile the new release. So they may have to similarly upgrade their dependency tree within two weeks, which may or may not be a problem for them.

I am not saying that such complications are common, but they seem plausible and the additional cost of waiting a bit longer should be minimal? That is, a policy of [medium bug -> pre-announcement + 1 month, and high bug -> pre-announcement + 6 months] or similar shouldn't cause any issues? I presume the announcement will be written at the same time as the fix is written (or latest when the pre-announcement is written), so the only difference for the disclosure people would be to set a different timer, which should not be an issue?

I worry that a pre-announcement, especially a pre-announcement long before the expected disclosure, is an advertisement for people to check a diff to see if they can find the patched vulnerability themselves. If that happens, they may find and disclose the vulnerability themselves before the end of the pre-announcement period, eliminating the expected benefit of it. That's happened before (example). Worse, because they don't know what vulnerability was going to be disclosed, they may discover and disclose a different vulnerability that the project wasn't ready to disclose yet.

I recommend the following:

• Low-risk vulnerabilities: no pre-announcement. Announce them two weeks after release.
• Medium/High-risk vulnerabilities: in every release notes, include the following template near the top: "Warning: with this release, versions XXX.* are now end-of-life. You must upgrade to YYY.* or above. If there are any significant vulnerabilities affecting only end-of-life versions, they will be announced on YYYY-MM-DD." Include this template even if there are no vulnerabilities planned to be disclosed, so there's never any special encouragement to check diffs.

I think we should check to have in place PGP signing of any disclosure report (at least for >= medium vulnerabilities). We already considered in the past how disclosure of security-critical bugs could have induced side-effect such as mempools congestion. One would wish to be careful on the announcement social effect, this not becoming a source of safety concerns on its own (e.g quid unauthenticated report leaking before agreed-on full disclosure date ?).

For all the vulnerabilities tracking, it could be great to have in place a native numbering system (e.g previous block hash + sha-256 of the original report, the whole committed in an op_return / pay-to-contract). That way vulnerabilities issue can be identified ahead of disclosure, if there is need for more mitigation coordination development and they can be easily tracked by softwares users / infrastructure operators after the full disclosure.

I think ideally we would avoid pre-announcement, as most of the time it's quite easy to reverse-engineer the fixes from the releases diff and other heuristic to know what has been concretely mitigated. This did happen in the lightning-side once with CVE-2019-12998 and sisters. That said, I don't think we can completely avoid pre-announcement if the vulnerability has been reported to the core security team by an external actor, and the mitigation / disclosure timeline is constrained by such actor. Under that scenario, pre-announcement can be better than nothing to fasten dissemination of mitigations across the ecosystem.

Finally, june to start disclosing old vulnerabilities as old as 0.11 sounds very spicy. There are many high market-caps cryptocurrencies codebases sharing bitcoin core components. Whatever one can think about the "moral value" of other cryptocurrencies, let's remember that the inflation bug was initially reported by a security researcher not belonging to the usual set of core folks. Sparkling with a too much aggressive disclosure policy could fire back on bitcoin core, less the technical security of the codebase than the reputation of the project.

(This response is my opinion, not speaking for anyone else)

This seems a bit aggressive.

I think it is important to consider the bug classes we are talking about here. Medium - High severity does not include bugs that affect the integrity of the network as a whole (i.e. they are not chain-splitting, supply inflating or coin theft bugs). If exploited, the impact would be localized to the users running the affected EOL versions.

Users are in charge of choosing the version/software they run. Them choosing not to upgrade to maintained versions is not our responsibility, just like pushing new updates to them directly (auto updates) isn't either. Users should upgrade to a new version before the one they are running goes EOL. Choosing to ignore this advice comes with trade-offs (e.g. you may have to scramble when bugs are made public). EOL dates are known well in advance, so really it's not "just 2 weeks" but rather months and the pre-announcement simply acts as a reminder.

Our timelines are already incredibly conservative. It will take ~12-18 months + 2 weeks from the time the bug is found to it being publicly disclosed. Software bugs affecting billions of devices (e.g. browser or mobile OS RCEs) and not even critical hardware bugs (e.g. spectre and meltdown) have this long of a grace period.

Thanks for all the comments.

@stickies-v i think the pre-announcements would be made through the bitcoin-dev mailing list at the same time as the announce of a release. Since announcing release N is effectively announcing release N-3 is now EOL. I expect when releasing version N we would now also state "Vulnerabilities affecting the previous version, if any, would be disclosed in approximately one year. Please upgrade as soon as you can.". Regarding the authentication of announcements, disclosures could be PGP signed. Pre-announcements could be too, but i'm not sure it's worth it.

@maflcko this is my personal opinion as well. That said, i believe this policy is better than no policy.

@harding it may be possible to figure out some covered fixes from the diff between two versions, but i don't expect it to be trivial in most cases for any realistic threat model. That said, we already intend to have the template you recommend. Then the pre-announcement period is just a reminder which in the worst case would reduce the disclosure period by 2 weeks.

@ariard i'm not sure how the first section is relevant for the disclosure policy. A security vulnerability report leak is already possible. Regarding the second section, i don't think using a convoluted numbering scheme in place of CVE numbers would be pertinent. Regarding reverse-engineering the fixes from the diff between two releases, as i mentioned above i don't think it's that trivial in most cases. CVE-2019-12998 is also not a good illustration of how Bitcoin Core usually does covered fixes. Regarding fork coins it's always a complicate decision. But for disclosing historical bugs i don't think this is anywhere near being aggressive.

@dergoegge

In a world where users are generally running application softwares on top of bitcoin core, severity impact becomes hard to evaluate in itself. Turns out most of bug types that are proposed to be classified as medium - high severity can easily escalate as coin thefts bugs on upper layer. E.g a medium local network remote crash of a full node can easy be leveraged to steal lightning channel funds.

Beyond, I believe your statement is uncorrect in saying that EOL dates are known well in advance. From a read of "Lifecycle" on Bitcoin Core website, there is still not displayed EOL dates for 25.0, 26.0, 27.0, neither there was an EOL date announced with 27.0 release email. In my opinion, EOL date should be available to users at code or tarballs download like done for linux kernels. (edited: corrected english and added link).

This is correct that ~12-18 months is what generally observed as an embargo period for browser and mobile OS RCEs, sometimes for side-channels attacks cycles can take more times as chips vendors have to develop and release micro-code before kernel devs can actually write their own mitigations. Beyond, this is ignoring the fact that chromium or ios / major android flavors have dedicated distribution channels, where most of the time they can "push" upgrades to their end-users.

About bitcoin, you certainly wish that the software definition of your money has not changed between two upgrades.

@darosior

About PGP signing of disclosure report, this lets you identify who has access to information-sensitive report and in case of any ulterior leak investigate the root causes. There are reasons why linux are pgp-encrypting as much as they can their security-issues coordination communications. I don't know if it it’s something to be done, it can certainly be put in place in the future.

On maintaining a custom numbering scheme, it turns out in my experience (a) the current CVE scheme which is based on usual OWASP top threats doesn't fit well bitcoin types of bug (e.g chain splitting) (b) the scheme doesn't work well when it's protocol-level issues, not affecting a specific codebase and (c) sometimes the CVE assignment authority can be slow in the weeks-span to assign CVEs. That said, it's more something that one can experiment with as a security researcher to watermark its own reports, I don't think there is a strong need to have this formalized in the core-specific disclosure policy.

On reverse-engineering core diffs, yes it all depends with the skills and experience of the security analyst you're talking to. I've seen lighting covert fixes well-done and I've seen bitcoin core covert fixes done in a ugly fashion. Never an easy art. The point being it might be better to go without pre-announcement by default, and then adjust on ad hoc basis if there is a need to do a pre-announcement.

About fork coins, I remember @gmaxwell's position on not pre-disclosing vulnerabilities to other codebases of the same breed than core. Very often fork coins are in an adversarial position w.r.t bitcoin or rarely software robustness is an end in its own sake. My very personal opinion, they somehow succeed in the kernel space to coordinate side-channel vulnerabilities among ISA vendors / chips with adverse interests by laying out strong ethical rules.

In the present case, if the intention is to start disclose 0.11+ historical bugs by June of this year, I think this is not realistic as you need time for any ethical rule to be internalized by the stakeholders of a security-bugs handling process. My recommendation would still be to reach out to the top 10 by market cap of fork coins maintainers and give them few ahead of pre-disclosure by politeness. This is neither my responsibility nor reputation engaged here, so ultimately and a bit cynically I have little care about what is shared with fork coins.

@harding "I worry that a pre-announcement, especially a pre-announcement long before the expected disclosure, is an advertisement for people to check a diff to see if they can find the patched vulnerability themselves."

It's worse than that. This policy ensures that every commit will be scrutinized to see if it patches a vuln. Resulting in information asymmetry where attackers are aware of vulnerabilities before users are.

@dergoegge "Our timelines are already incredibly conservative. It will take ~12-18 months + 2 weeks from the time the bug is found to it being publicly disclosed. Software bugs affecting billions of devices (e.g. browser or mobile OS RCEs) and not even critical hardware bugs (e.g. spectre and meltdown) have this long of a grace period."

That just isn't true, Chrome for example publishes a CVE and announces every security vulnerability as they are patched. They may not disclose the exact details but they certainly announce it.

This policy ensures that every commit will be scrutinized to see if it patches a vuln.

Bitcoin Core is fully open source software, so anyone can at any time inspect each commit, with or without this policy.

@maflcko this is my personal opinion as well. That said, i believe this policy is better than no policy.

I agree. Though, if a longer time before announcement is picked, going with the approach harding proposed makes more sense, for the reasons given.

Looking at https://bitcoincore.org/en/lifecycle/#schedule right now, I think it could be clarified to replace in the table MAJOR.0 with MAJOR.x. Otherwise, users may incorrectly assume that MAJOR.0 is supported with security fixes if they only happen to look at the table and not the previous section? Moreover, it could explicitly be mentioned in the schedule section that only the last point release contains the latest security fixes?

Looking at https://bitcoincore.org/en/lifecycle/#schedule right now, I think it could be clarified to replace in the table MAJOR.0 with MAJOR.x.

bitcoin-core/bitcoincore.org#1026

@maflcko "Bitcoin Core is fully open source software, so anyone can at any time inspect each commit, with or without this policy"

Yes exactly, which is why all vendors, especially open source vendors, disclose vulns so that users are aware instead of just attackers.

@fanquake

Looking at https://bitcoincore.org/en/lifecycle/#schedule right now, I think it could be clarified to replace in the table MAJOR.0 with MAJOR.x.
bitcoin-core/bitcoincore.org#1026

Again:

Beyond, I believe your statement is uncorrect in saying that EOL dates are known well in advance. From a read of "Lifecycle" on Bitcoin Core website, there is still not displayed EOL dates for 25.0, 26.0, 27.0, neither there was an EOL date announced with 27.0 release email. In my opinion, EOL date should be available to users at code or tarballs download like done for linux kernels. (edited: corrected english and added link).

The real thing would be to have the TBA dates clearly announced at time of binaries release. That way if people are doing things like putting some bitcoin core components on hardware which has been discussed for signers on the lightning side, they can have deployment and upgrades policy accordingly.

Any vulnerability which requires a consensus rule change to fix should receive the critical handling described here, due to the coordination difficulties of deploying 'fixes', which will almost certainly be highly issue specific, and also more vulnerable to impact misestimation. Alternatively, "a fix is released" might be changed to "fix is activated on the network" for issues fixed via consensus changes.

The specific concern I had there is that there can be (and have been) consensus issues that by impact might be low or medium, consider for example "I can create a tree fragment that allows a false SPV proof with 2^96 work due to a lack of interior node vs leaf", perhaps the difficult of exploiting means its low... but after a month of analysis it turns out that its actually possible to do it in 2^48 work, which would be immediately exploitable, and any consensus change will take much longer to activate than weeks from release.

Relatedly, resource wasting attacks in consensus rules might be low/medium from impact (just making slow to process blocks) but the long term impact of the fix is greatly reduced if the blockchain contains no examples of triggers ... as the codebase won't need to contain a conditional "this rules before this point, those rules after". So, again, it's extremely valuable to not have disclosed them until after they won't get mined.

I see I'm mentioned above. I think it's important to also recognize that policy itself is potentially a source of vulnerability, because it's an invitation to bad actors to demand the project behave against its best judgement in order to act predictably in the attackers interest. This realization favors a conservative approach which provides ample opportunity to do the right thing and to ignore outside actors which are potentially acting maliciously. It's always fine to have norms and traditions that go above and beyond the rules set by a policy but aren't precluded from it, the policy is essentially the minimum which you can expect that malicious actors are permitted to demand from your process. In other words "the project will do at least this, but may also do other things". Because of this the policy doesn't need to and shouldn't encompass every consideration.

When it comes to alternatives, the comment of "laying out strong ethical rules" made me laugh out loud. In some cases we're just talking about outright criminal fraudsters who have no issue with threatening to murder bitcoin developers and the families or whom will drag bitcoin developers through court for years and cause millions of dollars in costs because they hope to harass them and gum up bitcoin. No amount of "strong ethical rules" is going to help in those cases.

Historically, the position of the project has been that the the project and the effort of its contributors exist for the benefit of the bitcoin system. If someone wants to go create a competing system using the code the licensing permits that, but the good of bitcoin ought not be meaningfully compromised for their benefit. Because of this they're generally not informed of issues before fixes are deployed in Bitcoin except to the extent that the general public is informed. That said, it's kind and professional to not gratuitously screw them over (even when they wouldn't act the same). Unfortunately third parties can't all be trusted to not attack each other too, so it's difficult to have a formalized practice of disclosing to them prior to making issues public. Many also just don't apply fixes even if informed (as its not on the critical path of separating fools and their money or just because they're inactive or under-resourced). In the past this conflict was often addressed by just not disclosing things (or by waiting until other things had managed to pick up fixes accidentally) but this also has resulted in former issues just being forgotten and is clearly also not in the best interest of Bitcoin.

In light of that, I think the best you could say is that sometimes notice will be provided to other projects if its reasonable to do so... but that's always going to be a judgement call specific to the nature of the issue and the other projects.

@gmaxwell

I see I'm mentioned above. I think it's important to also recognize that policy itself is potentially a source of vulnerability, because it's an invitation to bad actors to demand the project behave against its best judgement in order to act predictably in the attackers interest. This realization favors a conservative approach which provides ample opportunity to do the right thing and to ignore outside actors which are potentially acting maliciously. It's always fine to have norms and traditions that go above and beyond the rules set by a policy but aren't precluded from it, the policy is essentially the minimum which you can expect that malicious actors are permitted to demand from your process. In other words "the project will do at least this, but may also do other things". Because of this the policy doesn't need to and shouldn't encompass every consideration.

When it comes to alternatives, the comment of "laying out strong ethical rules" made me laugh out loud. In some cases we're just talking about outright criminal fraudsters who have no issue with threatening to murder bitcoin developers and the families or whom will drag bitcoin developers through court for years and cause millions of dollars in costs because they hope to harass them and gum up bitcoin. No amount of "strong ethical rules" is going to help in those cases.

Historically, the position of the project has been that the the project and the effort of its contributors exist for the benefit of the bitcoin system. If someone wants to go create a competing system using the code the licensing permits that, but the good of bitcoin ought not be meaningfully compromised for their benefit. Because of this they're generally not informed of issues before fixes are deployed in Bitcoin except to the extent that the general public is informed. That said, it's kind and professional to not gratuitously screw them over (even when they wouldn't act the same). Unfortunately third parties can't all be trusted to not attack each other too, so it's difficult to have a formalized practice of disclosing to them prior to making issues public. Many also just don't apply fixes even if informed (as its not on the critical path of separating fools and their money or just because they're inactive or under-resourced). In the past this conflict was often addressed by just not disclosing things (or by waiting until other things had managed to pick up fixes accidentally) but this also has resulted in former issues just being forgotten and is clearly also not in the best interest of Bitcoin.

I think the mention of "laying out strong ethical rules" should have more context added as it was directly based on the image of what is done in the operating systems ecosystem for any security issue related to hardware e.g such as meltdown, specter or any other low-level micro-architectural screw up. The linux kernel has a formalized "memorandum of understanding" which can be a good example when security-sensitive information have to be embargoed for a while among groups maintaining different codebases or hardware vendors wih a wide range of economic incentives.

I can fully understand all the situations when some old fork coins are maintained by outright criminal fraudsters shamlessly engaging in death threats to bitcoin developers and their families and you can be assured that it was certainly not the type of interculators that meet the ethical threshold I have mind that is worthy to give notice. This is certainly not an always pleasant situation to receive death threats due to your open-source bitcoin contributions. Yet in my past experience of what did arrive to a good bitcoin friend of mine it's still more chill than the risk of being summarily shot in the head by special military forces during the early days of a full scale military invasion due to almost decade-old political opinions (— And that said if you’re attacked for your open-source bitcoin works an offensive reply can be always weighed on).

I also fully understand all the trade-offs when security-sensitive information is commonly shared between open-source softwares groups with different levels of contributors ressources and the not so perfect situation of waiting the laggards to have patched their stuff before a security issue can be the object of a full disclosure. From my experience with lightning, responsible coordination is indeed more an art influenced by norms and tradition than what can be fully fleshed out in a formalized policy. Yet, I think it would be good for the current group of bitcoin core maintainers to not have a monolithic view of the best interest of bitcoin, both based on the inflation bug experience (afterall awemany didn't belong to the usual group of contributors) or in face of changing landscape both downstream and upstream.

Upstream, where coordination with projects like lightning and other off-chain stuff supporting the substantial on-chain fee traffic could have a fireback effect on stability of the main chain if mishandled (especially in a post-subsidy wolrd). Downstream, if we’re seeing security issue at the internet stack level that could massive impact on the stability of the bitcoin blockchain, e.g an unauthenticated remote code execution in ssh.

I don't disagree with the conclusion that noticing shall be provided to toher projects if it's reasonable to do so and that it is always going to be a judgement specific due to the nature of the issue and the other project. I was just under the impression than the current people handling the establishment of this policy and advocating for short-pace timeline in matters of full disclosures of old issues were acting as "spoiled childs" which would have inherit an old stock of bitcoin security-sensitive issues, without getting the scarces of experience to act with the same level of ethical care than their elders.