[PITCH] Improvements to LLVM Decision Making

LLVMDecisionMaking.md

Introduction

The LLVM project is a pretty high functioning collection of individuals spanning many countries, time zones, and who are members of many organizations. It is also an established community, but one that needs to evolve and change to stay vibrant. This combination makes decision making both incredibly important, but also sometimes difficult to finalize, and very ambiguous/frustrating for contributors.

One note: While there are challenges with patch review and code owners, this proposal focuses on non-technical decisions that do not have a clear "code owner" escalation path today. This can include things like the introduction of new subprojects, introduction of a new social policies, change to core infrastructure like bug review tools or patch review processes, changes to the LLVM Developer Policy, etc.

Motivation

Decision making is incredibly important for a large and distributed community, and the challenges we face are not unique. These challenges manifest in many different ways, including for example:

• It isn't clear how to propose some changes in the first place, and it is often unclear who the decision makers are.
• It isn't clear what mailing lists to send such proposals to: llvm-dev gets a lot of traffic and many people don’t read it.
• It is hard to know whether something is a serious proposal that you must take seriously, and so it is easy to miss important changes because you don't have time to review everything. Even though you technically had a chance to participate, you can end up surprised when some change goes into effect.
• Sometimes people chime in late with dissent after a decision has been apparently made: this can be frustrating to people who need a decision made, because they aren't sure how to proceed.
• Sometimes people express a loud voice on discussion threads even if they aren't active contributors, and they can derail discussions. There is no "moderator" for these discussions.
• The initial discussion phase of a proposal can have lots of back and forth discussions to shape a idea, and the eventual proposal can have significant changes from that initial review. It can be difficult to discern what feedback from the initial review is addressed or ignored in the final rounds of discussions.
• Complex changes (e.g. the relicensing project) sometimes take many rounds of iteration, and it can be easy to lose track of where the proposal is and what the history was.
• Occasionally, Code Owners may be faced with a technical decision and not be sure what to do, particularly for highly impactful design decisions - e.g. for core LLVM IR changes. It could be useful to have a formal escalation process for these decisions.

Despite the challenges, it is important to note that many changes are smooth and non-contentious: the LLVM project has many like-minded individuals - we should not add overhead to things that are currently working well. This should be an optional process used by things that are contentious: we shouldn't force every small or unanimous thing into it.

Proposed Solution

I recommend that we add a process similar to (but adapted and changed where it makes sense) the Swift Evolution process. This process was designed to help guide decision making for high impact language and standard library changes to the Swift language. It was inspired by similar processes in other language communities (e.g. the Rust Language RFC process, the Python PEP process, etc) and has worked well in that context - it stands to reason that a variant should work well for LLVM as well.

The solution entails several parts. First, the process should be written down! This written process should include things like:

• An informal "pitch" phase to help collect requirements and shape the ultimate form of an idea, but which can be ignored by people who aren't interested in following all of the details of a discussion.
• A new mailing list (or Discourse channel) dedicated to formal proposal review. This would be relatively low volume, allowing effectively everyone to follow it. This could be named something like "llvm-proposals".
• A written proposal template, which provides guidelines for how to write a proposal. This proposal is written in an example template we can use, and the template can evolve over time.
• A new directory on Github that serves as the archive for official proposals.
• A review manager who helps shepherd the discussion in official rounds of review (which are time bound, e.g. to a week or two).
• A decision making body that evaluates the results, asking for additional discussions (with advice for how to change the proposal in the next round), and that ultimately approves or denies a proposal. This body should aim to get the community to consensus whenever possible, but can help split decisions in the case of extreme ambiguity when overwise all hope is lost. Denied proposals can of course be re-run if a pertinent situation changes or when revised.

It isn't clear who the best ultimate decision making body is. Here are some options:

• Use the LLVM Foundation Board, which is already tasked with making non-technical and policy decisions, and already makes final decisions on the addition of subprojects. Note that this would be an increase in responsibility if this process is used as an escalation process for technical decisions.
• We could create a new organization of elected or appointed members. One concern here is that this would be another contentious thing to introduce into the LLVM community and it is difficult enough to get community service and engagement with the existing structures we have.
• We could use a different decision making group for each decision, with the concern that this increases the cost of decision making.
• There are certainly other options (suggestions welcome!).

Detailed Design

None yet, TBD based on thoughts and feedback!

Alternatives considered

The most obvious alternative is to do nothing: LLVM has survived for 16 years as an open source project without a formal policy, and we aren't doing too bad. On the other hand, there is a reasonable argument that as the community continues to grow that the problem will get even worse over time.

A second alternative is to go with a BDFL model where (e.g.) Chris has to make all the decisions. This has several problems:

• While Chris would like to participate, he prefers to draw on the wisdom and judgement of other people as well.
• Chris doesn't know all areas of the LLVM project anymore.
• Chris could get hit by a bus some day (to be clear, definitely not "plan A"!) and we/you would have to determine a replacement process at that time.
• Chris overall isn't a fan of this model, and it is unlikely that the community would accept anyone else in such a role.