| title | When AI Builds Your Community, AI Must Manage Your Triage |
|---|---|
| series | AI-Managed OSS: How to Scale the Communities That Agents Are Building |
| article | 1 |
| published_date | |
| medium_url | |
| linkedin_url | |
| gist_url |
I run triage twice a day. Every day. Morning and afternoon, sixty minutes each, across multiple terminal windows. I am not reading issues. I am operating skills.
In the morning session, one terminal runs a labeling skill against everything that arrived since yesterday afternoon: new issues, new PRs, updated threads. A second terminal runs a community response skill against the items the labeling pass flagged — the issue that needs more information, the PR that supersedes an existing approach, the contribution that is directionally right but technically incomplete. Each gets a precise, friendly, actionable comment. Not a template. A response calibrated to the specific state of that item in the context of everything else open in the tracker. The skill has read all 847 of them.
By the time my first meeting starts, the queue is current. Contributors have responses. Labels are applied. The items that need developer attention are surfaced and organized.
This is not a future state. This is how I operate the project today.
Triage feels like a solved problem until it isn't. In the early days of a project, a small team reads everything. Issues are a signal, not a burden. The maintainer who triages on Monday morning has context on everything that came in over the weekend. Nothing falls through.
Then the project grows. The issue rate doubles. Then doubles again. The reading takes two hours instead of thirty minutes. Context is lost. Contributors who filed three days ago have heard nothing and are starting to wonder if anyone is home.
Here is what most scaling guides miss: this is not a throughput problem. It is a memory and accuracy problem.
No human triager can hold 800 open issues in working memory and triage each new arrival with full awareness of what it relates to, what it supersedes, what it duplicates, and what developer queue it belongs in. A well-constructed skill can. It reads the new issue against the full context of the open tracker and produces a precise, consistent result every time — not "close enough," but accurate in a way a human context-switching across two hundred items a day cannot reliably be.
The symptoms of a triage wall are predictable:
- Growing queue depth. More issues open than last week, more than the week before.
- Declining response time. First-response latency climbs from hours to days to weeks.
- Label drift. Issues without labels, issues with inconsistent labels, issues that were labeled and then relabeled and then abandoned.
- Unknown dependencies. A growing web of related issues, superseded PRs, and competing solutions that no single person holds in their head.
- Community noise. Duplicate issues multiply because contributors cannot find the original. Frustration surfaces in issue comments.
The wall is not a sudden collapse. It is a slow degradation that is easy to rationalize ("we've just been busy") until the queue size becomes undeniable.
There is a metric that cuts through the noise better than queue size alone: the ratio of resolved issues to closed ones.
A resolved issue means root cause was found, a fix was shipped, or a design decision was made. A closed issue means the issue was disposed of: duplicate, stale, needs-more-info with no follow-up, out of scope. Both are legitimate. But the ratio tells you something about project health that raw volume hides.
On our project, a typical day looks like this: 52 issues closed, 16 resolved. That is a 31% resolution rate. The rest — 69% — are hygiene closures: duplicates, stale reports, items that do not meet the bar for action.
That 31% resolution rate is not necessarily bad. In a large, active project with years of accumulated backlog, hygiene closures are healthy. They reduce noise. They help contributors find the canonical issues. They are part of maintenance.
But if that ratio drops below 20% and stays there, you have a signal worth investigating. It may mean your triage criteria are too aggressive. It may mean your documentation is poor and contributors keep filing the same gaps. It may mean the project is outpacing its own capacity to resolve issues substantively.
Track this number weekly. Watch the trend, not the point-in-time value.
Here is what most OSS scaling guides miss: the throughput assumptions they were written against are obsolete since OpenClaw launched.
When contributors use AI coding assistants, their output rate increases. They ship more PRs per week. They file more issues — some thoughtful, some generated from a prompt with insufficient context. The community becomes more productive and more noisy simultaneously.
But the volume increase is only part of the story. AI tooling is changing the composition of who is filing issues. When the barrier to contribution drops — when a developer can generate a PR from a description or file an issue from a stack trace paste — the community expands to include contributors who are technically curious but not yet fluent in the conventions of open source contribution. They file genuine issues that are underspecified. They open PRs that are directionally correct but need guidance on test coverage, documentation, or API conventions.
This community is valuable. It represents the future contributors and advocates for your project. But it requires a different kind of triage response than a seasoned contributor filing a well-formed bug report, and it requires it at volume.
The answer is not more human maintainers. The answer is AI-managed triage running at the same scale and pace as AI-generated contribution. You are managing AI with AI. The projects that understand this early will build the communities that compound. The ones that do not will watch their queues grow until the community stops showing up.
Here is what the triage function actually looks like when it is operating at scale from the CLI.
The labeling skill runs first, on every triage pass. It reads each new issue and PR against the full context of your open tracker — not the item in isolation, but its relationship to everything already open. It proposes a precise label set: type, priority, component, status. The labels are not guesses. They come from a taxonomy you have defined in natural language, where each label is described specifically enough that the skill can apply it consistently at volume. You review proposals in batches in the terminal, approve what looks right, and correct what does not. The skill applies your corrections going forward. This step is the foundation. Everything downstream depends on it.
The community response skill runs in parallel. It works through the items the labeling pass flagged for action: the issue waiting two weeks with no response, the PR that needs rebasing after a merge to main, the contribution that duplicates an existing open issue, the two approaches competing to solve the same problem from different angles. For each item, the skill drafts a precise, friendly, actionable comment — not a template, but a response calibrated to the specific state of that item in the context of everything else open in the tracker. The author gets a response that feels considered, because it is. The skill read the full context before it wrote a word.
The relationship intelligence skill runs on a scheduled cadence and finds what individual contributors cannot see at volume: that a recent PR supersedes the approach in an issue and the original reporter has not been notified; that three issues filed this week are all symptoms of the same undocumented behavior; that a pending PR has a merge conflict with main that the author has not addressed. The output arrives in your terminal: items grouped by type, suggested actions. You review, approve, and move on.
The developer queue skill takes the labeled, triaged backlog and organizes it for the engineering team. It surfaces logical groupings — the cluster of bugs in the authentication layer, the feature requests that share an underlying implementation, the P0 item that arrived Monday without attention. It advises on what to pick up for the most impact: the highest-priority items with the clearest path to closure. It can also surface themes for daily releases and tags, identifying which resolved items belong together in a meaningful release note.
The velocity reporting skill runs at end of day and end of week. Open and close rates. Resolution ratio. Label-level trends: which categories are accumulating, which are clearing, where the community is concentrating. These are the signals that tell you whether your triage system is working and where to look next.
All of this is connected by the label taxonomy. The labeling skill's accuracy is the foundation on which everything else runs. If the labels are wrong, the developer queues are wrong. If the developer queues are wrong, the team is working on the wrong things.
The time to build triage infrastructure is before you need it, when you have bandwidth to build it thoughtfully rather than under duress.
Start with the label taxonomy. Do not write it by hand. Run a skill against your repository: point it at your code structure, your CODEOWNERS file, your team's areas of ownership. Describe your team structure in plain language. The skill generates a working taxonomy with each label defined precisely enough for downstream skills to apply it consistently. Review, adjust, and version-control the result. When community patterns shift, run the skill again to propose additions.
Then add skills in layers, in this order:
- The labeling skill. This is the foundation. Nothing downstream works without consistent labels.
- The community response skill. This is the highest-leverage investment for community trust — fast, precise, considered responses at any volume.
- The relationship intelligence skill. This is what prevents your tracker from becoming an undifferentiated mass of open items with hidden dependencies.
- The developer queue skill. This turns labeled issues into a prioritized backlog developers can pull from in the CLI.
- The velocity reporting skill. This tells you whether the system is working.
The signal that it is time to start building: when triage takes more than an hour per day for a single maintainer, and the project is growing. At that point, the wall is visible on the horizon. You have a few months to prepare.
The weekly triage review is thirty minutes in the CLI: run the velocity skill, review what it surfaces, approve or override where judgment is needed. You are not reading issues. You are reviewing a short list of decisions that required a human and closing the week's triage backlog. Everything else already ran.
Run a labeling skill against your open issue tracker today. In an hour, you will have a proposed label taxonomy and suggested labels for your entire backlog. Review the proposals in your terminal, approve what looks right, correct what does not. You now have a baseline. The skill operates from it going forward without you touching it.
Your developers have a prioritized queue. They pull from it in the CLI, pick up the work the skill has surfaced, and close it. Triage is not a dashboard they visit when they remember to. It is the queue their tools read.
The wall is coming. It comes for every project that grows. The only variable is whether you see it in time to build the infrastructure before it arrives — or whether you discover it after the community has already noticed that nobody seems to be home.
Triage is not glamorous. It is the unsexy foundation that everything else runs on. Build it first.
The next article in this series covers what happens after the labeling skill is running: the community response architecture, and what it looks like when an AI skill is handling most of your contributor interactions.
Will Curran is a Principal Technical Program Manager specializing in large-scale open source program operations. This article is part of "AI-Managed OSS," a series on scaling open source communities in the age of agentic development.