Skip to content

Instantly share code, notes, and snippets.

@mikailcetinkaya
Last active April 19, 2026 06:37
Show Gist options
  • Select an option

  • Save mikailcetinkaya/226e09df63056b0523a1f2533ee2f3db to your computer and use it in GitHub Desktop.

Select an option

Save mikailcetinkaya/226e09df63056b0523a1f2533ee2f3db to your computer and use it in GitHub Desktop.

AI Workspace Memory & Quality Gate System

Theoretical Framework

Every AI output is a composite of signal and noise:

AI_output(t) = S(t) + N(t)

In a multi-step chain, each step builds on the previous one — noise included:

C(t+1) = f(Contribution(1), Contribution(2), ..., Contribution(t))

Unfiltered noise compounds. The human filter function F(B(t)) determines how much noise passes through:

Contribution(t) = S(t) + N(t) * (1 - F(B(t)))

This system encodes B(t) — domain knowledge, project context, architectural decisions — into a portable .ai/ directory. Instead of relying on human attention at every gate, the knowledge itself travels with the codebase. Every AI tool, every session, every team member starts with F already raised.

But encoded knowledge alone is not enough. The system also needs active quality gates — automated checkpoints where the agent evaluates its own readiness, output quality, and trajectory before proceeding.


Quality Gate System

Meta-Control: Iteration Size Evaluation

Before any gate runs, the agent must evaluate the size of the task itself. A large iteration with even low noise causes significant damage. A small, focused iteration keeps the filter effective.

Protocol:

The agent must assess task complexity before starting execution. If the task exceeds the complexity threshold, the agent must decompose it into smaller steps and present the decomposition to the user for approval before proceeding.

Assessment criteria:

  • Number of files/components affected
  • Number of independent decisions required
  • Degree of uncertainty in requirements
  • Cross-cutting concerns (does this touch multiple layers/services?)

Transparency output:

[ITERATION SIZE EVALUATION]
Task: "Implement payment gateway integration"
Complexity: HIGH
- Affects 4 services, 2 database schemas, 1 external API
- 6+ independent architectural decisions required
- Uncertainty: medium (API docs available, but edge cases unclear)

Recommendation: Decompose into 5 sub-tasks:
  1. Define data model for payment entities
  2. Implement gateway client with error handling
  3. Build service layer with retry logic
  4. Add webhook endpoint with signature verification
  5. Integration tests with sandbox environment

Awaiting approval before proceeding.

Gate 1: Context Sufficiency Check

The agent must verify that available context is sufficient for the task before producing any output.

Protocol:

After reading all available .ai/ files and task description, the agent evaluates whether it has enough information to proceed. If context is insufficient, the agent must stop and explicitly list what is missing.

Evaluation checklist:

  • Do I understand the project architecture? (from overview.md, analysis.md)
  • Do I know the code generation rules and anti-patterns? (from codegen.md)
  • Do I understand the domain model and invariants? (from model.md)
  • Do I know the relevant inter-repo contracts? (from contracts.md)
  • Do I have the specific technical context this task requires?
  • Are there relevant past decisions I should consider? (from decisions.md)

Transparency output:

[CONTEXT GATE]
Status: INSUFFICIENT
Available context:
  + Project architecture: understood (overview.md)
  + Code patterns: understood (codegen.md)
  + Domain model: understood (model.md)
Missing context:
  - Iyzico API rate limits and error codes — not in any .ai/ file
  - Current retry policy for external service calls — not documented
  - Whether partial refunds are a requirement — ambiguous in task description

Action: Requesting missing information before proceeding.
[CONTEXT GATE]
Status: SUFFICIENT
All required context available. Proceeding to execution.

Gate 2: Signal/Noise Ratio Evaluation

After producing output, the agent must evaluate the quality of its own output using three methods.

Method A: Self-Evaluation

The agent asks itself: "Which parts of this output are derived from the given context, and which parts are based on assumptions or general knowledge?"

[SELF-EVALUATION]
Output: PaymentService implementation (47 lines)
Context-derived:
  - Strategy pattern usage (matches existing codebase pattern)
  - Error handler structure (follows codegen.md rules)
  - Database schema (matches model.md entities)
Assumption-based:
  - Timeout value set to 30s (no documented standard found)
  - Retry count set to 3 (general best practice, not project-specific)

Assumption ratio: 2/7 decisions = 28%
Threshold: 30%
Status: PASS (but flagging assumptions for human review)

Method B: Confidence Scoring

The agent assigns a confidence score to its output, broken down by component.

[CONFIDENCE SCORING]
Overall confidence: 74%
Breakdown:
  - Data model:           92% (well-defined in model.md)
  - Service logic:        81% (pattern clear, some edge cases uncertain)
  - Error handling:       55% (Iyzico error codes not fully documented)
  - Webhook verification: 48% (HMAC implementation assumed from general knowledge)

Components below 60% threshold: error handling, webhook verification
Action: Requesting human review on low-confidence components before proceeding.

Method C: Context-Output Alignment

The agent checks whether each claim or decision in the output has a corresponding basis in the provided context. Claims without contextual backing = noise.

[CONTEXT-OUTPUT ALIGNMENT]
Checking 12 decisions in output against available context:
  [GROUNDED]  Use strategy pattern for payment providers    <- codegen.md line 34
  [GROUNDED]  PaymentTransaction entity with status enum    <- model.md line 78
  [GROUNDED]  Async webhook processing                     <- contracts.md line 12
  [UNGROUNDED] 3-second polling interval for status check   <- no source
  [UNGROUNDED] Base64 encoding for callback payload         <- no source

Ungrounded ratio: 2/12 = 17%
Threshold: 25%
Status: PASS (flagging ungrounded decisions)

Combined S/N Gate Decision:

The three methods produce a composite score. If any single method fails its threshold, or if two methods flag concerns on the same component, the gate blocks and requests human input.


Gate 3: Drift Detection

At each step, the agent checks whether the current output still aligns with the original plan and objective. Small per-step deviations compound into large trajectory shifts.

Protocol:

The agent compares the current step's output against:

  1. The original task objective
  2. The approved plan (if one exists)
  3. The cumulative direction of all previous steps

Transparency output:

[DRIFT CHECK]
Original objective: "Add Iyzico payment integration with 3D Secure support"
Current step: Step 3 of 5 — "Build service layer with retry logic"
Plan alignment: ON TRACK

Checking for scope drift:
  - Added generic payment abstraction layer (not in plan)
  - This adds value but was not requested
  
Drift detected: MINOR (scope expansion)
Action: Flagging addition. Should I include the generic abstraction 
        layer, or keep implementation Iyzico-specific as planned?
[DRIFT CHECK]
Original objective: "Refactor user authentication to support SSO"
Current step: Step 4 of 6 — "Update session management"
Plan alignment: DRIFTED

Cumulative drift analysis:
  Step 1: On track (SSO provider config)
  Step 2: On track (SAML response parsing)  
  Step 3: Minor drift (added JWT refresh logic — related but not planned)
  Step 4: Significant drift (rewriting session store from scratch — 
          original plan said "extend existing store")

Total drift: SIGNIFICANT
Action: Stopping. Steps 3-4 have deviated from the approved plan.
        Requesting human review before continuing.

Transparency Rules

All gates must report their reasoning to the user. The agent never silently passes or fails a gate.

Required behaviors:

  1. Always show your work. Every gate evaluation must produce a structured output block that the user can read.

  2. Flag uncertainty explicitly. "I don't know" is a valid and valuable output. Never fill gaps with assumptions without declaring them.

  3. Distinguish sources. When producing output, clearly separate what comes from project context (.ai/ files, codebase, documentation) versus general knowledge versus assumption.

  4. Report gate results even when passing. A passing gate still shows its evaluation — this builds trust and lets the user catch false passes.

  5. Summarize cumulative state. At each step, briefly report the state of all previous gates, not just the current one. This prevents drift blindness.

Example cumulative report:

[STEP 3/5 — GATE SUMMARY]
Iteration size:    OK (focused on single service layer)
Context gate:      PASS (all required context available)
S/N gate:          PASS with flags (2 assumptions noted, both low-risk)
Drift gate:        MINOR (scope addition flagged, awaiting decision)

Proceeding with step 3 pending drift decision.

Workspace Memory Architecture

Template Repo (stateless tooling)

The template repo (ai-workspace-template) is disposable infrastructure. It contains:

  • Skills: onboard, scaffold, refresh, sync, check
  • Templates: blank versions of all .ai/ files
  • Hooks: pre-commit hook for auto session capture
  • Index: generated manifest of all scaffolded repos

Scaffolded Repo Structure

Each scaffolded repo receives an .ai/ directory:

.ai/
  agents.md        # Entry protocol, quality gates, permissions
  overview.md      # Project purpose, architecture, consumers
  analysis.md      # Auto-generated: stack, structure, dependencies, coverage
  codegen.md       # Code generation rules, patterns, anti-patterns, boundaries
  model.md         # Domain model, entities, invariants, glossary, data flows
  contracts.md     # Inter-repo API and event contracts (exposes/consumes)
  decisions.md     # Append-only architectural decision log
  rules.md         # AI behavior instructions, session directives
  workspace.yaml   # Committed peer repo names and contract references
  sessions/        # Auto-captured session summaries
local.md           # (gitignored, repo root) Developer-specific config

File Details

agents.md — Entry Protocol & Quality Gates

This file is the first thing any AI agent reads. It defines:

Read order: The agent must read .ai/ files in a specific sequence before doing any work.

## Entry Protocol

1. Read this file completely
2. Read overview.md — understand what this project does
3. Read analysis.md — understand the technical landscape
4. Read codegen.md — understand how code should be written here
5. Read model.md — understand the domain
6. Read contracts.md — understand cross-repo dependencies
7. Read decisions.md — understand past architectural choices
8. Run Context Sufficiency Check (Gate 1)
9. Only then: begin working on the task

Agent identification: Every agent must identify itself in session logs.

## Identification

When starting a session, record:
- Agent type (Claude Code / Cursor / Copilot / other)
- Model (if known)
- Session start time
- Task description

Permission boundaries:

## Permissions

### Free (no confirmation needed)
- Reading any file
- Running tests
- Generating analysis or suggestions
- Asking clarifying questions

### Needs Human Confirmation
- Modifying existing code
- Creating new files
- Changing configuration
- Any action affecting more than 3 files
- Any step where S/N confidence is below 60%

### Forbidden
- Deleting files without explicit instruction
- Modifying .ai/ files (except sessions/)
- Changing CI/CD configuration
- Direct database operations
- Pushing to remote branches

Quality gate integration:

## Quality Gates

All gates defined in this document are mandatory. The agent must:
1. Run iteration size evaluation before starting
2. Run Gate 1 (context check) after reading all .ai/ files
3. Run Gate 2 (S/N evaluation) after producing each output
4. Run Gate 3 (drift detection) after each step in a multi-step task
5. Show all gate outputs to the user — never skip transparency

Multi-agent coordination:

## Multi-Agent Rules

When multiple agents work on the same repo:
- Check sessions/ for recent activity before starting
- Do not modify files another agent is currently working on
- Log your session immediately, including open questions
- Reference other agents' sessions when relevant

overview.md — Project Context

# [Project Name]

## Purpose
[What this project does and why it exists]

## Key Services
[Main components/services and their responsibilities]

## Architecture
[High-level architecture: monolith/microservices, key patterns, data flow]

## Consumers
[Who/what depends on this project: other repos, teams, external clients]

## Non-Goals
[What this project explicitly does NOT do — critical for reducing noise]

codegen.md — Code Generation Rules

# Code Generation Rules

## Architectural Patterns (follow these)
[List patterns used in this project with examples]

## Anti-Patterns (never generate these)
[List patterns explicitly forbidden with reasoning]

## Generation Boundaries
[What the AI should and should not auto-generate]

## Style & Conventions
[Naming, file structure, import order, error handling patterns]

workspace.yaml — Cross-Repo Awareness

# workspace.yaml (committed)
name: payment-service
peers:
  - name: user-service
    contracts:
      - consumes: UserProfile API v2
      - publishes: PaymentCompleted event
  - name: notification-service
    contracts:
      - publishes: PaymentFailed event
# local.md (gitignored, repo root)
developer: mikail
ide: cursor
hub_path: ~/workspace/ai-workspace-template
peers:
  user-service: ~/workspace/user-service
  notification-service: ~/workspace/notification-service

Session Auto-Capture

Sessions are captured through two mechanisms:

Pre-commit hook: Automatically diffs staged files and generates a summary appended to the daily session file.

Rules-based instruction: The AI agent writes a session summary at the end of each session.

Both append to the same file: .ai/sessions/YYYY-MM-DD.md

## Session: 2026-04-19 14:30

Agent: Claude Code (claude-opus-4-6)
Developer: mikail
Duration: 45 min

### Task
Implement Iyzico payment integration — step 2 of 5 (gateway client)

### Gate Results
- Iteration size: OK (single client class)
- Context: PASS
- S/N: PASS (1 assumption flagged — timeout value)
- Drift: ON TRACK

### Changes
- Created src/payments/iyzico_client.py
- Modified src/payments/gateway_factory.py
- Added tests/payments/test_iyzico_client.py

### Decisions Made
- Used 30s timeout (assumption — needs team confirmation)
- Implemented idempotency key as UUID v4

### Open Questions
- Should we support partial refunds in v1?
- Rate limit handling: queue or retry?

Multi-Tool Bridge Configuration

The system is IDE/CLI agnostic. The scaffold skill generates the appropriate bridge file that points to .ai/rules.md.

Claude Code

The scaffold generates a CLAUDE.md at the repo root:

# CLAUDE.md
Read and follow all instructions in .ai/agents.md before doing any work.
All code generation must comply with .ai/codegen.md.
All architectural decisions must be logged in .ai/decisions.md.
At the end of each session, write a summary to .ai/sessions/.

Cursor

The scaffold generates .cursor/rules/ai-workspace.mdc:

---
description: AI Workspace Memory System rules
globs: **/*
alwaysApply: true
---

Read and follow all instructions in .ai/agents.md before doing any work.
All code generation must comply with .ai/codegen.md.
All architectural decisions must be logged in .ai/decisions.md.
At the end of each session, write a summary to .ai/sessions/.

GitHub Copilot

The scaffold generates .github/copilot-instructions.md:

Read and follow all instructions in .ai/agents.md before doing any work.
All code generation must comply with .ai/codegen.md.

Windsurf

The scaffold generates .windsurfrules at the repo root:

Read and follow all instructions in .ai/agents.md before doing any work.
All code generation must comply with .ai/codegen.md.
All architectural decisions must be logged in .ai/decisions.md.
At the end of each session, write a summary to .ai/sessions/.

Update & Versioning Strategy

What updates and when

File Update trigger Method
overview.md Architecture changes, new consumers Manual
analysis.md Significant code changes refresh skill (auto-generates)
codegen.md New patterns adopted, anti-patterns discovered Manual
model.md Domain model changes Manual after schema migrations
contracts.md API changes, new events Manual, cross-checked with peers
decisions.md Any architectural decision Append-only, never edit past entries
workspace.yaml New peer repos, contract changes Manual
sessions/ Every session Auto (hook + agent)

Refresh skill

The refresh skill re-analyzes the codebase and updates analysis.md. It should be run:

  • After major refactors
  • When onboarding new team members (ensures analysis is current)
  • Periodically (weekly or bi-weekly for active repos)

Sync skill

The sync skill cross-checks contracts.md across peer repos:

  • Verifies that what repo A says it exposes matches what repo B says it consumes
  • Flags contract mismatches
  • Reports stale contracts (referenced but no longer existing)

Check skill

The check skill validates the health of the .ai/ directory:

  • Are all required files present?
  • Are files non-empty and recently updated?
  • Do cross-references resolve?
  • Are session logs being captured?

Onboarding Flow

  1. Place the template repo in your workspace
  2. Trigger the onboard skill in your IDE/CLI
  3. Skill detects IDE type, collects developer info
  4. Select repos to scaffold
  5. Each repo is scaffolded with user confirmation
  6. Bridge files are generated for detected IDE
  7. Index is generated, connectivity verified
  8. First check runs to validate setup

Design Principles

  1. All knowledge lives in target repos. The template is disposable. Each repo's .ai/ is the source of truth, versioned in its own git history.

  2. IDE/CLI agnostic. Bridge files adapt to whatever tool the developer uses. The .ai/ directory is the single source; bridges are pointers.

  3. Auto-capture. Session logs are generated through hooks and agent rules. No manual effort required for basic session tracking.

  4. Agent protocol is mandatory. Every AI agent reads agents.md first. Quality gates are non-negotiable. Transparency is enforced.

  5. Cross-repo awareness without coupling. workspace.yaml (committed) holds logical names. local.md (gitignored) maps to filesystem paths. Teams share structure without assuming identical setups.

  6. Team-oriented. Sessions capture who did what with which tool. Decisions are logged with participants. Open questions serve as async communication through the AI layer.

  7. Quality gates are first-class citizens. Not an afterthought. The gate system (iteration size, context check, S/N evaluation, drift detection) is embedded in the entry protocol and enforced at every step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment