Our current documentation process relies on a waterfall of monoliths. Product creates a massive project scope in a vacuum, and Engineering responds with a massive, single-option design document that dives straight into implementation. This siloed approach risks wasting weeks of engineering effort building the wrong solution because we didn't weigh alternative paths or collaborate early enough.
PDD is a framework designed to break these monoliths down into a collaborative, three-step process that ensures we build the right things, at the right time, in the right way.
-
Project Document (💡 The "What & When"): Owned jointly by Product and Engineering. Together, the team aligns on high-level goals, requirements, and breaks the project down into bite-sized milestones.
-
Discovery Document (💡 The "How Else"): A collaborative technical exploration focused entirely on evaluating different architectural options for a single milestone. Zero implementation details allowed. This is run past technical leaders to ensure due diligence before a single line of code is written.
-
Design Document (💡 The "Exactly How"): The technical blueprint for the chosen path. Because the strategy was already validated in the Discovery phase, this document remains hyper-focused, concise, and fast to approve.
PDD protects our engineering sanity and our roadmap. It shifts our culture from individual "brain dumps" to team-wide consensus, guaranteeing that when we finally start building, we are executing the best possible approach with total alignment.
- Preliminary Milestones: This describes significant ‘stages’ in the project's timeline and serves as markers of progress.
- Key Objectives: This describes the high-level outcomes that are desired.
- Current Architecture: Visualise the architecture at a high-level.
- Target Architecture Vision: The macro-level direction (without baking in specific solution implementations yet).
- Design Benefits:
- Functional Requirements: This defines the specific behaviors, actions, or functions that the software must perform.
- Non-functional Requirements: This describes the characteristics of the system (e.g. performance, security, reliability etc).
- Miscellaneous: This section includes miscellaneous documents and links relevant to the overall discussion.
📖 Example Project Document
📖 Document 1: Project Document
Project: Next-Gen Notification Platform (NGNP)
Preliminary Milestones
- Milestone 1 (Q3): Core Message Ingestion & Pub/Sub Pipeline setup.
- Milestone 2 (Q3): Delivery Providers Integration (Email, SMS, Push).
- Milestone 3 (Q4): User Preferences & Throttling Engine.
- Milestone 4 (Q4): Deprecation and migration of the legacy notification system.
Key Objectives
- Consolidate all disparate notification systems across the company into a single, unified platform.
- Enable real-time tracking and delivery observability for downstream product teams.
Current Architecture
Currently, individual microservices (Billing, Shipping, Auth) talk directly to third-party APIs (SendGrid, Twilio) via synchronous HTTP calls. This causes tight coupling, poor error handling, and duplicate implementation of retry logic.
Target Architecture Vision
A centralized, decoupled, event-driven platform. Upstream services will asynchronously emit events to a durable ingestion and messaging layer, insulating our core applications from third-party vendor down-time. The platform will absorb traffic spikes, apply routing and throttling rules, and hand off payloads to delivery providers via a pool of background workers.
[!NOTE] The exact messaging infrastructure and broker technologies will be evaluated and chosen in the Discovery phase.
Design Benefits
- Decoupling: Upstream services no longer care if an email provider is down.
- Centralization: Audit logs, unsubscribe lists, and compliance features are managed in one place.
- Cost Efficiency: Bulk dispatching and rate-limiting prevent expensive vendor overages.
Functional Requirements
- The system must accept notification requests via a REST API and a CLI tool.
- Users must be able to opt-out of specific notification categories via a Web UI.
- The system must retry failed vendor deliveries using an exponential backoff strategy.
Non-functional Requirements
- Availability: 99.95% uptime for the ingestion API.
-
Latency: Critical notifications (e.g., 2FA codes) must be handed off to vendors within
$< 2$ seconds of ingestion. - Scalability: Peak throughput handling of up to 5,000 requests per second.
Miscellaneous
This document presents an objective assessment of the existing system's requirements and deficiencies (for a specific milestone that was defined in the project document), and proposes a high-level approach that will inform the creation of a more detailed design document.
Important
Jointly created by the team.
Run past technical leaders/architects for approval before moving to Design.
No implementation details at this point!
- Glossary: Make sure everyone is on the same page as to what certain words mean.
- Customer Abstraction: What the customer perceives is happening. It’s what they buy/use/derive value from.
- Description: What is the current situation and what are the concerns?
- Problem: What are the underlying problems contributing to this situation?
- Approach: What will be our approach to resolve these issues?
- Business Outcomes: The desired, measurable results of this project that directly benefit the business.
📖 Example Discovery Document
- Ingestion Engine: The infrastructure responsible for receiving, queuing, and durable-storing incoming notification events.
- Fan-out: The process of delivering a single incoming message to multiple destination queues or processing services.
The internal developer (our customer) perceives a simple, never-failing API endpoint. They hit POST /v1/notify, get a 202 Accepted back immediately, and trust the system will reliably handle the delivery in the background.
Our current system handles notifications synchronously. When traffic spikes (e.g., during a marketing campaign), our API threads lock up waiting for external vendor responses, leading to cascading failures across the checkout and signup funnels.
- No message buffering during traffic spikes.
- Lack of a persistent queue results in dropped notifications when downstream vendors experience outages.
- No visibility into message processing states once a request is accepted.
We need to introduce an asynchronous message broker layer between our ingestion API and the delivery workers. Below is the evaluation matrix for potential solutions.
| Use Cases / Requirements | Approach A: Apache Kafka | Approach B: AWS SQS | Approach C: Redis Pub/Sub |
|---|---|---|---|
| High Throughput (5k+ req/sec) | Excellent. Built for massive horizontal scale. | Good. Scales automatically but can hit API rate limits. | Excellent. In-memory speed but bounded by RAM. |
| Durability & Persistence | High. Commits to disk across a distributed cluster. | High. Managed durability across multiple AZs. | Low. Primarily in-memory; risk of data loss on crash. |
| Developer Tooling (CLI/UI) | Moderate. Requires dedicated tools (Kafkacat, UI proxies). | Excellent. Strong AWS CLI, SDK, and Management Console support. | Excellent. Simple CLI operations (redis-cli). |
| Infrastructure Overhead | High. Requires cluster management, Zookeeper/KRaft. | Zero. Fully managed serverless offering. | Low to Moderate. Easy to deploy, but cluster mode requires tuning. |
- Elimination of dropped critical business notifications (e.g., order confirmations).
- 0% impact on core checkout performance due to notification vendor latency.
- Goals: What specifically are we trying to accomplish? By when? Are there guiding principles to follow?
- Requirements: Include any technical (storage, accessibility, latency/speed, monitoring & alerting), security, compliance or Product-requested deliverables.
- Out of Scope (Non-Goals): What are we thoughtfully and purposefully excluding from this project?
- Success metrics: How will we know when the project is done? Include any timelines to meet.
- Proposed Design: How are we proposing to solve the problems: what are we doing, what systems will change?
- Rationale: This must link directly back to the approved Discovery Document (which contains our evaluated solution matrix; example matrix).
- Interactions with existing systems: What systems/components are available to build from or interact with? Are there relationships to common Fastly architecture principles?
- Resources: What resources do we need for this proposed design? Are there licenses, versions, hardware, specific expertise and training,etc to include?
- Development Stages: Identify the implementation stages of the project, so people understand what roll out looks like and how it will be approached.
- Considerations, Risks or Constraints: Add any special considerations, risks, or constraints.
- Open Questions: Note anything for further investigation or future problems to solve (create action items and assign for follow-up if needed).
- Stakeholders & Approval: Primary stakeholders listed check the box to indicate sign-off or add comments with questions/concerns.
- References: Include any links to related documents: Problem Statement, PRD, POC proposals, boards or architecture documentation for additional context.
📖 Example Design Document
📖 Document 3: Design Document
Design: AWS SQS Message Pipeline Implementation
Goals
- Implement the core ingestion API and back it with AWS SQS queues by the end of Q3.
- Ensure zero message loss during vendor outages.
Requirements
-
Technical: SQS standard queues for high throughput; Dead Letter Queues (DLQ) for failing messages; CloudWatch alerting on DLQ depth
$> 0$ . - Security: IAM execution roles restricting queue access to the ingestion service; encryption at rest using AWS KMS keys.
Out of Scope (Non-Goals)
- Building the user preference UI (Deferred to Milestone 3).
- Migrating legacy data or hooks (Deferred to Milestone 4).
Success Metrics
- Ingestion API response time remains under 50ms at 5,000 requests/sec.
- Successful processing of 10 million test messages through the SQS pipeline with zero drops.
Proposed Design
We will deploy a Go-based ingestion microservice behind an Application Load Balancer. This service will validate incoming payloads and publish them directly to an AWS SQS queue named notification-ingestion-queue.fifo. A pool of worker instances will poll this queue, unpack the payloads, and route them to vendors.
[Client App] ➔ [ALB] ➔ [Ingestion Service] ➔ [AWS SQS] ➔ [Worker Pool] ➔ [Vendors]
↳ [DLQ]
Rationale
We chose Approach B (AWS SQS) from our Discovery Matrix Spreadsheet. While Kafka offers higher raw throughput capabilities, AWS SQS gives us zero infrastructure maintenance overhead, built-in DLQ features, and scales well within our target metrics.
Interactions with Existing Systems
- Auth Service: The ingestion API will interface with our existing JWT validation middleware to authenticate client applications.
- Fastly Edge: We will utilize Fastly at the edge to rate-limit malicious or misconfigured internal clients before they hit the ingestion layer.
Resources
- Infrastructure: Terraform AWS Provider v5.0 access.
- Expertise: 1 Senior DevOps Engineer (allocated for 2 weeks) to review IAM policies and KMS configurations.
Development Stages
- Stage 1 (Local/Sandbox): Set up LocalStack environment; write Go publishers and consumers.
- Stage 2 (Staging Deployment): Provisions SQS queues via Terraform; execute load testing up to 5k rps.
- Stage 3 (Canary Production): Route 5% of internal non-critical traffic (e.g., marketing updates) to the new pipeline.
- Stage 4 (Full General Availability): Switch all remaining traffic paths.
Considerations, Risks, or Constraints
- Constraint: SQS payload limit is 256KB. If a notification contains heavy template metadata, it will fail.
- Mitigation: Payloads exceeding 150KB will automatically store the body in an S3 bucket and pass the object reference pointer through the queue message attributes instead.
Open Questions
- Do we need a FIFO queue to guarantee absolute ordering, or is standard SQS acceptable for marketing alerts? (Assigned to @johndoe for performance benchmarking by Friday)
Stakeholders & Approval
- @lead_architect - Architecture Review
- @security_champion - IAM & Encryption Sign-off
- @product_manager - Timeline Sign-off