specification

Executable specs are behavior descriptions that also run as tests. Instead of treating documentation and tests as separate artifacts, the spec itself is executable, making it the most trusted source of truth for both humans and agents. Why it matters Single source of truth: the spec and the test are the same artifact, reducing drift. Dual audience: readable narratives for humans and deterministic checks for machines. Agent steering: a rich corpus of examples becomes the behavioral contract an agent can follow when implementing or refactoring. Common forms Markdown-based test suites: scenarios are written as text with fenced code blocks that a harness executes. Data-driven specs: inputs and expected outputs are expressed as fixtures with a thin runner. Literate-style specs: prose describes the intent; code blocks demonstrate behavior. Trade-offs and risks Ambiguity risk: if comments or prose stand in for assertions, behavior can be under-specified. Tooling burden: you need a runner that executes blocks, reports failures, and keeps the format pleasant to edit. Coverage bias: specs can emphasize readable cases and miss edge-case rigor. Design guidelines Keep scenarios small and focused; one behavior per block. Use explicit, machine-checkable assertions near the example. Make failure output tight and readable so the spec remains a usable debugging surface. Treat the spec as the contract for refactors: if the spec passes, the refactor is valid. Related notes Spec-Driven Development References Charlie Marsh, “Markdown test suites as executable specs” (thread). https://x.com/charliermarsh/status/2007117912801427905 Simon Willison on language-independent test suites (quoted in the same thread). https://x.com/simonw

A two-phase approach for building large features: specification through structured Q&A, followed by execution in a separate session. Pattern Phase 1: Spec refinement Start with a minimal spec or prompt. Ask the AI to interview you using interactive questioning to expand and refine the spec. Phase 2: Execution Create a new session and implement the refined spec. Why separate phases Clarity of intent: Specification mode focuses on decisions, not code. Deeper exploration: 40+ questions can surface edge cases, UX details, and trade-offs. Control: You shape the spec through answers, not by reviewing generated code. Clean context: Execution session starts with complete requirements, not discovery artifacts. Example prompt (Phase 1) read this @SPEC.md and interview me in detail using the AskUserQuestionTool about literally anything: technical implementation, UI & UX, concerns, tradeoffs, etc. but make sure the questions are not obvious be very in-depth and continue interviewing me continually until it's complete, then write the spec to the file When to use Building large features or new projects When requirements are incomplete but you know the rough shape When you want systematic exploration of edge cases and trade-offs When you prefer decision-making over code review as the primary feedback loop Trade-offs Overhead: More upfront time before code is written Question fatigue: 40+ questions requires sustained attention Assumes good questions: AI must ask non-obvious, useful questions Two sessions: Context doesn’t flow automatically from spec to execution Reference Thariq Shihipar (@trq212), December 28, 2024: ...

A git branching strategy using a parallel ‘spec’ branch to iterate on messy specifications without polluting the main codebase, enabling seamless context injection for AI agents.

A development methodology where high-level technical specifications, rather than manual code entry, serve as the primary driver for implementation by AI agents.