Website of Peter Westerman

There is a failure mode in AI-assisted development that is more common than bugs and more expensive than architectural mistakes. It is the failure where the AI produces output that is technically correct — properly structured, completely functional, exactly what was described — but does not match what was actually needed. The AI built what you said. What you said was not what you meant.

This is not an AI problem. It is a requirements problem — a gap between the intent in your head and the specification you produced. AI systems are very good at building to specification. They are not good at bridging the gap between what was specified and what was intended when those two things differ, because they have no access to intent that was not expressed. Every instance of this failure mode is a communication failure on the human side, not a capability failure on the AI side.

This is also the most correctable of the failure modes in this case study. Bugs require debugging time. Architectural mistakes require architectural redesign. Requirements gaps only require better requirements — and better requirements are available before the build begins, not after.

A Specific Example

In the Journey Mapper project, the requirement for the stage editor was: “Allow users to add and edit stages in their customer journey map, with the ability to move stages up and down.” The AI built exactly that: an interface where users could add stages, edit them inline, and move them using up and down arrow buttons.

The problem: the product owner had imagined drag-and-drop reordering. Every comparable customer journey mapping tool uses drag-and-drop for stage ordering. The user expectation was drag-and-drop. The requirement said “move up and down” and the AI reasonably interpreted that as arrow buttons rather than drag interaction — both are valid implementations of “move up and down.”

Two sessions of work produced the arrow-button implementation. Rebuilding it with drag-and-drop took another session. The total cost was three sessions instead of one. The requirements gap was a single missing phrase: “via drag-and-drop reordering.” One clause in one sentence would have produced the correct implementation in the first pass. Instead, two sessions were spent on rework that added no new capability.

This example illustrates the most common type of requirements gap: an implicit assumption about implementation that was obvious to the product owner because of their mental model, but genuinely ambiguous in the written requirement. The mental model was “drag-and-drop because that’s how this type of feature works.” The written requirement contained nothing that conveyed the mental model. The AI filled the ambiguity with a plausible interpretation that happened to be wrong.

The Three Sources of Requirements Gaps

Looking across twenty products, requirements gaps almost invariably fall into three categories.

Implicit assumptions about implementation: Things that seem obvious from the product owner’s mental model but are not stated anywhere. The drag-and-drop example. The assumption that a date picker means a calendar widget rather than three dropdowns. The assumption that “edit” means inline editing rather than a modal. Each of these is an assumption about implementation style that is obvious in context but invisible in a written requirement. The fix: before writing requirements, ask “what am I assuming about how this will be implemented that I have not stated?” If the assumption is significant, state it.

Missing constraints: Things the solution must not do, or performance requirements, or integration requirements, that were not stated. “The form should save data” without “the form should pre-populate with existing data when editing a record.” “The search should return results” without “the search should return results in under two seconds on a standard shared hosting server.” The AI builds to the stated requirement and produces code that satisfies it — while violating the unstated constraint. Missing constraints are the most common source of technically correct but contextually wrong implementations.

Undefined failure behavior: What should happen when things go wrong. What does the user see when the Claude API returns a rate limit error? What happens when required fields are submitted empty? What is the behavior when the database is unavailable? Requirements written from the happy-path perspective leave failure behavior undefined. The AI fills the undefined cases with its best guess. The guesses are often plausible. They are frequently not what was intended. Defining failure behavior explicitly is the most commonly omitted requirement element and the source of some of the worst user experiences in production.

The User Scenario Method

The most reliable technique for surfacing requirements gaps before they become corrective iteration is writing concrete user scenarios before writing feature requirements.

A user scenario is a one-paragraph description of a real person doing a specific thing with the product: “A content editor with a queue of 15 articles opens the editorial review panel, clicks on the first article, reads the AI-generated summary, decides it misses the key point of the article, edits the summary in place to add a sentence about the main finding, marks the article as reviewed and ready to publish, and clicks Next to move to the second article in the queue.”

Writing this scenario surfaces requirements that a feature list misses. The “marks as reviewed and ready to publish” step implies a status system with specific states. The “clicks Next to move to the second article” step implies queue navigation in a defined order. The “edits the summary in place” step implies inline editing on a field that might otherwise have been read-only. The “15 articles” context implies the list view needs to handle at least that volume without performance issues.

None of these requirements appear in “build an editorial review panel.” All of them would need to be built before the feature is complete. Finding them in a scenario-writing exercise before the build begins costs nothing. Finding them during testing after the build costs one to three additional sessions each.

When you provide user scenarios to the AI in addition to feature requirements, the AI can trace the technical chain required to support each step in the scenario. It will surface its own questions: “To support the sequential navigation in your scenario, I’ll need to define the queue ordering. Should that be by publication date, by article priority score, or by manual reordering?” That question, answered before the build begins, is a requirements gap closed at zero cost.

Evaluating Against Intent, Not Specification

The other half of the requirements gap problem is evaluation. The natural mode for evaluating AI output is specification compliance: did the AI do what I said? The better mode is intent compliance: does this do what I actually need?

These produce different evaluations when the specification was imperfect — which is most of the time. In specification-compliance mode, the drag-and-drop outcome passes evaluation: the AI built “move stages up and down” exactly as specified. In intent-compliance mode, it fails: the feature does not produce the user experience that was intended.

Evaluating against intent requires walking through the actual user scenarios rather than checking feature items off a list. Not “does the stage reordering feature exist?” but “can I drag a stage from position two to position four and have it land correctly, with the other stages shifting appropriately?” Every step of the scenario is a test. Any step that fails or produces unexpected behavior is a requirements gap, regardless of whether anything in the written requirements was violated.

Building this evaluation habit is one of the higher-return investments in a vibe coding practice. It reveals requirements gaps when they are still cheap to fix — during testing of the first implementation — rather than when they have been compounded by features built on top of the incorrect foundation. And it builds the analytical skill of thinking precisely about what software should do versus what it does, which is a transferable capability that applies to every piece of software you use, manage, or commission.

The Template for a Complete Requirements Document

The requirements document that consistently produced the best first-pass output across the full portfolio combined five elements: purpose statement, user scenarios, technical constraints, v1 feature list, and failure behavior specifications. Omitting any of these predictably produced specific types of requirements gaps.

Omitting the purpose statement produced features that were technically correct but missed the product’s core use case — the AI optimized for a general version of the feature rather than the specific version the product context required. Omitting user scenarios produced feature lists that were individually correct but did not combine into a coherent user experience. Omitting technical constraints produced implementations that worked in a development environment but failed under production conditions. Omitting the v1 feature list produced implementations with scope that expanded past what could be tested. Omitting failure behavior specifications produced products that worked for the happy path and failed unpleasantly when anything went wrong.

The complete requirements template takes thirty to sixty minutes to fill out for a new product or major feature. It prevents three to five sessions of corrective iteration that would otherwise be needed to address the gaps. The investment ratio is consistently favorable. Building the habit of completing the template fully before any build session begins is the single most impactful practice in the first month of any vibe coding practice.

The Evaluation Discipline in Practice

Building the habit of evaluating against intent rather than specification requires a concrete practice that makes the distinction operational. The practice that worked: after every build session, before reporting any issues to the AI, write down what was supposed to happen (from the user’s perspective, not the feature specification), what actually happened when you walked through the primary user scenarios, and what the gap is between those two things.

This three-part structure — supposed to happen, actually happened, gap — forces the evaluation to happen at the intent level rather than the specification level. You are not asking “did the AI build what I said?” You are asking “when a real user does what they would do with this product, does it produce the experience I intended?” The gap between those two questions is where requirements gaps live.

Writing down the evaluation before reporting it also produces more useful feedback to the AI. “The stage editor works as described, but when I walk through the scenario of a user reordering five stages in a single session, it takes approximately twenty clicks where drag-and-drop would take five interactions. The current interaction model is technically correct but creates friction that the intended use case does not call for.” That feedback directs a specific, meaningful correction rather than a vague “improve the reordering.”

How Recent AI Innovations Change This Picture

The requirements gap problem described in this post — the AI producing exactly what was asked for rather than what was needed — is fundamentally a human communication failure, not an AI capability failure. AI innovations do not eliminate the requirements gap by themselves. They do, however, provide new tools for catching requirements gaps before they manifest as expensive builds.

Extended thinking changes the quality of requirements analysis. In the original methodology, a requirements document was handed to the AI and the AI began building. The review step — does the AI’s proposed architecture match what was actually needed? — was entirely dependent on the human’s ability to anticipate gaps at architecture review time. With extended thinking enabled, Claude can reason through a requirements document before proposing an architecture: identifying ambiguities, flagging missing scenarios, questioning assumptions. The AI becomes a requirements reviewer, not just a requirements executor.

MCP integrations change what requirements can reference. Requirements that previously said “integrate with the existing user database” required the human to manually export and document the database schema. With MCP database connections, the AI can query the live schema while reviewing requirements, catching integration mismatches before they become build problems. Requirements gaps related to integration assumptions — which were among the most expensive failures in this case study — become visible before the build starts.

The Files API, introduced in 2025, allows storing and accessing files across sessions. For requirements management, this means a requirements document is no longer a paste-and-forget artifact: it can be maintained as a living document accessible to the AI across all related sessions. Scope creep becomes detectable by comparing the current session’s direction against the stored requirements document. The human discipline of “scope management” gets AI-assisted enforcement.