How to Build a Constitutional AI Critique Pipeline with Claude and DSPy in 2026

A developer ships a content moderation endpoint. First week goes fine. Week two, a crafted input slips through — a policy violation the system prompt didn’t anticipate. They add another rule. Then another. By month three, there’s a dense system prompt nobody can fully audit and the model applies it inconsistently.

That’s not a prompt problem. That’s a specification problem. The developer never built a pipeline — they built a single gate with hope stapled to it. Constitutional AI Prompting gives you the architecture to replace hope with a testable loop.

Before You Start

You’ll need:

• Claude API access — Claude Sonnet 4.6 (claude-sonnet-4-6) is the recommended build target: best balance of generation quality and cost at $3/$15 per MTok (Anthropic Docs)
• DSPy 3.2.1 — pip install dspy==3.2.1 or uv add dspy, requires Python 3.10+ (DSPy Docs)
• A working draft of your AI Constitution — a list of principles your output must satisfy

This guide teaches you: How to decompose a constitutional AI critique pipeline into four distinct components, specify the input/output contract for each, and validate that the loop catches violations before they reach your users.

What this guide does NOT build: The RL-CAI phase from the original Constitutional AI paper (Anthropic / Bai et al. 2022) — where AI-generated preference labels replace human annotation during model training — is a training-time method. You cannot replicate it by calling the Claude API. What you can build is the SL-CAI critique-revise loop: model generates → self-critiques against each principle → revises where it fails. That’s the prompting pattern, and it’s the one that transfers directly to production pipelines without touching model weights.

The Real Failure Mode: A Prompt Is Not a Policy

Here’s the incident pattern. A developer writes a system prompt with a list of prohibited behaviors. The model follows it most of the time. The remainder is where things get expensive — edge cases, adversarial inputs, context shifts that weren’t in the original list.

Prompt-as-policy breaks because context shifts. A principle like “don’t produce harmful content” means something different in a medical support context than in a developer sandbox. The model guesses, inconsistently. The fix isn’t a longer prompt. The fix is a loop that critiques and revises until the output passes each principle — one at a time, verifiably.

Self-critique loops are not new. The Self Refine paper from Madaan et al. 2023 measured an average performance gain across seven diverse tasks when models iterated through feedback cycles rather than generating in a single pass — with results that varied significantly by task type. The key finding: no fine-tuning or training data required. The same pre-trained model generates, critiques, and revises. That’s your architecture. You just need to specify it properly.

Step 1: Decompose Your Pipeline into Four Components

Before you open DSPy or the Claude API, draw the boxes. A Constitutional AI critique pipeline has exactly four components. Miss one and the pipeline either halts on unhandled output or ships unverified content.

Your system has these parts:

• Constitution — a structured list of principles your output must satisfy. Each principle needs a plain-language statement, a testable violation condition, and a revision instruction. This is not a system prompt paragraph — it’s a data structure the critique module reads one entry at a time.
• Critique module — takes model output plus one principle, returns a structured judgment: PASS or FAIL with a specific failure reason. One principle per call. “Generally evaluate this for safety” is not a critique spec — the model can’t prioritize vague instructions.
• Revise module — takes the original output plus the critique failure reason, returns a revised output. Constrained to address the specific failure only. A revise module that rewrites the entire output introduces new violations.
• Gate — a loop controller with a hard cap on iterations. Without it, an adversarial input causes an infinite critique-revise cycle. The gate stops the loop and flags for human review when the cap is hit.

The Architect’s Rule: If you can’t name what each component receives, returns, and is not allowed to do — the AI can’t build it correctly. Define the boundaries before you write a single module.

Step 2: Define the Constitution and Critique Contract

This is where most implementations skip the hard part. They say “define your principles” and show three bullet points. Here’s what you actually need to specify for each component.

Your constitution contract:

• Principle format: {id, statement, violation_example, revision_guidance} — four fields, not one. The model needs a concrete example of a violation and specific guidance on how to fix it. A statement alone produces vague critique.
• Principle count: start with the minimum set that covers your use case. The original CAI training run at Anthropic used roughly ten human-written constitutional principles (Anthropic / Bai et al. 2022). Ship five principles, measure coverage, expand. More principles means more critique calls, more latency, and more cost.
• Violation taxonomy: decide what a FAIL means in your context. Severity-1 violations (style, tone) and severity-2 violations (safety-critical content) require different revision strategies. Mixing them in one schema means the revise module can’t prioritize correctly.

Your critique module contract:

• Input: model_output: str, principle: {id, statement, violation_example, revision_guidance}
• Output: {result: "PASS" | "FAIL", reason: str | None, principle_id: str}
• Constraint: returns reason=null on PASS — no hallucinated explanations for passing outputs
• One principle per call — batching multiple principles in one critique call reduces precision

Your revise module contract:

• Input: original_output: str, failed_principle: dict, critique_reason: str
• Output: revised_output: str
• Constraint: revises only the section that violated the principle — full rewrites introduce new violations elsewhere

DSPy configuration checklist:

• DSPy version: 3.2.1 (stable, released May 5, 2026, via PyPI). Do not use the 3.3.0b1 beta in production — a BaseLM API migration is underway and custom LM adapter code may break (DSPy GitHub).
• LM config: dspy.LM('anthropic/claude-sonnet-4-6', api_key=ANTHROPIC_API_KEY) — for high-volume critique steps where cost matters, Claude Haiku 4.5 (claude-haiku-4-5) at $1/$5 per MTok is worth benchmarking against Sonnet 4.6 on your specific violation patterns (Anthropic Docs).
• Python version: 3.10+

The Spec Test: If your constitution stores principles as a flat list of strings, the critique module has no way to return a judgment tied to a specific principle ID. The gate can’t track which principles passed. You’ll get output, not traceability.

Step 3: Wire the Critique Loop in Build Order

Build order matters. Each component’s output is the next component’s input contract. Get the order wrong and you spend time reworking already-built modules.

Build order:

1. Constitution loader first — every other component depends on the principle schema you define here. Get the data structure right before writing a single module. A constitution stored as raw strings forces rewrites of critique and revise modules later.
2. Critique module second — no upstream dependencies beyond the constitution schema. Build it, test it against a known violation, confirm the output schema matches {result, reason, principle_id}.
3. Revise module third — takes critique output as direct input. If your critique module’s failure reason is vague, the revise module produces vague repairs. These two are tightly coupled — their schemas need to match.
4. Gate last — the loop controller is the simplest component once the other three are correct. Its only job: count iterations, stop the loop, and return a human-review flag when the cap is hit.

For each module, specify:

• What it receives (exact input schema)
• What it returns (exact output schema, including null cases)
• What it must NOT do (this is where Hallucination enters — a revise module that rewrites scope beyond the flagged section is hallucinating intent)
• How to handle failure (gate behavior when critique returns a schema-invalid response)

The Prompt Chaining pattern wires these four components in sequence. Each step has a defined contract. The model isn’t improvising the pipeline — you’re specifying it. DSPy Signatures enforce output schemas so the critique module can’t return free-form text when your gate expects {result, reason, principle_id}.

Step 4: Validate the Loop Before It Ships

Validation here is not “does it run?” It’s “does it catch the violations we know about, and does it stop at the iteration cap?”

Validation checklist:

• Principle coverage test — failure looks like: you wrote five principles, but critique calls cycle through only three. Check principle IDs in output objects across your test set. A principle that never fires is either malformed or your test cases don’t exercise it. Fix before shipping.
• Iteration cap test — failure looks like: an adversarial input causes critique to FAIL on every iteration. Without the gate, the loop runs until rate limit. Inject a deliberately-unfixable input and verify the gate triggers at your defined maximum.
• Revision scope test — failure looks like: the revise module rewrites the entire output when only one sentence violated a principle. Audit revision diffs — the delta should be minimal and targeted to the flagged section.
• Schema validation — failure looks like: critique returns a plain string instead of the {result, reason, principle_id} object. The gate can’t parse it. Use Structured Output constraints in your DSPy Signatures to enforce output format at the API call layer.

Common Pitfalls

Pro Tip

Every critique-revise loop is a specification test harness in disguise. The loop doesn’t just fix outputs — it surfaces gaps in your constitution. Track principle fire rates across your validation set. A principle with zero fires either means your test cases don’t exercise it or the principle is too vague for the model to operationalize. Neither is acceptable in production. One well-specified principle that reliably catches a real violation beats five aspirational ones that never trigger.

This transfers to every safety specification you build. Untestable rules are unenforceable rules.

Frequently Asked Questions

Q: How do I use constitutional AI prompting for content moderation pipelines? A: Wire your moderation rules as individual principles, each with a {statement, violation_example, revision_guidance} schema. Run one critique call per principle per output — not a single omnibus call that evaluates everything at once. Common confusion: ReAct Prompting and Tree of Thoughts solve different problems — React handles tool selection, ToT handles planning. Constitutional critique enforces output principles on already-generated content. They compose; they don’t substitute for each other.

Q: How do I use self-critique prompting to reduce hallucinations in LLM outputs? A: Add a factual accuracy principle: the model must not assert claims unsupported by the input context. On critique FAIL, the revise module receives the flagged claim and the source context — it grounds the claim or removes it. Key caveat: a model critiquing its own Hallucination is not the same as retrieval-grounded fact-checking. Use this loop to reduce confabulation patterns; treat it as a complement to RAG, not a replacement.

Q: When should I choose constitutional AI prompting over RLHF for LLM safety? A: Choose constitutional AI prompting when you can’t fine-tune the model. RLHF requires labeled preference data, a training run, and weights you control. The critique-revise loop runs at inference time — deployable today via API, no training required. Trade-off: each critique call adds latency and cost per request. RLHF bakes behavior into weights, making production calls cheaper at scale. Use the loop for policy prototyping; consider RLHF when your policy is stable and inference overhead becomes prohibitive.

Q: How do I implement a constitutional AI self-critique loop step by step? A: Four components in order: (1) Constitution — {id, statement, violation_example, revision_guidance} per principle. (2) Critique — one API call per principle, returns {result, reason, principle_id}. (3) Revise — targets only the failing section of the output. (4) Gate — halts at a defined maximum iteration count, flags for human review on cap exceeded. Wire them with prompt chaining in DSPy Signatures that enforce output schemas. Pin to DSPy 3.2.1 for a stable build target.

Your Spec Artifact

By the end of this guide, you should have:

• Constitution schema — {id, statement, violation_example, revision_guidance} for each principle, starting set of five, with at least one known-violation test case per principle
• Module contracts — input/output schema plus constraint list for critique module, revise module, and gate
• Validation test set — a known-pass input and a known-fail input per principle, plus one adversarial input designed to exceed the iteration cap

Your Implementation Prompt

Copy this into Claude Code, Cursor, or your preferred AI coding tool. Fill every bracket before submitting — each placeholder maps directly to the spec work you did in Steps 1-4.

You are implementing a Constitutional AI critique-revise pipeline using DSPy 3.2.1 and Claude.

ARCHITECTURE (4 components — implement in this order):

1. Constitution loader
   - Reads principles from [your constitution file path or inline data structure]
   - Principle schema: {id: str, statement: str, violation_example: str, revision_guidance: str}
   - Starting principles: [list your principle IDs here]

2. Critique module (DSPy Signature)
   - Input: model_output: str, principle: dict
   - Output: result: Literal["PASS", "FAIL"], reason: str | None, principle_id: str
   - Constraint: returns reason=None on PASS — do not generate explanations for passing outputs
   - One principle per call — no multi-principle batching

3. Revise module (DSPy Signature)
   - Input: original_output: str, failed_principle: dict, critique_reason: str
   - Output: revised_output: str
   - Constraint: revise ONLY the section flagged in critique_reason — no full rewrites

4. Gate (loop controller)
   - Max iterations: [2 or 3 — choose one, document why]
   - On max exceeded: return original_output plus flag {"status": "review_required", "reason": "max_iterations_exceeded"}
   - On all principles PASS: return final revised_output plus flag {"status": "pass", "iterations": N}

TECH STACK:
- DSPy 3.2.1 (pin this version: pip install dspy==3.2.1)
- LM config: dspy.LM('anthropic/[claude-sonnet-4-6 or claude-haiku-4-5]', api_key=[your API key env var name])
- Python 3.10+

VALIDATION (build these alongside the implementation, not after):
- Principle coverage test: verify each principle fires FAIL on at least one known-violation input
- Iteration cap test: inject a deliberately-unfixable input, verify gate triggers at your defined maximum
- Revision scope test: verify revision diffs are minimal — targeted section only, not full rewrite
- Schema validation: verify critique output always matches {result, reason, principle_id} — test with malformed inputs

Do NOT replace this pipeline with a single system prompt. The four-component critique-revise loop IS the safety mechanism.

Ship It

You now have a four-component spec — not a prompt, a pipeline. Constitution, critique, revise, gate. Each component has a defined input contract, output contract, and constraint list. The AI coding tool knows what to build because you specified what “done” looks like for each piece. The loop catches what the single system prompt missed. That’s the delta.