
Inside Code LLMs: Fill-in-the-Middle and the Training Data Behind Them
Fill-in-the-middle reorders code into prefix-suffix-middle triplets, letting code LLMs like StarCoder 2 complete code using context after the cursor.
This theme is curated by our AI council — see how it works.
AI coding assistants are development tools that use large language models to write, review, test, document, and repair code alongside a human developer. The theme spans the whole software lifecycle — from the inline suggestion that appears as you type, through the pull request and the CI pipeline, to the long maintenance game of debugging, refactoring, and technical debt. This page maps that lifecycle: what to read first, what each tool class actually does, and where the categories blur into each other.
This is the AI domain that reaches developers first, whether they choose it or not: the assistant is already in the IDE, the review bot is already on the pull request, and team policy questions about AI-written code arrive before anyone has read a paper. For an experienced engineer the real work is not learning to prompt — it is deciding which parts of the workflow to delegate, how to verify what comes back, and where responsibility lands when generated code breaks in production. Each topic below is one of those decisions.

Two topics carry everything else in this theme, because every tool in the later tiers — the reviewer commenting on your PR, the agent rewriting a module — is a different wrapper around the same core capability.
The visible entry point is AI code completion: the model predicting your next tokens inline as you type. What AI code completion is and how LLM-powered inline suggestions predict the next token is the best first read in the theme, because it establishes the mechanism every other tool class reuses; when you want it running in your own editor, the Cursor Tab, GitHub Copilot, and Continue setup guide covers the practical trade-offs between the main options.
Beneath the editor sits the layer most developers never pick consciously: code LLMs, the models trained or fine-tuned specifically on source code. What code LLMs are and how CodeLlama, DeepSeek Coder, and StarCoder understand source code explains what separates them from general-purpose models, and if your constraints point toward running your own — privacy, cost, air-gapped environments — the self-hosting and fine-tuning guide with Qwen3-Coder, DeepSeek Coder, and Ollama walks the whole stack.
With these two, every other topic in the theme reads as a variation: same model family, different point in the workflow, different blast radius when it is wrong.
This tier is where AI assistance leaves the individual editor and enters the team’s shared workflow — which is also where its output starts binding other people.
The first station past the commit is AI code review: bots that read a pull request and comment before a human does. How LLM-powered PR reviewers catch bugs before humans is the orientation read, and integrating Qodo, CodeRabbit, and Greptile into a GitHub workflow turns it into a working setup. Review’s natural companion is AI test generation — models writing the unit and integration tests that guard the code. Start with how LLMs write tests from existing code, then use the Qodo Cover-Agent, Diffblue, and Claude Code guide when you want generated tests that are worth keeping rather than coverage theater.
The quietest win in the tier is AI documentation generation — docstrings, API references, and architecture docs produced from the code itself. How LLMs turn code into docstrings, API references, and architecture docs maps the approaches, and the Mintlify, Swimm, and DocuWriter guide to living docs shows how to keep generated documentation from going stale the week after it is written. Finally, all of these converge in AI in CI/CD pipelines, where analysis, test prioritization, and deployment risk checks run automatically on every change — how automated code analysis and deployment checks work is the entry point, and adding AI test prioritization and PR review to your pipeline is the hands-on route.
Once these four are in place, AI is no longer a typing aid — it is part of the machinery that decides what merges. That is exactly the posture you need before entering the maintenance tier, where the changes get bigger and the supervision gets thinner.
The maintenance tier is where AI coding assistance is hardest and most valuable: the code already exists, the context is large, and being subtly wrong is worse than being obviously wrong.
AI-assisted debugging puts the model on the other side of the failure — reading stack traces and error context to locate and explain bugs. How LLMs analyze stack traces to find bugs covers the mechanism, and debugging production bugs with Claude Code, Cursor, and Copilot Chat is the workflow read for real incidents rather than toy examples. One step more ambitious is AI-assisted refactoring: restructuring code across many files while preserving behavior. How agentic tools restructure code without breaking it explains what makes this categorically harder than generating new code, and refactoring a legacy codebase with Claude Code, Cursor, and Aider is the guide for the day you point an agent at code nobody wants to touch.
The tier closes at the portfolio level with AI for technical debt — using models to find code smells and hotspots and to decide which refactoring actually pays. How machine learning detects code smells and hotspots is the orientation read, and prioritizing refactoring with SonarQube and CodeScene quality gates makes it operational. There is a loop hiding in this tier worth noticing: the same assistants that generate code quickly also generate debt quickly, and the debt tooling increasingly exists to clean up after them.
The costliest confusion in this theme is treating “AI writes code” as one capability. The three classes below share a model but not a contract:
| Code completion | AI code review | AI-assisted refactoring | |
|---|---|---|---|
| When it acts | As you type | At the pull request | On existing code, on demand |
| Scope of change | Next tokens, one cursor | Comments only — changes nothing | Multi-file, structural |
| Human checkpoint | You accept each suggestion | You resolve each comment | Often only the final diff |
| Failure mode | Plausible-looking wrong line | Missed bug or noisy nitpicks | Silent behavior change |
Three finer distinctions trip people just as often:
Q: Where should I start with AI coding assistants as an experienced developer? A: Start at the foundations tier: completion first, because it teaches you the model’s strengths and failure patterns at the lowest possible cost — a rejected suggestion costs seconds. The completion setup guide gets you a working baseline before you delegate anything that binds your team.
Q: Do I need a dedicated code LLM, or is a frontier model enough? A: For most IDE and review use, the frontier model behind your existing tool is enough. A dedicated or self-hosted code model earns its place with privacy requirements, cost at scale, or fill-in-the-middle completion workloads — the 2026 dedicated-vs-frontier comparison breaks down where each wins.
Q: Should AI be allowed to merge changes without a human in the loop? A: Treat autonomous merge as a policy decision, not a tool feature — the failure cost lands on whoever owns the pipeline, not the bot. Who’s accountable when AI auto-merges a broken fix works through the accountability gap before you flip that switch.
Q: Can AI-generated tests be trusted to validate AI-generated code? A: Not by themselves — a model asserting that its own kind of output is correct inherits the same blind spots twice. Generated tests need an independent signal, such as mutation testing or human-owned acceptance cases; the accountability gaps in automated test generation explains why the loop cannot close itself.
Q: What should I read before letting an agent refactor a legacy codebase? A: The refactoring prerequisites: behavior preservation only works when test coverage and AST-level awareness are in place first, and legacy code usually lacks both. The prerequisites for AI-assisted refactoring lists what must exist before the agent touches anything.
Q: Why is my codebase degrading even though every AI-written change passed review and tests? A: Because review and tests check changes one at a time, while debt accumulates across them — duplication, drift, and hotspots that no single diff reveals. That gap is exactly what codebase-level analysis exists for; the AI-generated-debt surge and agentic refactoring covers why fast generation makes it worse.
AI code completion is the technology behind real-time, inline suggestions that appear as a developer types in an editor. …
AI code review uses large language models to automatically inspect pull requests, flag likely bugs, suggest fixes, and …
AI documentation generation uses large language models to read source code and automatically produce docstrings, API …
AI for technical debt uses machine learning to find, measure, and prioritize the messy or aging parts of a codebase — …
AI in CI/CD pipelines means adding machine learning and language models to your build, test, and deployment automation. …
AI test generation uses large language models to automatically write unit tests, integration tests, and edge case …
AI-assisted debugging uses large language models to read error messages, stack traces, and surrounding code so they can …
AI-assisted refactoring uses large language models to restructure, simplify, and improve existing code while preserving …
Code LLMs are large language models trained or fine-tuned specifically to read, understand, and generate source code. …
MONA's articles build your mental model — how things work, why they work that way, and what intuition to develop.
Updated May 31, 2026
Concepts covered

Fill-in-the-middle reorders code into prefix-suffix-middle triplets, letting code LLMs like StarCoder 2 complete code using context after the cursor.

Code LLMs are transformers trained on billions of code tokens, not prose. Fill-in-the-Middle training lets them complete code from both directions.

Inline AI code completion rests on four prerequisites: tokenization, context windows, FIM, and speculative decoding. Each explains a failure mode.

AI code completion uses Fill-in-the-Middle prompting on small, latency-optimized LLMs that score tokens against code before and after the cursor.

AI in CI/CD requires a deterministic pipeline-as-code foundation first. Its main failure mode: misclassifying real regressions as flaky tests.

AI in CI/CD pipelines uses ML trained on build history to prioritize tests, predict build failures, and score deployment risk as a forecast.

AI documentation generators don't read code — they parse ASTs, embed chunks, and retrieve context. Open-source models invent 21.7% of imported packages.

AI documentation generation uses LLMs to produce docstrings, API references, and architecture docs by traversing code dependencies in topological order.

Coverage measures whether tests run code. Mutation testing measures whether assertions catch bugs. AI test generators optimize for the wrong signal.

AI test generation uses LLMs to write unit tests from source code. A two-phase pipeline produces candidates, then filters for compile, pass, and coverage delta.

AI code review uses LLM agents to inspect pull requests for bugs, security flaws, and architectural drift before human reviewers see them.

AI code review combines retrieval-augmented context, static analysis, and LLM triage. Without grounding, models hallucinate one in five package names.

AI technical-debt tools measure proxies like complexity and code churn, not debt itself. Vendor false-positive rates near 3% clash with study findings.

AI for technical debt combines ML and behavioral code analysis to find decay hotspots — a small fraction of files drives most defects in a codebase.

AI-assisted refactoring needs four prerequisites: behavior preservation, AST awareness, test coverage, and mitigation of hallucinations and context drift.

AI-assisted refactoring parses code into a syntax tree, plans behavior-preserving edits, and verifies them with tests — not text find-and-replace.

AI debuggers sample probable fixes from a finite context window. Even top 2026 models invent plausible symbols, the dominant failure mode for code errors.

AI-assisted debugging uses LLMs to parse stack traces, generate fix hypotheses, and validate them against runtime evidence inside an agent loop.