Fill-in-the-Middle

Fill-in-the-middle (FIM) is a technique that teaches a language model to generate the missing middle of a sequence using both the text that comes before it and the text that comes after it, instead of only continuing forward from left to right.

What It Is

Standard language models are trained to predict the next token, so they only see context to the left of where they are writing. That is a poor fit for coding, where the cursor usually sits between existing code, with relevant logic on both sides. FIM solves this by rearranging training examples into a prefix, a suffix, and the middle to be filled, so the model learns to condition on both directions.

This is the mechanism behind much of modern inline code completion. When an editor suggests code at your cursor, a FIM-trained model is reading the lines above and below to produce a suggestion that fits the surrounding code, not just a plausible continuation of what you typed last. The same idea applies to infilling missing sections in any structured text.

One Sentence to Remember

Fill-in-the-middle lets a model write into a gap using context on both sides, which is why code assistants can complete code at your cursor rather than only at the end of a file.