What inference means
AI models live two very different lives. During training, a model learns: it processes enormous datasets over weeks or months and its internal weights change. During inference, a model performs: its weights are frozen, and it applies what it learned to generate an output for a specific input. Every ChatGPT reply, every Perplexity answer, every Gemini response is an act of inference — the trained model, running.
The word comes from statistics: the model infers the most plausible continuation given its input. In practice, "inference" covers everything that happens between a user hitting enter and the finished answer appearing — which makes it the moment where AI visibility is actually decided, one answer at a time.
What happens during inference
For a modern AI engine answering a category question, inference is a pipeline, not a single step:
- Input assembly. The user's prompt, system instructions, and conversation history are gathered.
- Optional retrieval. Grounded engines search the web — often expanding the question via query fan-out — and pull selected sources into the context window.
- Generation. The model produces the answer token by token, each token chosen from a probability distribution over possible next tokens.
- Post-processing. Citations get attached, formatting applied, safety filters run.
Step three explains a property every GEO practitioner must internalize: generation is sampled, not looked up. Because tokens are drawn from probability distributions (controlled by settings like temperature), the same prompt can yield different answers on different runs — naming your brand at 10 a.m. and omitting it at noon, with nothing about the world having changed.
Why inference matters for GEO
Three practical consequences fall out of inference's nature:
- Answers are probabilistic, so visibility is a rate, not a fact. "Does ChatGPT mention us?" has no yes/no answer — only a frequency across repeated runs. That's why credible measurement relies on scheduled, repeated sampling (prompt tracking) rather than one-off spot checks, and why a single screenshot of a good answer proves little.
- Inference is where your influence cashes out. Everything GEO optimizes — training-data footprint, citable content, entity consistency — exists to tilt those next-token probabilities toward your brand at the moment of generation.
- Inference costs shape engine behavior. Serving answers costs real compute per token, which is why engines summarize sources aggressively, cap answer lengths, and select few sources — the economics behind many of the selection pressures your content must survive.
Inference vs training: the two clocks
The training/inference split is also GEO's two-timeline model. Training determines what the model knows by heart and changes only when providers retrain — quarters apart, gated by a knowledge cutoff. Inference happens billions of times a day and, for retrieval-grounded engines, reads the current web each time. Content you publish today can change inference-time answers within days, while your training-data footprint catches up over release cycles — the practical timelines are laid out in how long GEO takes to work.