Top-P
Also called nucleus sampling — a method that selects from the smallest set of tokens whose combined probability exceeds a threshold p, adapting dynamically to the model's confidence at each step.
Top-p sampling, also called nucleus sampling, selects tokens from the smallest set whose cumulative probability reaches p. With p=0.9, the model considers only the most probable tokens that together account for 90% of the probability mass. On confident steps, this might be just 2–3 tokens; on uncertain steps, it could be hundreds.
The dynamic nature of top-p is its key advantage over top-k. When the model is highly confident about the next word (say, the article 'the' after 'dogs love to chase'), nucleus sampling with p=0.9 might restrict to just a handful of tokens. When the model has many plausible continuations, it naturally expands the candidate set.
Top-p is now the most widely used sampling method for generation tasks. Most LLM APIs default to values between 0.9 and 1.0. Values below 0.9 produce more focused text but may become repetitive. Values approaching 1.0 are nearly equivalent to no top-p filtering. Combining top-p=0.9 with temperature=0.7 is a common starting configuration for creative tasks.
Read next
Temperature and Sampling: Controlling LLM Creativity
A clear explanation of temperature, top-p, top-k, and how sampling parameters control the balance between determinism and creativity in LLM outputs.
Sampling Strategies: How LLMs Choose the Next Word
Every token an LLM generates is chosen via a sampling strategy. Understanding temperature, top-p, and top-k reveals how models balance quality and creativity.
Top-K Sampling: Limiting Randomness in Text Generation
Top-K sampling restricts token selection to the K most probable options at each step, balancing quality and diversity in LLM outputs.