18 August 2026

Smaller AI models can predict optimal training data repetition

• Researchers found that repeating high-quality training data helps larger language models learn better, but only slightly more repetition is needed as models grow.
• Smaller test models can estimate the right amount of data repetition for much larger models, potentially saving compute resources during development.
• The benefit of repeating training data holds steady across different model sizes when measured against a fixed ratio of tokens per parameter, a standard training metric.

How it was covered

TLDR AITLDR editorial team

The optimal amount of high-quality domain data repetition increased mildly with model size at a fixed tokens-per-parameter ratio. Smaller proxy models could help estimate repetition schedules for larger models.