Knowing when your fine-tuned model is actually better than the base model requires systematic evaluation. This guide covers benchmark datasets, LLM-as-judge evaluation, and metrics for task-specific assessment.
Before training, define what 'better' means for your task. Is it lower perplexity on your validation set? Higher accuracy on specific test cases? Better human preference ratings? Without a clear definition, you cannot know if fine-tuning helped.
Create a held-out test set of 50–200 examples that the model will never train on. This is your ground truth for evaluation. Run the base model on it before fine-tuning to establish a baseline.
For classification tasks: accuracy, F1 score, and confusion matrix. These are unambiguous and fully automated.
For generation tasks: ROUGE (overlap with reference text) and BLEU are traditional but correlate poorly with human judgment. BERTScore (semantic similarity) is a better choice for tasks where there are multiple valid phrasings.
Perplexity on your validation set is a fast sanity check: lower is better and signals the model has learned the target distribution.
The most useful evaluation for generation quality is using a strong LLM (GPT-4o or Claude Opus) to compare your fine-tuned model's outputs against the base model, side by side. Present both outputs without labels (blind) and ask the judge: 'Which response better answers the question and follows the correct format? Respond with A or B and a one-sentence reason.'
Run this comparison on 50–100 test cases and compute the win rate: what percentage of the time did the fine-tuned model win? A win rate above 65% indicates a meaningful improvement. Below 55% suggests the fine-tuning had minimal effect or may have caused regression.
Fine-tuning for a specific task can degrade general capabilities — this is called catastrophic forgetting. Always test the fine-tuned model on a few general-knowledge and instruction-following examples that are unrelated to the fine-tuning task.
Use Eleuther AI's `lm-evaluation-harness` to run standard benchmarks on both the base and fine-tuned models: `python -m lm_eval --model hf --model_args pretrained=my-fine-tuned-model --tasks mmlu,hellaswag,arc_easy`. Any benchmark drop of more than 5% suggests the fine-tuning was too aggressive.
Unsloth is an optimised fine-tuning library that makes LoRA training 2–5× faster with 70% less VRAM. This guide walks through fine-tuning a Llama or Mistral model on your own data using Unsloth on a free Google Colab GPU.
Read guideThe quality of your training data is the biggest factor in fine-tuning success. This guide covers data collection strategies, formatting standards, quality filtering, and the minimum viable dataset size for different tasks.
Read guideDPO trains models to prefer good responses over bad ones using human preference data — without the complexity of reinforcement learning. This guide covers collecting preference data, training with TRL's DPO trainer, and evaluating results.
Read guide