Axolotl is a config-driven fine-tuning framework that wraps HuggingFace and Unsloth with a single YAML configuration. This guide covers setting up Axolotl, writing a config, and running multi-GPU training.
Axolotl abstracts the complexity of fine-tuning setup into a single YAML config file. Instead of writing Python training scripts, you describe your model, dataset, LoRA settings, and training parameters in YAML. Axolotl handles dataset formatting, model loading, distributed training setup, and checkpoint saving.
Axolotl supports every major fine-tuning technique: LoRA, QLoRA, full fine-tuning, and DPO — all switchable by changing a few config lines. It also integrates with Unsloth for 2–5× faster training on supported models.
Clone and install: `git clone https://github.com/axolotl-ai-cloud/axolotl && cd axolotl && pip install -e '.[flash-attn,deepspeed]'`. Flash Attention 2 is strongly recommended — it speeds up training by 30–50% and reduces memory usage.
Verify GPU setup: `python -c 'import torch; print(torch.cuda.is_available(), torch.cuda.device_count())'`. Axolotl works on 1 to 8+ GPUs. For multi-GPU, install DeepSpeed: `pip install deepspeed`.
Create a YAML config file. Minimal QLoRA example: `base_model: mistralai/Mistral-7B-Instruct-v0.3; load_in_4bit: true; adapter: qlora; lora_r: 16; lora_alpha: 32; datasets: [{path: my_dataset, type: alpaca}]; num_epochs: 3; micro_batch_size: 2; gradient_accumulation_steps: 4; output_dir: ./output`.
The `type` field in `datasets` specifies the formatting. Common values: `alpaca` (instruction/input/output), `sharegpt` (chat format with turns), `completion` (raw text). Axolotl automatically applies the correct chat template for each model family.
Single GPU: `axolotl train config.yaml`. Multi-GPU with DeepSpeed: `axolotl train config.yaml --deepspeed deepspeed_configs/zero2.json`. Axolotl saves checkpoints to `output_dir` and logs to Weights & Biases if configured.
For large models or long training runs, enable gradient checkpointing (`gradient_checkpointing: true`) to trade compute for memory. This allows training models that would otherwise OOM, at the cost of ~30% slower training speed.
After training, merge the LoRA weights into the base model: `axolotl merge-lora config.yaml --lora-model-dir ./output`. This creates a full model that can be deployed without the adapter overhead.
Export to GGUF for local deployment: `python convert_hf_to_gguf.py ./merged-model --outtype q4_k_m --outfile ./my-model-q4.gguf`. Then load it with Ollama: `ollama create my-model -f ./Modelfile` where the Modelfile points to your GGUF file.
Unsloth is an optimised fine-tuning library that makes LoRA training 2–5× faster with 70% less VRAM. This guide walks through fine-tuning a Llama or Mistral model on your own data using Unsloth on a free Google Colab GPU.
Read guideThe quality of your training data is the biggest factor in fine-tuning success. This guide covers data collection strategies, formatting standards, quality filtering, and the minimum viable dataset size for different tasks.
Read guide