Fine-tune a model with Hugging Face PEFT

By Ryze AI Team

This tutorial fine-tunes a small language model with a LoRA adapter using the PEFT library. It covers setup, training, saving, and loading the adapter.

Install

pip install transformers peft datasets accelerate torch. A GPU is strongly recommended; CPU training is slow but possible for tiny models.

Load model and apply LoRA

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model

model_name = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["c_attn"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)
model = get_peft_model(model, config)
model.print_trainable_parameters()

target_modules depends on the architecture; for Llama-style models use q_proj, v_proj, and others. print_trainable_parameters shows that only a small fraction of parameters are trainable.

Prepare data

from datasets import load_dataset
dataset = load_dataset("json", data_files="train.jsonl")
def tokenize(ex):
    return tokenizer(ex["text"], truncation=True, max_length=512)
dataset = dataset.map(tokenize, batched=True)

Train

from transformers import TrainingArguments, Trainer
args = TrainingArguments(output_dir="./out", num_train_epochs=2, per_device_train_batch_size=4, learning_rate=2e-4, fp16=True)
trainer = Trainer(model=model, args=args, train_dataset=dataset["train"])
trainer.train()
model.save_pretrained("./adapter")

Use the adapter

from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained(model_name)
model = PeftModel.from_pretrained(base, "./adapter")
# Or merge for simpler serving:
model = model.merge_and_unload()

Tips

Start with r=8 or 16. Train one to three epochs and watch eval loss. If general capability degrades, reduce epochs or learning rate. Evaluate on both the target task and a general benchmark to catch forgetting. Use Unsloth for faster training on consumer GPUs.

Docs: PEFT, Unsloth.