Fine-tuning used to mean renting a cluster of GPUs and burning through thousands of dollars. Not anymore. With techniques like LoRA and QLoRA, you can fine-tune a 7B parameter model on a single consumer GPU in a few hours.
Let me show you how to do it without the headaches.
When Should You Fine-Tune?
Before diving into code, let's be clear: fine-tuning isn't always the answer. Consider it when:
- Prompt engineering and RAG aren't giving you the quality you need
- You need consistent output format or style
- You have domain-specific knowledge the base model lacks
- You want to reduce inference costs by using a smaller, specialized model
Understanding LoRA: The Game Changer
LoRA (Low-Rank Adaptation) is brilliant in its simplicity. Instead of updating all model weights, it adds small trainable matrices to specific layers. This reduces trainable parameters from billions to millions.
# Traditional fine-tuning: Update ALL weights # Parameters: 7,000,000,000 (7B model) # VRAM needed: 28GB+ (full precision) # LoRA fine-tuning: Add small adapter matrices # Trainable parameters: ~4,000,000 (0.06% of original) # VRAM needed: 8-16GB # QLoRA: LoRA + 4-bit quantization # VRAM needed: 4-8GB (runs on consumer GPUs!)
Setting Up Your Environment
Let's set up everything you need:
pip install transformers datasets peft accelerate bitsandbytes trl
Preparing Your Dataset
The quality of your fine-tuned model depends entirely on your data. Here's how to structure it:
from datasets import Dataset
# Format: instruction-input-output pairs
training_data = [
{
"instruction": "Summarize the following customer feedback",
"input": "The product arrived late and the packaging was damaged...",
"output": "Negative feedback: Delivery delay and packaging issues."
},
{
"instruction": "Summarize the following customer feedback",
"input": "Absolutely love this product! Works exactly as described...",
"output": "Positive feedback: Product meets expectations, customer satisfied."
},
# Add 100-1000+ examples for good results
]
def format_prompt(example):
"""Format data into the prompt template."""
return f"""### Instruction:
{example['instruction']}
### Input:
{example['input']}
### Response:
{example['output']}"""
dataset = Dataset.from_list(training_data)
dataset = dataset.map(lambda x: {"text": format_prompt(x)})Fine-Tuning with QLoRA
Here's the complete fine-tuning script:
import torch
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
TrainingArguments
)
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from trl import SFTTrainer
# Model configuration
model_name = "mistralai/Mistral-7B-v0.1" # Or any HuggingFace model
# 4-bit quantization config (QLoRA)
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True
)
# Load model with quantization
model = AutoModelForCausalLM.from_pretrained(
model_name,
quantization_config=bnb_config,
device_map="auto",
trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token
# Prepare model for training
model = prepare_model_for_kbit_training(model)
# LoRA configuration
lora_config = LoraConfig(
r=16, # Rank - higher = more capacity, more VRAM
lora_alpha=32, # Scaling factor
target_modules=["q_proj", "k_proj", "v_proj", "o_proj"], # Which layers to adapt
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters() # Shows ~0.06% trainable# Training configuration
training_args = TrainingArguments(
output_dir="./results",
num_train_epochs=3,
per_device_train_batch_size=4,
gradient_accumulation_steps=4,
learning_rate=2e-4,
weight_decay=0.001,
warmup_ratio=0.03,
lr_scheduler_type="cosine",
logging_steps=10,
save_strategy="epoch",
fp16=True, # Mixed precision training
)
# Create trainer
trainer = SFTTrainer(
model=model,
train_dataset=dataset,
tokenizer=tokenizer,
args=training_args,
dataset_text_field="text",
max_seq_length=512,
)
# Train!
trainer.train()
# Save the adapter weights
model.save_pretrained("./my-fine-tuned-model")Using Your Fine-Tuned Model
Loading and using your model is straightforward:
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
"mistralai/Mistral-7B-v0.1",
device_map="auto",
torch_dtype=torch.float16
)
# Load your LoRA adapter
model = PeftModel.from_pretrained(base_model, "./my-fine-tuned-model")
tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-v0.1")
# Generate
prompt = """### Instruction:
Summarize the following customer feedback
### Input:
The delivery was super fast and the product quality exceeded my expectations!
### Response:"""
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))"By adapting a pre-trained model, fine-tuning reduces the need for training from scratch, saving time and resources."
Common Pitfalls to Avoid
1. Overfitting — If your model memorizes training data instead of learning patterns, reduce epochs or increase data diversity.
2. Poor data quality — Garbage in, garbage out. Clean your data thoroughly.
3. Wrong learning rate — Too high causes instability, too low means slow convergence. Start with 2e-4 for LoRA.
4. Catastrophic forgetting — The model forgets general knowledge. LoRA helps prevent this by keeping base weights frozen.
Fine-tuning has never been more accessible. With QLoRA, you can customize powerful language models on hardware you probably already own. Start small, iterate on your data, and you'll be surprised how quickly you can build models that outperform generic ones on your specific tasks.