Fine-Tune a Model Efficiently with LoRA

The problem with full fine-tuning

Updating every weight of a large model needs enormous memory. LoRA (Low-Rank Adaptation) freezes the original weights and trains tiny "adapter" matrices instead — often <1% of the parameters.

from peft import LoraConfig, get_peft_model
config = LoraConfig(r=8, lora_alpha=16, target_modules=["q_proj", "v_proj"])
model = get_peft_model(base_model, config)
model.print_trainable_parameters()  # a fraction of the total

Why it's a big deal

LoRA makes fine-tuning possible on a single consumer GPU, produces small adapter files you can swap in and out, and rarely hurts quality. It's the default way most people customize open models today.