What is LoRA?
Fine-Tuning
LoRA (Low-Rank Adaptation) — A parameter-efficient fine-tuning technique that freezes the original model weights and trains small rank-decomposition matrices. Reduces trainable parameters by 10,000x while maintaining quality.
FAQ
What is LoRA?
A technique to fine-tune LLMs cheaply by only training small adapter matrices instead of all model weights.
LoRA vs full fine-tuning?
LoRA uses 10,000x fewer parameters, trains in hours instead of days, and produces models nearly as good as full fine-tuning.
What is QLoRA?
LoRA applied to a 4-bit quantized base model. Even more memory efficient — fine-tune a 65B model on a single GPU.
Related Terms
Learn LoRA in depth
Free hands-on course with code examples and Google Colab notebooks.
Start Course →