All terms
Optimization
Prefix Tuning
Training small added vectors at every layer to steer a frozen model.
Definition
Prefix tuning adds a short sequence of trainable number vectors — the prefix — inside every layer of a frozen (unchanged) model, where the model decides what to focus on. Because it touches every layer rather than just the input, it has more expressive power than prompt tuning. Only the prefix is trained, making it parameter-efficient (cheap to tune), and it can match full fine-tuning on some tasks while updating a tiny fraction of the model.