Also known as: Parameter efficient Tuning
Parameter-efficient tuning is a family of methods for adapting a large pre-trained model by training only a small subset of additional or selected parameters. It reduces compute and storage costs while often approaching the performance of full fine-tuning.
A set of techniques to fine-tune a large pre-trained language model (PLM) more efficiently than full fine-tuning. Parameter-efficient tuning typically fine-tunes far fewer parameters than full fine-tuning, yet generally produces a large language model that performs as well (or almost as well) as a large language model built from full fine-tuning. Compare and contrast parameter-efficient tuning with: - instruction tuning - prompt tuning Parameter-efficient tuning is also known as parameter-efficient fine-tuning.