Quantization is a model or data optimization technique that reduces numerical precision, groups values into buckets, or represents values with fewer bits. In machine learning, it is often used to make models smaller, faster, and less resource-intensive for inference. The tradeoff is that lower precision can sometimes reduce prediction quality.
Overloaded term that could be used in any of the following ways: - Implementing quantile bucketing on a particular feature. - Transforming data into zeroes and ones for quicker storing, training, and inferring. As Boolean data is more robust to noise and errors than other formats, quantization can improve model correctness. Quantization techniques include rounding, truncating, and binning. - Reducing the number of bits used to store a model's parameters. For example, suppose a model's parameters are stored as 32-bit floating-point numbers. Quantization converts those parameters from 32 bits down to 4, 8, or 16 bits. Quantization reduces the following: - Compute, memory, disk, and network usage - Time to infer a predication - Power consumption However, quantization sometimes decreases the correctness of a model's predictions.
Quantization is a model optimization technique used to reduce the precision of the numbers used to represent a model's parameters. This can lead to smaller models, lower power consumption, and reduced inference latency.