2026-10-11T01:34:47Z https://www.scidb.cn/oai

10.57760/sciencedb.42815 2026-07-20T17:59:42Z

2026-07-20 Constraint-Free Post-Training Quantization for Low-Bit Transformers-raw data doi:10.57760/sciencedb.42815 en The prohibitive parameter counts and computational overhead of Transformer models hamper their deployment on resource-constrained edge devices, thereby restricting their real-world applications. Quantized low-bit Transformers demonstrate a competitive edge by requiring less storage and delivering significantly accelerated inference. However, mainstream methods are limited by strict constraints such as fixed quantization intervals, thus leading to large quantization errors (i.e., clipping and rounding errors) and substantial performance degradation. To address this issue, we propose Constraint-Free Post-training Quantization (CFQuant), which enables flexible quantization by modeling activation distributions through three key steps. First, CFQuant adaptively estimates the density of activation distributions and minimizes quantization errors through iterative search during calibration. Second, an Efficient Scale-shift Algorithm (ESA) is designed to dynamically adjust activation distributions, reducing the distribution shift between the calibration and inference stages. Moreover, Matrix Multiplication with Lookup Table (MM-LUT) is designed to accelerate inference, where floating-point multiplications are converted into pre-computed lookup operations during inference, thereby bringing substantial efficiency gains. Extensive experiments on vision, language, and multimodal tasks across various Transformermodels demonstrate the flexibility and effectiveness of CFQuant. Deep learning; post-training quantization; Transformer; quantization error; self-attention jiang jia hao Yin Peng Wang Xuanhan Zeng Pengpeng Song Jingkuan PUBLIC https://creativecommons.org/licenses/by/4.0/ dataset Science Data Bank