Optimizing BERT for Bengali Emotion Classification: Evaluating Knowledge Distillation, Pruning, and Quantization  

作　　者：Md Hasibur Rahman Mohammed Arif Uddin Zinnat Fowzia Ria Rashedur M.Rahman 

机构地区：[1]Department of Electrical and Computer Engineering,North South University,Dhaka,1229,Bangladesh

出　　处：《Computer Modeling in Engineering & Sciences》2025年第2期1637-1666,共30页工程与科学中的计算机建模(英文)

摘　　要：The rapid growth of digital data necessitates advanced natural language processing(NLP)models like BERT(Bidi-rectional Encoder Representations from Transformers),known for its superior performance in text classification.However,BERT’s size and computational demands limit its practicality,especially in resource-constrained settings.This research compresses the BERT base model for Bengali emotion classification through knowledge distillation(KD),pruning,and quantization techniques.Despite Bengali being the sixth most spoken language globally,NLP research in this area is limited.Our approach addresses this gap by creating an efficient BERT-based model for Bengali text.We have explored 20 combinations for KD,quantization,and pruning,resulting in improved speedup,fewer parameters,and reduced memory size.Our best results demonstrate significant improvements in both speed and efficiency.For instance,in the case of mBERT,we achieved a 3.87×speedup and 4×compression ratio with a combination of Distil+Prune+Quant that reduced parameters from 178 to 46 M,while the memory size decreased from 711 to 178 MB.These results offer scalable solutions for NLP tasks in various languages and advance the field of model compression,making these models suitable for real-world applications in resource-limited environments.

关 键 词：Bengali NLP black-box distillation emotion classification model compression post-training quantization unstructured pruning 

分 类 号：TP391.41[自动化与计算机技术—计算机应用技术]