硬件有限环境中低资源文本分类新策略  

Low⁃resource text new strategy classification in hardware⁃limited environments

作　　者：尹照煜 宋文爱[1] 刘宏昊 YIN Zhaoyu;SONG Wenai;LIU Honghao(School of Software,North University of China,Taiyuan 030051,China)

机构地区：[1]中北大学软件学院,山西太原030051

出　　处：《现代电子技术》2025年第8期56-62,共7页Modern Electronics Technique

基　　金：山西省研究生实践创新项目(2023SJ223)。

摘　　要：针对在硬件资源有限的情况下,难以有效提升低资源分类任务性能这一难题,提出使用基于新一代大型语言模型Claude3 Haiku的数据增强,并设计了一种更小更快的文本分类模型EQSBERT。首先基于Claude3 Haiku,使用分布式比例增强法和动态均衡过采样增强法来增强低资源文本分类任务数据集。通过二次自蒸馏、多目标知识蒸馏、多维剪枝,提出一种更小更快的BERT系列文本分类模型EQSBERT,采用EQSBERT对增强后的数据集进行分类。结果表明:Claude3 Haiku具有较好的性能,且成本效益高于GPT-4;EQSBERT在参数大幅减少的情况下也能维持其高性能,显著降低了运行成本。两者结合搭配使用,可以有效解决在硬件条件有限的情况下低资源分类任务方面的问题。该方案为资源受限情况下的自然语言处理任务提供了新的解决策略,在自动化数据标注、社交媒体监控以及内容审核系统应用方面有巨大潜力。In allusion to the problem that it is difficult to effectively improve the performance of low-resource classification tasks under the condition of limited hardware resources,a data enhancement based on the new-generation large language model Claude3 Haiku is proposed,and a smaller and faster text classification model EQSBERT is designed.Based on Claude3 Haiku,the distributed scale enhancement method and dynamic equalization oversampling enhancement method are used to enhance the low-resource text classification task dataset.A smaller and faster BERT text classification model EQSBERT is proposed by means of secondary self-distillation,multi-objective knowledge distillation and multi-dimensional pruning,and EQSBERT is used to classify the enhanced dataset.The results show that Claude3 Haiku has better performance and higher cost-effectiveness than GPT-4;EQSBERT can maintain its high performance under the condition of greatly reduced parameters,which significantly reduces operating costs.The combined use of both approaches effectively addresses the issue of low-resource classification tasks under limited hardware conditions.This solution can provide a novel strategy for natural language processing tasks in resource-constrained environments,with significant potential applications in automated data annotation,social media monitoring,and content moderation systems.

关 键 词：自然语言处理 低资源文本分类 大型语言模型Claude3 Haiku 数据增强 GPT-4 多目标知识蒸馏 

分 类 号：TN919.72-34[电子电信—通信与信息系统] TP391.1[电子电信—信息与通信工程]