一种基于词树的高效解码算法  

An Efficient Decoding Algorithm Based on Word Tree

作　　者：张志强[1] 张太红[1,2] 董峦[1,3] 

机构地区：[1]新疆农业大学计算机与信息工程学院,新疆乌鲁木齐830052 [2]中国农业大学信息与电气工程学院,北京100083 [3]河海大学计算机与信息工程学院,江苏南京210098

出　　处：《计算机技术与发展》2017年第8期43-46,51,共5页Computer Technology and Development

基　　金：新疆维吾尔自治区科技计划项目(2015X0106)

摘　　要：音字转换是汉语言信息处理的一个重要方面,在语音识别、汉语拼音输入等方面都有广泛的应用。为了找到一种行之有效的音字转换解码算法,在研究拼音分词与词树理论并分析词树求解过程的基础上,提出了基于语言模型实现音字转换的高效解码算法。该算法采用零概率重估、路径剪枝和多音字处理等多项技术,通过对词树进行的剪枝处理、对常用词的处理以及对解码过程中所产生多音字的处理,实现了普遍意义上的音字转换。为验证所提算法的有效性和可行性,基于新疆维吾尔自治区科技计划项目《多语种民族特色文化信息资源处理及共享服务平台》所提供的三组数据进行了对比实验。实验结果表明,提出的新算法取得了97.78%的转换准确率,优于其他传统算法。Phonetic conversion is an important aspect of Chinese language information processing, which has been widely used in speech recognition, Chinese Pinyin input and so on. In order to find an effective syllable-to-character decoding algorithm, an efficient decoding algorithm is proposed based on the study of phonetic word segmentation, the word tree theory and the analysis of word tree solving. It uses zero probability reassessment,path pruning, processing of polyphonic words to realize the syllable-to-character conversion generally by pruning of word tree, processing of common words and processing of polyphonic words in the decoding process. In order to verify the va- lidity and feasibility of the proposed algorithm, the contrast experiments on three sets of data provided by Xinjiang Uygur Autonomous Region Science and Technology Program, Multilingual Ethnic Cultural Information Resource Processing and Sharing Service Platform, have been conducted. The experimental results show that it has achieved 97.78% conversion accuracy, which is superior to other tradi- tional algorithms.

关 键 词：拼音分词 词树 语言模型 N-GRAM模型 音字转换 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]