一种集成的字符集检测方法研究  

Research on an Integrated Character Set Detecting Method

作　　者：张卫丰[1] 周国强[1] 张迎周[1] 

机构地区：[1]南京邮电大学计算机学院,江苏南京210003

出　　处：《南京邮电大学学报（自然科学版）》2008年第5期15-19,共5页Journal of Nanjing University of Posts and Telecommunications：Natural Science Edition

基　　金：国家自然科学基金青年基金(60503020;60703086;60503033);苏州大学江苏省计算机信息处理技术重点实验室基金(KJS0714);南京邮电大学科研基金(NY207052;NY207082);Microsoft Research Asia Internet Services Theme 2008;江苏省自然科学基金(BK2006094)资助项目

摘　　要：有些文本缺少字符集信息,因而阻碍了字符信息的正确解析。为了获得字符集信息可以根据文本中字符信息的统计特征以及各种字符集之间的差异来识别出文本的字符集。集成的字符集检测方法通过综合应用编码模式检测方法、字符分布方法和双字符序列检测方法来检测字符集。它根据算法复杂度和每种算法的准确率来调度算法,使得在用一种方法不能确定字符集的情况下利用其它方法来检测出字符集实验结果表明:集成的字符检测技术具有较高的准确率。集成的字符检测方法在某些情况下通过适度增加一些时间来获得较高的精度。For the lack of character set information, the character information of some texts can not be correctly gotten. Based on the statistical information in the text characters and the differences between the character sets, the character set of text can be automatically detected. In the integrated character set detecting method, the methods like coding mode, character distribution, and dual-character sequence detection are comprehensively used to detect character set of text. The character detection methods can be scheduled in the integrated character set detecting method according to complexity and accuracy of each algorithm. Other methods of character detecting can be used to detect character set when a method can not make sure the character set. Our experiment shows that the integrated character set detection technology has higher accuracy. The integrated character set detection method can improve the accuracy in certain circumstances through appropriate time increase.

关 键 词：字符集 字符集检测 编码模式 字符分布 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]