维吾尔语网站识别方法  

Uyghur website identification

作　　者：阿力木.木拉提 艾孜尔古丽[1] 玉素甫.艾拜都拉 

机构地区：[1]新疆师范大学计算机科学技术学院,新疆乌鲁木齐830054 [2]中国科学院新疆理化技术研究所,新疆乌鲁木齐830011

出　　处：《计算机工程与设计》2016年第5期1417-1420,F0003,共5页Computer Engineering and Design

基　　金：新疆维吾尔自治区自然科学基金项目(2014211A045);新疆维吾尔自治区哲学社会科学研究规划基金项目(14CYY093);教育部人文社会科学一般基金项目(14YJC740001);新疆维吾尔自治区高校科研计划青年教师科研启动基金项目(20140706213103147);国家自然科学基金重点项目(61132009);国家自然科学基金项目(61262066);国家社科基金重点项目(14AZD11)

摘　　要：通过调查与分析大量维吾尔文网页结构,对互联网中现有的维吾尔文网页进行网页识别研究。探讨维吾尔文不规范网页的字符编码转换方法,为此进行维吾尔文编码标准化处理。考虑维吾尔文网页独有的节点属性,提出一种基于维吾尔文网页文本节点特征的方法,但该方法显示略低的准确率及较多的误判率。结合维吾尔语常用候选词语料和向量空间模型(vector space module),提出一种基于维吾尔语常用词统计学特征的方法,利用测试网页内容和维吾尔语常用候选词语料,并将其以向量来表示,计算测试网页文本的词语在维吾尔语常用候选词中的相应权值,依据相应权值来计算测试网页为维吾尔文网页的概率,进行网页识别。对两种方法进行比较,准确率分别是69.85%和91%。实验结果表明,基于维吾尔语常用词统计学特征的方法比网页文本节点特征的方法具有更高的准确率,验证了该方法的有效性。After investigating and analyzing large quantity of Uyghur website＇s structure,identification to Uyghur website on the internet was studied.A conversion method for character encoding of nonstandard Uyghur website was proposed and implemented.Taking unique node attributes of Uyghur website,a web text node basis method was put forward,but it showed slightly lower accuracy and major error rate.A method of Uyghur common word statistics was proposed by combining Uyghur most frequent terms and vector space module.Web text and Uyghur most frequent terms were used and represented as vectors,corresponding weights of words that in current webpage occurred in Uyghur most frequent terms were computed,the probability that current test page was a Uyghur website was generated according to its weight.Comparing two methods,the accuracies are69.85% and 91% respectively.Experimental results verify the Uyghur common word statistics method is highly effective and feasible than web text node method.

关 键 词：维吾尔文 网页文本节点 向量空间模型 网页识别 常用词 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]