基于Web的大规模语料库构建方法  

Construction Approach of Large-scale Corpus Based on Web

作　　者：李培峰[1] 朱巧明[1] 钱培德[1] 

机构地区：[1]苏州大学计算机科学和技术学院,苏州215006

出　　处：《计算机工程》2008年第7期41-43,46,共4页Computer Engineering

基　　金：国家自然科学基金资助项目(60673041);江苏省高技术研究基金资助项目(BG2005020);江苏省自然科学基金资助项目(BK2003030)

摘　　要：低成本、短周期构建大规模语料库是目前研究工作的难点之一。该文提出一种建设大规模语料库的新方法,主要解决如何基于Web构建大规模的语料库及对语料库纠错,从而提高其质量。该方法利用网格技术的大规模计算能力与Wiki的开放编辑环境去收集和处理语料,根据可信度模型挑选出不可信的语料并由人工进行校对,计算校对后结果的可信度,选择出最可信的结果作为正确语料存储到语料库中。Nowadays, it’s hard to build a large-scale corpus with low cost and short period. A new approach is provided to build that on Web. It mainly focuses on how to build a large-scale corpus on Web and then how to correct the mistakes in the corpus. The language materials are collected and processed based on grid and Wiki. The untrustworthy language materials in the corpus are picked out to be checked manually on Wiki according to their trustworthiness. After the check finishes, the approach calculates the trustworthiness of each checked result and selects ones with highest trustworthiness as the correct result.

关 键 词：大规模语料库 网格 可信度 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]