融合汉字多语义与文本统计特征的中文医学新词发现研究  

Chinese Medical New Word Detection by Chinese Character’s Multi-Semantic Word Vector and Statistical Text Features

作　　者：王巍洁 任慧玲[1] 李晓瑛 王勖 张颖[1] Wang Weijie;Ren Huiling;Li Xiaoying;Wang Xu;Zhang Ying(Institute of Medical Information/Medical Library,Chinese Academy of Medical Sciences&Peking Union Medical College,Beijing 100020)

机构地区：[1]北京协和医学院/中国医学科学院医学信息研究所/图书馆,北京100020

出　　处：《图书情报工作》2024年第6期119-128,共10页Library and Information Service

基　　金：科技创新2030-“新一代人工智能”重大项目“面向医学人工智能服务的知识体系构建和应用研究”中文医学术语体系构建(项目编号:2020AAA0104901)研究成果之一。

摘　　要：[目的/意义]为提高机器理解医学文本的能力,提高医学自然语言处理等上层任务效果,保障医学知识内容更新及时性、覆盖完整性,提出一种融合汉字多语义信息与文本统计特征的医学新词发现方法。[方法/过程]以规范用词的医学文献摘要数据为新词发现来源,基于N-gram模型获取N元词串,将词串存入字典树,从词的内部凝固度、词的自由程度、词的语义相似度3个角度同时计算每个N-gram词串的关联置信度、左右邻接熵、多语义相似度(包括汉字细粒度字符语义信息、BERT词向量信息),遍历上述各指标阈值评估N-gram词串为医学新词的可能。[结果/结论]从中华医学会收录的截至2022年10月20日的最新1000篇文摘中发现医学新词3263个,去除重复项后,共获得764个医学新词。提出的融合汉字多语义与文本统计特征的医学新词发现方法对比现有方法具有一定提升,且在应用上可以有效提高医学分词任务效果,使医学分词后的名词类别更清晰、概念更明确、内涵更丰富。结合汉字内在多语义信息与字词外部统计特征的医学新词发现方法,不仅可以提高计算机的新词发现能力,还可提高计算机面对专业且复杂的医学文本自然语言处理效果,对及时更新领域知识内容等具有重要帮助。of medical literature with canonical words as the source of new word detection,the paper obtained N-gram word string based on the N-gram model and stored it into the dictionary tree.From the word’s internal solidification degree,the freedom degree,and the semantic similarity,it calculated the correlation confidence,left-right adjacency entropy,and multi-semantic similarity(including the semantic information of Chinese characters with finegrained characters,BERT word vector information),and traversed the thresholds of each of the above indicators to evaluate the possibility of N-gram word strings as medical new words.[Result/Conclusion]From the latest 1000 abstracts in the Chinese Medical Association as of October 20,2022,the medical new word detection method identified 3263 new words,of which 764 were retained after removing duplicates.The method incorporating multi-semantic information of Chinese characters has made some progress over existing methods,and can effectively improve the effectiveness of the medical segmentation task.After the medical word segmentation,the noun category is clearer,the concept is more explicit,and the connotation is richer.This algorithm can not only improve the computer’s new word detection ability,but also its natural language processing effect in the face of specialized and complex medical texts,which is important to timely update the domain knowledge content.

关 键 词：医学新词发现 N-GRAM 多语义词向量 关联置信度 左右信息熵 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]