视听联觉在多通道人机自然交互中的应用研究  

Application of Audio-Visual Relevance in Natural Multi-Modal Human-Machine Interaction

作　　者：孙晓枫[1] 赵新军[1] 吕春梅[1] 张义民[2] SUN Xiao-feng;ZHAO Xin-jun;LV Chun-mei;ZHANG Yi-min(Northeastern University,Liaoning Shenyang 110819,China;Shenyang University of Chemical Technology,Liaoning Shenyang 110412,China)

机构地区：[1]东北大学,辽宁沈阳110819 [2]沈阳化工大学,辽宁沈阳110142

出　　处：《机械设计与制造》2021年第10期236-240,共5页Machinery Design & Manufacture

基　　金：国家自然科学基金联合项目(U1708254);科技部创新方法工作专项(No.2018IM030200);东北大学机械学院教改项目。

摘　　要：研究视听联觉现象以及视听各因素之间存在的具体关联性,并将其应用在多通道人机自然交互中。通过实验系统的分析声音的五种心理和生理要素(音调、音速、粗糙度、尖锐度、非连续性)与多色彩组合的色相、饱和度、对比度以及位置关系等存在的交叉关联性。实验结果表明声音的部分元素如音调、粗糙度等确实与多色彩的色相、饱和度、对比度以及位置关系存在着明显的联觉效应。基于视听联觉效应和视听各元素之间存在的关联属性,设计了基于视听联觉的多通道人机自然交互模型,在模型中构建视觉因素和听觉因素之间的交叉联动关系,提升人机交互的内部处理效率和输出反馈精准度,从而有效提升人机交互的效率。It studies the phenomenon of audio-visual relevance between audio-visual factors and applies it to the natural interaction of multi-modal human-machine.We design the experiments and then analyzes the cross-correlation between the five psychological and physiological elements of the sound(pitch,tempo,roughness,sharpness,and discontinuity)and the presence of hue,saturation,contrast and positional relationships of multiple color combinations.The results of experiments show that some elements of the sound(such as pitch,tempo,sharpness,roughness,discontinuity)do have a significant synaptic effect to the hue,saturation,contrast and positional relationship of multi-color.We design the visual and audio multi-channel human-machine natural interaction model based on the audiovisual joint effect.It constructed a cross-linkage relationship between visual factors and auditory factors in the model,improve the internal processing efficiency of human-computer interaction and the accuracy of output feedback,thereby effectively improving the efficiency of human-computer interaction.

关 键 词：视听联觉 多通道 人机自然交互 

分 类 号：TH16[机械工程—机械制造及自动化] U462.1[机械工程—车辆工程]