基于韵母发音事件匹配与位置时延分析的音唇一致性判决方法

针对传统一致性判决方法主要对整句(段)话进行分析,并无对分析内容加以筛选,存在字典规模过大、计算复杂度高及结果易受静音等弱关联片段影响等不足,本文以唇型变化显著的韵母为代表性发音事件,结合音唇初始时延分布范围的统计结果,提出基于韵母发音事件匹配与位置时延分析的一致性判决方法.先利用提出的音视频结合韵母切分法对字典学习数...

Lip Motion and Voice Consistency Recognition Based on Audio-Visual Matching of Vowel Pronunciation Events and Position Delay Analysis

For the mainstream lip motion and voice coherence judgment method,the whole sentence (segment) is analyzed without screening the content.This leads to large dictionary size and high computational complexity,and the result is vulnerable to weak related segments such as mute.Considering the vowel with significant lip shape changes as a representative pronunciation event and combining with the statistical results of the audio-visual initial delay distribution range,a consistent decision method based on audio-visual matching of vowel pronunciation events and position delay analysis is proposed.Firstly,the dictionary learning data is selected by the proposed audio-visual vowel segmentation method,and then the vowel dictionary is used to analyze the matching of the vowel event,and the time delay distribution of each vowel position is statistically scored.A consistency judgment is made by a scoring mechanism in which the vowel pronunciation event lip matching score and the position delay analysis score are combined.Experimental results show that the proposed method is superior to compared algorithms in recognition performance and reduces the amount of computation compared with the traditional dictionary method.