基于乘积HMM的双模态语音识别方法

针对噪声环境中的语音识别，提出一种用于双模态语音识别的乘积隐马尔可夫模型(HMM)。在独立训练音频HMM和视频HMM的基础上，建立二维训练模型，表征音频流和视频流之间的异步特性。引入权重系数，根据不同噪声环境自适应调整音频流与视频流的权重。实验结果证明，与其他双模态语音识别方法相比，该方法的识别性能更高。

Bimodal Speech Recognition Approach Based on Product HMM

Aiming at speech recognition in noise environment, this paper proposes a product Hidden Markov Model(HMM) used for bimodal speech recognition. A two-dimension training model is built based on dependently trained audio-HMM and visual-HMM, which reflects the asynchronous characteristic between audio stream and video stream. Weight coefficient is introduced to automatically adjust weight of video stream and audio stream according to different noise environment. Experimental results show that compared with other bimodal speech recognition approaches, this approach exhibits better performance on speech recognition.