Adaptation of hybrid deep neural network-hidden Markov model speech recognition system using a sub-space approach

Abstract

The performance of automatic speech recognition (ASR) system can be enhanced by adaptation of the ASR for a particular speaker or a group of speakers. In ASR, training and testing data often do not follow the same statistics; they are often mismatched, which leads to a gap in performance. The difference between training and testing statistics can be minimized by speaker adaptation techniques, which require adaptation data from a target speaker to optimize system performance. In many cases, only a limited amount of adaptation data is available for the target speaker. This thesis proposes multiple methods for the adaptation of speech recognition system by using a limited amount of data (a few words). The first method classifies accent of a speaker to identify variability in speaking style. Results indicated that using multiple words from a speaker can be efficient and can provide better accent classification accuracy. Next adaptive phoneme classification method is proposed based on target speaker similarity with speakers in the training data. DNNs last hidden layer activations are found to be more useful in identifying phoneme classes of frames as compared with traditional raw Mel-frequency cepstral coefficients as features. Finally, speaker adaptation of ASR is presented by augmenting the speech features with the speaker features. The universal background sparse coding can provide useful speaker information for the speaker adaptation. These methods may lead to some new opportunities for research for the adaptation of the ASR.