Automatic speech recognition and speech variability: A review
---
paper_title: The Phonetic Bases of Speaker Recognition

paper_content:

How reliably can individuals be recognised by their voices? This question has recently been the subject of much debate among speech researchers and forensic scientists and the controversial and crucial nature of that debate has stimulated a wide range of empirical research. In this book Dr Nolan argues convincingly that both the design and interpretation of many of these experiments are vitiated by the lack of a comprehensive model of variability between speakers and within the speech of an individual. This volume clearly demonstrates that any valid theory of speaker recognition must integrate the approaches of a number of disciplines and it is itself an important step towards that integration. It will be of interest to phoneticians and to speech scientists, including those with an engineering background and also to forensic scientists specialising in this area.
---
paper_title: Automatic accent classification of foreign accented Australian English speech

paper_content:

An automatic classification system for foreign accents in Australian English (AuE) speech based on accent-dependent parallel phoneme recognition (PPR) has been developed. The classifier is designed to process continuous speech and to discriminate between native AuE speakers and two migrant speaker groups with foreign accents, whose first languages are Lebanese Arabic (LA) and South Vietnamese (SV). The training of the system can be automated and is novel in that it does not require manually labelled accented data. The test utterances are processed in parallel by three (AuE, SV and LA) accent-specific recognizers incorporating accent-specific HMMs and phoneme bigram language models to produce accent discrimination likelihood scores. The best average accent classification rates were 85.3% and 76.6% for accent-pair and three-accent class discrimination tasks, respectively. Analyses of the contributions to accent discrimination by phoneme-level processing and by the language model, are described.
---
paper_title: Accent identification

paper_content:

Foreign accent identification is a new challenging problem closely related to other relatively recent fields of the multilingual area such as dialect identification and language identification. The paper describes an automatic identification system for English accents from 6 different European countries. The approach is based on a parallel set of ergodic nets with context independent HMM units. The ergodic topology was also substituted by pronunciation transcription constraints in order to integrate accent specific automatic word recognisers. Considering the complexity of the task, the results can be considered encouraging for further research.
---
paper_title: Innovative approaches for large vocabulary name recognition

paper_content:

Automatic name dialing is a practical and interesting application of speech recognition on telephony systems. The IBM name recognition system is a large vocabulary, speaker independent system currently in use for reaching IBM employees in the United States. We present some innovative algorithms that improve name recognition accuracy. Unlike transcription tasks, such as the Switchboard task, recognition of names poses a variety of different problems. Several of these problems arise from the fact that foreign names are hard to pronounce for speakers who are not familiar with the names and that there are no standardized methods for pronouncing proper names. Noise robustness is another very important factor as these calls are typically made in noisy environments, such as from a car, cafeteria, airport, etc. and over different kinds of cellular and land-line telephone channels. We have performed a systematic analysis of the speech recognition errors and tackled the issues separately with techniques ranging from weighted speaker clustering, massive adaptation, rapid and unsupervised adaptation methods to pronunciation modeling methods. We find that the decoding accuracy can be improved significantly (28% relative) in this manner.
---
paper_title: Interaction between the native and second language phonetic subsystems

paper_content:

The underlying premise of this study was that the two phonetic subsystems of a bilingual interact. The study tested the hypothesis that the vowels a bilingual produces in a second language (L2) may differ from vowels produced by monolingual native speakers of the L2 as the result of either of two mechanisms: phonetic category assimilation or phonetic category dissimilation. Earlier work revealed that native speakers of Italian identify English/eI/ tokens as instances of the Italian /e/ category even though English /eI/ is produced with more tongue movement than Italian /e/ is. Acoustic analyses in the present study examined /eI/s produced by four groups of Italian-English bilinguals who differed according to their age of arrival in Canada from Italy (early versus late) and frequency of continued Italian use (low-L1- use versus high-L1-use). Early bilinguals who seldom used Italian (Early-low) were found to produce English /eI/ with significantly more movement than native English speakers. However, both groups of late bilinguals (Late-low, Late-high) tended to produced /eI/ with less movement than NE speakers. The exaggerated movement in /eI/s produced by the Early-low group participants was attributed to the dissimilation of a phonetic category they formed for English /eI/ from Italian /e/. The undershoot of movement in /eI/s produced by late bilinguals, on the other hand, was attributed to their failure to establish a new category for English /eI/, which led to the merger of the phonetic properties of English /eI/ and Italian /e/.
---
paper_title: Automatic scoring of pronunciation quality

paper_content:

We present a paradigm for the automatic assessment of pronunciation quality by machine. In this scoring paradigm, both native and nonnative speech data is collected and a database of human-expert ratings is created to enable the development of a variety of machine scores. We first discuss issues related to the design of speech databases and the reliability of human ratings. We then address pronunciation evaluation as a prediction problem, trying to predict the grade a human expert would assign to a particular skill. Using the speech and the expert-ratings databases, we build statistical models and introduce different machine scores that can be used as predictor variables. We validate these machine scores on the Voice Interactive Language Training System (VILTS) corpus, evaluating the pronunciation of American speakers speaking French and we show that certain machine scores, like the log-posterior and the normalized duration, achieve a correlation with the targeted human grades that is comparable to the human-to-human correlation when a sufficient amount of speech data is available.
---
paper_title: Automatic text-independent pronunciation scoring of foreign language student speech

paper_content:

SRI International is currently involved in the development of a new generation of software systems for automatic scoring of pronunciation as part of the Voice Interactive Language Training System (VILTS) project. This paper describes the goals of the VILTS system, the speech corpus and the algorithm development. The automatic grading system uses SRI's Decipher/sup TM/ continuous speech recognition system to generate phonetic segmentations that are used to produce pronunciation scores at the end of each lesson. The scores produced by the system are similar to those of expert human listeners. Unlike previous approaches, in which models were built for specific sentences or phrases, we present a new family of algorithms designed to perform well even when knowledge of the exact text to be used is not available.
---
paper_title: Phone-level pronunciation scoring and assessment for interactive language learning

paper_content:

This paper investigates a method of automatic pronunciation scoring for use in computer-assisted language learning (CALL) systems. The method utilises a likelihood-based ‘Goodness of Pronunciation’ (GOP) measure which is extended to include individual thresholds for each phone based on both averaged native confidence scores and on rejection statistics provided by human judges. Further improvements are obtained by incorporating models of the subject’s native language and by augmenting the recognition networks to include expected pronunciation errors. The various GOP measures are assessed using a specially recorded database of non-native speakers which has been annotated to mark phone-level pronunciation errors. Since pronunciation assessment is highly subjective, a set of four performance measures has been designed, each of them measuring diAerent aspects of how well computer-derived phone-level scores agree with human scores. These performance measures are used to cross-validate the reference annotations and to assess the basic GOP algorithm and its refinements. The experimental results suggest that a likelihood-based pronunciation scoring metric can achieve usable performance, especially after applying the various enhancements. ” 2000 Elsevier Science B.V. All rights reserved.
---
paper_title: Analysis of speaker variability

paper_content:

Analysis and modeling of speaker variability, such as gender, accent, age, speech rate, and phones realizations, are important issues in speech recognition. It is known that existing feature representations describing speaker variations can be of very high dimension. In this paper, we introduce two powerful multivariate statistical analysis methods, namely, principal component analysis (PCA) and independent component analysis (ICA), as tools for analysis of such variability and extraction of low dimensional feature representation. Our findings are the following: (1) the first two principal components correspond to the gender and accent, respectively. The result that the second component corresponding to the accent has never been reported before, to the best of our knowledge. (2) It is shown that ICA based features yield better classification performance than PCA ones. Using 2dimensional ICA representation, we achieved about 6.1% and 13.3% error rate in gender and accent classification, respectively, for 980 speakers.
---
paper_title: Different aspects of expert pronunciation quality ratings and their relation to scores produced by speech recognition algorithms

paper_content:

The ultimate aim of the research reported on here is to develop an automatic testing system for Dutch pronunciation. In the experiment described in this paper automatic scores of telephone speech produced by native and non-native speakers of Dutch are compared with specific, i.e., temporal and segmental, and global pronunciation ratings assigned by three groups of experts: three phoneticians and two groups of three speech therapists. The goals of this experiment are to determinutee (1) whether specific expert ratings of pronunciation quality contribute to our understanding of the relation between human pronunciation scores and machine scores of speech quality, (2) whether different expert groups assign essentially different ratings, and (3) to what extent rater pronunciation scores can be predicted on the basis of automatic scores. The results show that collecting specific ratings along with overall ones leads to a better understanding of the relation between human and automatic pronunciation assessment. Furthermore, after normalization no considerable differences are observed between the ratings by the three expert groups. Finally, it appears that the speech quality scores produced by our speech recognizer can predict expert pronunciation ratings with a high degree of accuracy.
---
paper_title: Fast accent identification and accented speech recognition

paper_content:

The performance of speech recognition systems degrades when speaker accent is different from that in the training set. Accent-independent or accent-dependent recognition both require collection of more training data. In this paper, we propose a faster accent classification approach using phoneme-class models. We also present our findings in acoustic features sensitive to a Cantonese accent, and possibly other Asian language accents. In addition, we show how we can rapidly transform a native accent pronunciation dictionary to that for accented speech by simply using knowledge of the native language of the foreign speaker. The use of this accent-adapted dictionary reduces recognition error rate by 13.5%, similar to the results obtained from a longer, data-driven process.
---
paper_title: Comparative Experiments On Large Vocabulary Speech Recognition

paper_content:

We describe recent changes to the BYBLOS system's training and recognition algorithms and report on numerous experiments in large vocabulary speech recognition. In earlier work, we performed five key experiments that were designed to answer questions related to different training scenarios. We investigated (1) the effect of varying the number of training speakers if the total amount of training data remains constant, (2) data pooling versus model averaging for generating speaker-independent (SI) HMMs, (3) the benefit of doubling the acoustic training data, (4) SI versus SD performance when the SI training data is twelve times greater, (5) the effect of cross-domain training for both the acoustic and language models. Our recent work was focused on four specific problem areas sharing the common thread that the test condition exposes the recognizer to phenomena not observed in the training data. Here we investigated (1) words outside the vocabulary, (2) spoken language effects due to subject variability and spontaneous dictation, (3) non-native dialects of the language, and (4) new microphones not used in training. >
---
paper_title: Phoneme-less hierarchical accent classification

paper_content:

This paper introduces a novel accent classification method. Compared with existing methods, it has two unique features. First, it does not explicitly utilize phoneme information. Second, it is built on top of the gender classification. We have tested the proposed algorithm on datasets that are completely independent of training data. The accuracy of distinguishing American accent and British accent is 83%. We have also compared the accent classification with the gender classification in terms of accuracy and the saturation behavior with respect to length of utterance.
---
paper_title: Generating proper name pronunciation variants for automatic speech recognition

paper_content:

Generating correct pronunciation of proper names remains one of the most difficult tasks in text-to-phoneme transcription. Although phonetic rules can be efficient in processing proper names of one language, foreign family names cannot be always correctly generated without additional pronunciation rules. The present study addresses the problem of pronunciation variants for French and foreign family names in a French grapheme to phoneme translator. 12 language and language group filters process the names and when necessary additional, language dependent pronunciation variants are added to the basic pronunciation yielded by the French transcription rules. In speech recognition based interactive vocal services, adding pronunciation variants to the basic phonetic transcription reduces the substitution error rate on proper names and makes applications using proper names more user friendly.
---
paper_title: Combination of Machine Scores for Automatic Grading of Pronunciation Quality

paper_content:

This work is part of an effort aimed at developing computer-based systems for language instruction; we address the task of grading the pronunciation quality of the speech of a student of a foreign language. The automatic grading system uses SRI's DecipherTM continuous speech recognition system to generate phonetic segmentations. Based on these segmentations and probabilistic models we produce different pronunciation scores for individual or groups of sentences that can be used as predictors of the pronunciation quality. Different types of these machine scores can be combined to obtain a better prediction of the overall pronunciation quality. In this paper we review some of the best-performing machine scores and discuss the application of several methods based on linear and nonlinear mapping and combination of individual machine scores to predict the pronunciation quality grade that a human expert would have given. We evaluate these methods in a database that consists of pronunciation-quality-graded speech from American students speaking French. With predictors based on spectral match and on durational characteristics, we find that the combination of scores improved the prediction of the human grades and that nonlinear mapping and combination methods performed better than linear ones. Characteristics of the different nonlinear methods studied are discussed.
---
paper_title: Recognizing speech of goats, wolves, sheep and...non-natives

paper_content:

Abstract This paper reviews the current understanding of acoustic–phonetic issues and the problems arising when trying to recognize speech from non-native speakers. Conceptually, regional accents are well modeled by systematic shifts in pronunciation. Therefore, simultaneous recognition of multiple regional variants may be performed by using multiple acoustic models in parallel, or by adding pronunciation variants in the dictionary. Recognition of non-native speech is much more difficult because it is influenced both by the native language of the speaker and non-native target language. It is characterized by a much greater speaker variability due to different levels of proficiency. A few language-pair specific transformation rules describing prototypical nativized pronunciations was found to be useful both in general speech recognition and in dedicated applications. However, due to the nature of the errors and the cross-language transformations, non-native speech recognition will remain inherently much harder. Moreover, the trend in speech recognition towards more detailed modeling seems to be counterproductive for the recognition of non-native speech and limits progress in this field.
---
paper_title: Language accent classification in American English

paper_content:

Abstract It is well known that speaker variability caused by accent is one factor that degrades performance of speech recognition algorithms. If knowledge of speaker accent can be estimated accurately, then a modified set of recognition models which addresses speaker accent could be employed to increase recognition accuracy. In this study, the problem of language accent classification in American English is considered. A database of foreign language accent is established that consists of words and phrases that are known to be sensitive to accent. Next, isolated word and phoneme based accent classification algorithms are developed. The feature set under consideration includes Mel-cepstrum coefficients and energy, and their first order differences. It is shown that as test utterance length increases, higher classification accuracy is achieved. Isolated word strings of 7–8 words uttered by the speaker results in an accent classification rate of 93% among four different language accents. A subjective listening test is also conducted in order to compare human performance with computer algorithm performance in accent discrimination. The results show that computer based accent classification consistently achieves superior performance over human listener responses for classification. It is shown, however, that some listeners are able to match algorithm performance for accent detection. Finally, an experimental study is performed to investigate the influence of foreign accent on speech recognition algorithms. It is shown that training separate models for each accent rather than using a single model for each word can improve recognition accuracy dramatically.
---
paper_title: An acoustic-phonetic and articulatory study of speech-speaker dichotomy

paper_content:

The acoustic speech signal naturally contains information pertaining to both the linguistic message and the identity of the speaker. Separation of these two, primary sources of variability has been of considerable concern to speech scientists for many years, yet we still lack a model of the speech communication process that can account for a wide variety of speakers even for a restricted set. of vowels of a given dialect of spoken English. ::: Basic research was therefore undertaken to gain a better understanding of the separate influences and of the interactions between phonetic and speaker-specific attributes of vowel sounds. First, a new methodology for performing computer vowel recognition was developed, which has the ability to reveal the contrastive influence of vowel-speaker interactions across different regions of the frequency spectrum. The results obtained from applying this methodology have: (i) yielded significant insights into the long-standing problem of phonetic-speaker dichotomy; and (ii) prompted a search for an even more fundamental explanation in terms of the physical properties of the speech production mechanism. ::: To this end, a new functional representation of vocal-tract shapes was derived, which depends directly on resonance parameters while retaining the uniqueness properties of the Linear-Prediction model of speech production. This hybrid modelling approach was used together with a new, articulatory method of speaker normalisation, to quantify speaker differences in vocal-tract shapes, and thus to define physical correlates of the phonetic-speaker interactions which were earlier shown to adversely affect vowel recognition accuracy in certain frequency bands. ::: In sum, this research work embraces two major domains of computer speech science and technology—namely, speech acoustics and speech production. Beyond advancing knowledge in speaker characterisation, it ultimately has implications in the still unresolved problem of speech or speaker recognition by computer.
---
paper_title: Long-term feature averaging for speaker recognition

paper_content:

The potential benefits of long-term parameter averaging for speaker recognition were investigated. Parameters studied were pitch, gain, and reflection coefficients. Parameter variability was computed over various averaging lengths from one frame averaging (in effect, no averaging) to 1000 frame averaging (about 70 s of speech). It was demonstrated that the between-to-within speaker variance ratio, measured over several speakers, was significantly increased by performing long-term averaging of the parameter sets. The reflection coefficient averages for k 2 and k 6 , respectively, were shown to produce the highest variance ratios.
---
paper_title: Adaptation to a speaker's voice in a speech recognition system based on synthetic phoneme references

paper_content:

Abstract A speech recognition system based on synthetic generation of reference prototypes is described. The vocabulary and grammar are described in a finite-state phoneme network. In the transformation from symbolic to spectral representation, reduction rules modify the initial phoneme target values and a coarticulation module inserts interpolated transition states at phoneme boundaries. The phoneme templates are specified in terms of control parameters to a seriel formant synthesiser. At each state, a 16-channel filter bank section is computed from the synthesis parameters. The recognition process uses a time-synchronous dynamic programming technique to find the path in the network that minimises the accumulated spectral distance to the input utterance. A technique for dynamic adaptation to the speaker's voice source spectrum is performed during recognition. Without adaptation, the average recognition for ten male speakers was 88% on an isolated-word task using a 26-word vocabulary. Adding voice source adaptation raised the performance to 96%. On a vocabulary of 3 connected digits, the adaptation technique improved the recognition rate for six male speakers from 87.7% to 92.8%. The improvement was largest for subjects with low initial recognition rate, indicating the benefit of the voice source adaptation technique for certain voices. Changing the voice source model and optimising the adaptation time constant raised the recognition rate further to 96.1%. Current work is directed towards speaker adaptation of phoneme parameters and modelling of the variability of the parameter dynamics at phoneme boundaries.
---
paper_title: The BBN BYBLOS Continuous Speech Recognition System

paper_content:

In this paper we describe the algorithms used in the BBN BYBLOS Continuous Speech Recognition system. The BYBLOS system uses context-dependent hidden Markov models of phonemes to provide a robust model of phonetic coarticulation. We provide an update of the ongoing research aimed at improving the recognition accuracy. In the first experiment we confirm the large improvement in accuracy that can be derived by using spectral derivative parameters in the recognition. In particular, the word error rate is reduced by a factor of two. Currently the system achieves a word error rate of 2.9% when tested on the speaker-dependent part of the standard 1000-Word DARPA Resource Management Database using the Word-Pair grammar supplied with the database. When no grammar was used, the error rate is 15.3%. Finally, we present a method for smoothing the discrete densities on the states of the HMM, which is intended to alleviate the problem of insufficient training for detailed phonetic models.
---
paper_title: Perceptual and Physical Space of Vowel Sounds

paper_content:

Experiments were carried out to investigate the correlation between the perceptual and physical space of 11 vowel sounds. The signals were single periods out of the constant vowel part of normally spoken words of the type h (vowel) t, generated continuously by computer. Pitch, loudness, onset, and duration were equalized. These signals were presented to 15 subjects in a triadic‐comparison procedure, resulting in a cumulative similarity matrix. Multidimensional scaling (Kruskal) of this matrix resulted in a three‐dimensional perceptual space with 1.6% stress. The signals were also analyzed physically with 13‐oct band filters. Principal‐components analysis of the decibel values per frequency band indicated that three dimensions accounted for 81.7% of the total variance. Matching the perceptual and the physical configurations to maximal congruence yielded an excellent result with correlation coefficients of 0.992, 0.971, and 0.742 along the corresponding dimensions. The formant frequencies and levels were co...
---
paper_title: Principles of Phonetics

paper_content:

Introduction Part I. General Concepts: 1. The semiotic framework 2. The relationship between phonetics and phonology 3. Accent, dialect and language Part II. The Anayltical Framework: 4. The phonetic analysis of speech 5. The architecture of phonetic classification Part III. Initiation and Phonotation: 6. Airstream mechanisms 7. Phonation Part IV. Linear Segmental Analysis: 8. Stop articulations 9. Fricative articulations 10. Resonant articulations 11. Multiple articulations Part V. Articulatory Co-ordination and Phonetic Settings: 12. Inter-segmental co-ordinations 13. Phonetic similarity and multi-segmental settings Part VI. Temporal, Prosodic and Metrical Analysis: 14. The temporal organization of speech segmental duration 15. The prosodic organization of speech: pitch and loudness 16. The metrical organization of speech: stress, syllable weight, prominence and rhythm 17. The temporal organization of speech continuity and rate Part VII. Principles of Transcription: 18. Types of transcription Part VIII Conclusion: 19. Evaluating general phonetic theory Envoi References Appendices Index of names Subject index.
---
paper_title: Frequency spectrum deviation between speakers

paper_content:

An ink jet recording element comprising the following layers in the order recited: I) a support of a solvent-absorbing, porous material, and II) an image-recording layer comprising a vinyl, latex polymer having the following formula: wherein: A is a hydrophilic, vinyl monomer; B is a hydrophobic, vinyl monomer; C is a cationic monomer; x is from about 10 to about 80 mole %; y is from about 10 to about 80 mole %; and z is from about 2 to about 20 mole %.
---
paper_title: Robust prosodic features for speaker identification

paper_content:

The paper describes the use of prosodic features for speaker identification. Features based on the pitch and energy contours of speech are described and the relative importance of each feature for speaker identification is investigated. The mean and variance of the pitch period in voiced sections of speech are shown to be particularly useful at discriminating between speakers. Fusing these features with a hidden Markov model speaker identification system gave a marked improvement in figure of merit; over 30% gain was achieved on the six NIST 1995 evaluation tests presented. Handset variability is known to have an adverse effect on performance when traditional spectral features are used, e.g. cepstra. Results are presented showing that the prosodic features are more robust to handset variability.
---
paper_title: Toward the Specification of Speech

paper_content:

This is an interim report on studies of the specification of speech sounds from acoustical measurements. Methods based upon analysis, synthesis, and vocal tract models are described. Included are the results of preliminary measurements on the vowel sounds of 25 speakers. Some of the problems in specifying the vowel sounds as indicated by these results are discussed.
---
paper_title: Information Conveyed by Vowels

paper_content:

Most speech sounds may be said to convey three kinds of information: linguistic information which enables the listener to identify the words that are being used; socio‐linguistic information, which enables him to appreciate something about the background of the speaker; and personal information which helps to identify the speaker. An experiment has been carried out which shows that the linguistic information conveyed by a vowel sound does not depend on the absolute values of its formant frequencies, but on the relationship between the formant frequencies for that vowel and the formant frequencies of other vowels pronounced by that speaker. Six versions of the sentence Please say what this word is were synthesized on a Parametric Artificial Talking device. Four test words of the form b‐(vowel)‐t were also synthesized. It is shown that the identification of the test word depends on the formant structure of the introductory sentence. Some psychological implications of this experiment are discussed, and hypot...
---
paper_title: A study on speaker adaptation of the parameters of continuous density hidden Markov models

paper_content:

For a speech-recognition system based on continuous-density hidden Markov models (CDHMM), speaker adaptation of the parameters of CDHMM is formulated as a Bayesian learning procedure. A speaker adaptation procedure which is easily integrated into the segmental k-means training procedure for obtaining adaptive estimates of the CDHMM parameters is presented. Some results for adapting both the mean and the diagonal covariance matrix of the Gaussian state observation densities of a CDHMM are reported. The results from tests on a 39-word English alpha-digit vocabulary in isolated word mode indicate that the speaker adaptation procedure achieves the same level of performance as that of a speaker-independent system, when one training token from each word is used to perform speaker adaptation. It shows that much better performance is achieved when two or more training tokens are used for speaker adaptation. When compared with the speaker-dependent system, it is found that the performance of speaker adaptation is always equal to or better than that of speaker-dependent training using the same amount of training data. >
---
paper_title: Modeling of the glottal flow derivative waveform with application to speaker identification

paper_content:

An automatic technique for estimating and modeling the glottal flow derivative source waveform from speech, and applying the model parameters to speaker identification, is presented. The estimate of the glottal flow derivative is decomposed into coarse structure, representing the general flow shape, and fine structure, comprising aspiration and other perturbations in the flow, from which model parameters are obtained. The glottal flow derivative is estimated using an inverse filter determined within a time interval of vocal-fold closure that is identified through differences in formant frequency modulation during the open and closed phases of the glottal cycle. This formant motion is predicted by Ananthapadmanabha and Fant (1982) to be a result of time-varying and nonlinear source/vocal tract coupling within a glottal cycle. The glottal flow derivative estimate is modeled using the Liljencrants-Fant (1986) model to capture its coarse structure, while the fine structure of the flow derivative is represented through energy and perturbation measures. The model parameters are used in a Gaussian mixture model speaker identification (SID) system. Both coarse- and fine-structure glottal features are shown to contain significant speaker-dependent information. For a large TIMIT database subset, averaging over male and female SID scores, the coarse-structure parameters achieve about 60% accuracy, the fine-structure parameters give about 40% accuracy, and their combination yields about 70% correct identification. Finally, in preliminary experiments on the counterpart telephone-degraded NTIMIT database, about a 5% error reduction in SID scores is obtained when source features are combined with traditional mel-cepstral measures.
---
paper_title: Control Methods Used in a Study of the Vowels

paper_content:

Relationships between a listener's identification of a spoken vowel and its properties as revealed from acoustic measurement of its sound wave have been a subject of study by many investigators. Both the utterance and the identification of a vowel depend upon the language and dialectal backgrounds and the vocal and auditory characteristics of the individuals concerned. The purpose of this paper is to discuss some of the control methods that have been used in the evaluation of these effects in a vowel study program at Bell Telephone Laboratories. The plan of the study, calibration of recording and measuring equipment, and methods for checking the performance of both speakers and listeners are described. The methods are illustrated from results of tests involving some 76 speakers and 70 listeners.
---
paper_title: The Phonetic Bases of Speaker Recognition

paper_content:

How reliably can individuals be recognised by their voices? This question has recently been the subject of much debate among speech researchers and forensic scientists and the controversial and crucial nature of that debate has stimulated a wide range of empirical research. In this book Dr Nolan argues convincingly that both the design and interpretation of many of these experiments are vitiated by the lack of a comprehensive model of variability between speakers and within the speech of an individual. This volume clearly demonstrates that any valid theory of speaker recognition must integrate the approaches of a number of disciplines and it is itself an important step towards that integration. It will be of interest to phoneticians and to speech scientists, including those with an engineering background and also to forensic scientists specialising in this area.
---
paper_title: On the use of bandpass liftering in speech recognition

paper_content:

In this paper, we extend the interpretation of distortion measures, based upon the observation that measurements of speech spectral envelopes (as normally obtained from analysis procedures) are prone to statistical variations due to window position fluctuations, excitation interference, measurement noise, etc. and may possess spurious characteristics because of analysis model constraints. We have found that these undesirable spectral measurement variations can be controlled (i.e. reduced in the level of variation) through proper cepstral processing and that a statistical model can be established to predict the variances of the cepstral coefficient measurements. The findings lead to the use of a bandpass "liftering" process aimed at reducing the variability of the statistical components of spectral measurements. We have applied this liftering process to various speech recognition problems; in particular, vowel recognition and isolated word recognition. With the liftering process, we have been able to achieve an average digit error rate of 1%, which is about half of the previously reported best results, with dynamic time warping in a speaker-independent isolated digit test.
---
paper_title: On speaker-independent, speaker-dependent, and speaker-adaptive speech recognition

paper_content:

The DARPA Resource Management task is used as the domain to investigate the performance of speaker-independent, speaker-dependent, and speaker-adaptive speech recognition. The authors already have a state-of-the-art speaker-independent speech recognition system, SPHINX. The error rate for RM2 test set is 4.3%. They extended SPHINX to speaker-dependent speech recognition. The error rate is reduced to 1.4-2.6% with 600-2400 training sentences for each speaker, which demonstrated a substantial difference between speaker-dependent and -independent systems. Based on speaker-independent models, a study was made of speaker-adaptive speech recognition. With 40 adaptation sentences for each speaker, the error rate can be reduced from 4.3% to 3.1%. >
---
paper_title: Explaining Phonetic Variation: A Sketch of the H&H Theory

paper_content:

The H&H theory is developed from evidence showing that speaking and listening are shaped by biologically general processes. Speech production is adaptive. Speakers can, and typically do, tune their performance according to communicative and situational demands, controlling the interplay between production-oriented factors on the one hand, and output-oriented constraints on the other. For the ideal speaker, H&H claims that such adaptations reflect his tacit awareness of the listener’s access to sources of information independent of the signal and his judgement of the short-term demands for explicit signal information. Hence speakers are expected to vary their output along a continuum of hyper- and hypospeech. The theory suggests that the lack of invariance that speech signals commonly exhibit (Perkell and Klatt 1986) is a direct consequence of this adaptive organization (cf MacNeilage 1970). Accordingly, in the H&H program the quest for phonetic invariance is replaced by another research task: Explicating the notion of sufficient discriminability and defining the class of speech signals that meet that criterion.
---
paper_title: Word perception in fast speech: artificially time-compressed vs. naturally produced fast speech

paper_content:

Natural fast speech differs from normal-rate speech with respect to its temporal pattern. Previous results showed that word intelligibility of heavily artificially time-compressed speech could not be improved by making its temporal pattern more similar to that of natural fast speech. This might have been due to the extrapolation of timing rules for natural fast speech to rates that are much faster than can be attained by human speakers. The present study investigates whether, at a speech rate that human speakers can attain, artificially time-compressed speech is easier to process if its timing pattern is similar to that of naturally produced fast speech. Our first experiment suggests, however, that word processing speed was slowed down, relative to linear compression. In a second experiment, word processing of artificially time-compressed speech was compared with processing of naturally produced fast speech. Even when naturally produced fast speech is perfectly intelligible, its less careful articulation, combined with the changed timing pattern, slows down processing, relative to linearly time-compressed speech. Furthermore, listeners preferred artificially time-compressed speech over naturally produced fast speech. These results suggest that linearly time-compressed speech has both a temporal and a segmental advantage over natural fast speech. � 2003 Elsevier B.V. All rights reserved.
---
paper_title: Context dependent "long units" for speech recognition.

paper_content:

It is expected that longer-than-phoneme units such as syllables or multi-phone units can deal with sources of performance degradation, such as pronunciation variation or coarticulation, better than phoneme-sized units like triphones. The possible number of contextual realizations of those “long units” (LU) is very high, causing an explosion of the number of parameters to be estimated. As the training data are limited, the usual solution is to share parameters between different units to improve parameter estimation. Another problem is how to provide a model for a unit (syllable/multiphones) that was not present during training (unseen unit). In this paper we evaluate and compare syllable and automatically derived multi-phone units. We introduce a method called “contextual factorization” to share parameters between different models and we propose a figure of merit to decide which decomposition of an unseen syllable is the most appropriate. Performance is improved comparing to a triphone based system.
---
paper_title: Effects of disfluencies, predictability, and utterance position on word form variation in English conversation

paper_content:

Function words, especially frequently occurring ones such as (the, that, and, and of), vary widely in pronunciation. Understanding this variation is essential both for cognitive modeling of lexical production and for computer speech recognition and synthesis. This study investigates which factors affect the forms of function words, especially whether they have a fuller pronunciation (e.g., thi, thaet, aend, inverted-v v) or a more reduced or lenited pronunciation (e.g., thax, thixt, n, ax). It is based on over 8000 occurrences of the ten most frequent English function words in a 4-h sample from conversations from the Switchboard corpus. Ordinary linear and logistic regression models were used to examine variation in the length of the words, in the form of their vowel (basic, full, or reduced), and whether final obstruents were present or not. For all these measures, after controlling for segmental context, rate of speech, and other important factors, there are strong independent effects that made high-frequency monosyllabic function words more likely to be longer or have a fuller form (1) when neighboring disfluencies (such as filled pauses uh and um) indicate that the speaker was encountering problems in planning the utterance; (2) when the word is unexpected, i.e., less predictable in context; (3) when the word is either utterance initial or utterance final. Looking at the phenomenon in a different way, frequent function words are more likely to be shorter and to have less-full forms in fluent speech, in predictable positions or multiword collocations, and utterance internally. Also considered are other factors such as sex (women are more likely to use fuller forms, even after controlling for rate of speech, for example), and some of the differences among the ten function words in their response to the factors.
---
paper_title: Measuring and Compensating for the Effects of Speech Rate in Large Vocabulary Continuous Speech Recognition

paper_content:

This report describes a series of experiments that measure speech rate and that attempt to improve speech recognition accuracy for rapidly-spoken speech. Descriptions of several measures of speech rate are presented, with their advantages and disadvanatges. Speech recognition results obtained using several compensation methods are compared to identify methods by which compensation for the effects of fast speech may yield the greatest improvement in recognition accuracy. Very simple measures of speech rate such as the word rate or phone rate are found to be unsuitable for detection of both long-term and short-term speech rate since they are sensitive to the lexical content of speech. In contrast, the phone duration percentile, a comparison of measured versus expected phone duration, is shown to be robust with respect to lexical content and consistent with previous findings about the statistics of long-term and short-term speech rate. Using this metric, speakers with a speech rate in the top 30% are found to produce a 50 to 150% increase in word error rate. The compensation techniques explored contain modifications to five components of the recognition system: the models of the acoustical characteristics of speech sounds, the models of the HMM state-transition probabilities, the pronunciations of words in the dictionary, the weight with which acoustic and linguistic evidence are combined, and the base phone set. Optimizing the language weight reduced the word error rate of fast speech by 10.3% relative to baseline performance. Adapting the state-transition probabilities to fast speech reduced the word error rate for fast speech by 2.6%. Using one of the modified pronunciation dictionaries reduced the word error rate of fast speech by 2.6%. The other techniques yielded little or no reduction in the word error rate.
---
paper_title: Combining multiple estimators of speaking rate

paper_content:

We report progress in the development of a measure of speaking rate that is computed from the acoustic signal. The newest form of our analysis incorporates multiple estimates of rate; besides the spectral moment for a full-band energy envelope that we have previously reported, we also used pointwise correlation between pairs of compressed sub-band energy envelopes. The complete measure, called mrate, has been compared to a reference syllable rate derived from a manually transcribed subset of the Switchboard database. The correlation with transcribed syllable rate is significantly higher than our earlier measure; estimates are typically within 1-2 syllables/second of the reference syllable rate. We conclude by assessing the use of mrate as a detector for rapid speech.
---
paper_title: Word perception in fast speech: artificially time-compressed vs. naturally produced fast speech

paper_content:

Natural fast speech differs from normal-rate speech with respect to its temporal pattern. Previous results showed that word intelligibility of heavily artificially time-compressed speech could not be improved by making its temporal pattern more similar to that of natural fast speech. This might have been due to the extrapolation of timing rules for natural fast speech to rates that are much faster than can be attained by human speakers. The present study investigates whether, at a speech rate that human speakers can attain, artificially time-compressed speech is easier to process if its timing pattern is similar to that of naturally produced fast speech. Our first experiment suggests, however, that word processing speed was slowed down, relative to linear compression. In a second experiment, word processing of artificially time-compressed speech was compared with processing of naturally produced fast speech. Even when naturally produced fast speech is perfectly intelligible, its less careful articulation, combined with the changed timing pattern, slows down processing, relative to linearly time-compressed speech. Furthermore, listeners preferred artificially time-compressed speech over naturally produced fast speech. These results suggest that linearly time-compressed speech has both a temporal and a segmental advantage over natural fast speech. � 2003 Elsevier B.V. All rights reserved.
---
paper_title: Language model and speaking rate adaptation for spontaneous presentation speech recognition

paper_content:

The paper addresses adaptation methods to language model and speaking rate (SR) of individual speakers which are two major problems in automatic transcription of spontaneous presentation speech. To cope with a large variation in expression and pronunciation of words depending on the speaker, firstly, we investigate the effect of statistical and context-dependent pronunciation modeling. Secondly, we present unsupervised methods of language model adaptation to a specific speaker and a topic by 1) selecting similar texts based on the word perplexity and TF-IDF measure and 2) making direct use of the initial recognition result for generating an enhanced model. We confirm that all proposed adaptation methods and their combinations reduce the perplexity and word error rate. We also present a decoding strategy adapted to the SR. In spontaneous speech, SR is generally fast and may vary a lot. We also observe different error tendencies for portions of presentations where speech is fast or slow. Therefore, we propose a SR-dependent decoding strategy that applies the most appropriate acoustic analysis, phone models, and decoding parameters according to the SR. Several methods are investigated and their selective application leads to improved accuracy. The combined effect of the two proposed adaptation methods is also confirmed in transcription of real academic presentation.
---
paper_title: The uninvited guest: Information's role in guiding the production of spontaneous speech

paper_content:

Statistical analysis of a large, phonetically transcribed corpus of spontaneous, American English dialogue suggests that an important factor governing the production of spoken language is the information associated with a given element (whether it be feature, phone, syllable, word or phrase), and that it is difficult to fully account for articulatory patterns (as inferred from the acoustic signal) purely on the basis of biomechanical factors. This entropic foundation of articulation is observed in terms of the probability of canonical pronunciation relative to the position within the syllable, as well as with respect to speaking rate and frequency of lexical occurrence. Such patterns of pronunciation variability imply that the phonetic realization (and hence production) of spoken language is highly dependent on the entropy associated with syllabic, lexical and phrasal contexts, and thus it is likely that the production of spontaneous speech is largely governed by mechanisms operating at an exceedingly high level of linguistic organization.
---
paper_title: On-line speaking rate estimation using Gaussian mixture models

paper_content:

Gaussian mixture models (GMM) are a widespread tool in applications like speaker identification or verification. In contrast to hidden Markov models (HMM) Gaussian mixture models are designed to model the general properties of an underlying acoustic source. In our paper we extend the application of GMMs to the assessment of speaking rate. Directly trained on the acoustic data, they can be either applied directly to estimate the speech rate category or-with the help of a mapping function-they can provide a continuous measure for the speaking rate. The mapping function can be realized by means of a neural net. First experiments showed a correlation coefficient of 0.66 between the lexical phoneme rate and our estimation based on speech rate dependent spectral variation. Moreover, our approach can be used simultaneously for high accuracy on-line gender detection.
---
paper_title: Improvements in children's speech recognition performance

paper_content:

There are several reasons why conventional speech recognition systems modeled on adult data fail to perform satisfactorily on children's speech input. For instance, children's vocal characteristics differ significantly from those of adults. In addition, their choices of vocabulary and sentence construction modalities usually do not conform to adult patterns. We describe comparative studies demonstrating the performance gain realized by adopting to children's acoustic and language model data to construct a children's speech recognition system.
---
paper_title: Speaker normalization using efficient frequency warping procedures

paper_content:

In an effort to reduce the degradation in speech recognition performance caused by variation in vocal tract shape among speakers, a frequency warping approach to speaker normalization is investigated. A set of low complexity, maximum likelihood based frequency warping procedures have been applied to speaker normalization for a telephone based connected digit recognition task. This paper presents an efficient means for estimating a linear frequency warping factor and a simple mechanism for implementing frequency warping by modifying the filter-bank in mel-frequency cepstrum feature analysis. An experimental study comparing these techniques to other well-known techniques for reducing variability is described. The results showed that frequency warping was consistently able to reduce word error rate by 20% even for very short utterances.
---
paper_title: Investigating Recognition Of Children's Speech

paper_content:

Recognition of children's speech was investigated by considering a phone recognition task. Two baseline systems were trained, one for children and one for adults, by exploiting two Italian speech databases. Under matching conditions, for training and recognition performed with data from the same population group, the phone recognition accuracy was 77.30% and 79.43% for children and adults, respectively. It was found that, for many children, recognition results were as good as for adults. However, a higher variability in phone recognition accuracy across speakers was observed for children than for adults. Vocal tract length normalization, under matched and mismatched training and testing conditions, was also investigated. For both adults and children, a performance improvement, with respect to the baseline systems, was observed.
---
paper_title: A study of speech recognition for children and the elderly

paper_content:

Although children and the elderly have obvious needs for voice operated interfaces, hardly anything is known about the performance of the current automatic speech recognition technology with these people. In this paper we report the results of a thorough investigation into this field using a connected digit recognizer and a major telephone speech database. One would generally assume that the recognition of speech from these people would only be a matter of having enough, sufficiently representative training data. This turns out to be true only, as long as the speakers belong to the age range 15 to approximately 70. Outside this range the error rates increase dramatically, even with balanced amounts of training data. For males, the lower limit is very sharp and can be attributed to the change of pitch frequency during puberty. For females, the lower limit is gradual and caused by the slowly changing dimensions of the vocal tract length only. For both genders, the upper limit is very gradual and can possibly be attributed to changes in the glottis area and the internal control loops of the human articulatory system. The paper presents some supporting evidence for the above assertions and gives results for various attempts to improve the performance. Recognition of children and the elderly will require much more research if we are to fully understand the characteristics of these age group on current and future speech recognition systems.
---
paper_title: Comparing speech recognition for adults and children

paper_content:

This paper presents initial studies of the performance of a speech recogniser on children's speech when trained on children or adults. A connected-digits recogniser was used for this purpose. The individual digit accuracy among the children is correlated to some features of the child, such as age, gender, fundamental frequency and height. A strong correlation between age and accuracy was found. The accuracy was also found to be lower for child recognition than for adult recognition, even though the recognisers were trained on the correct class of speakers.
---
paper_title: Perception of Speaker Sex, Age, and Vocal Effort

paper_content:

Speech material recorded for the purpose of studying the acoustic properties of speech as a function of speaker sex, age, and vocal effort (induced by varying the distance between the speaker and t ...
---
paper_title: Robust recognition of children's speech

paper_content:

Developmental changes in speech production introduce age-dependent spectral and temporal variability in the speech signal produced by children. Such variabilities pose challenges for robust automatic recognition of children's speech. Through an analysis of age-related acoustic characteristics of children's speech in the context of automatic speech recognition (ASR), effects such as frequency scaling of spectral envelope parameters are demonstrated. Recognition experiments using acoustic models trained from adult speech and tested against speech from children of various ages clearly show performance degradation with decreasing age. On average, the word error rates are two to five times worse for children speech than for adult speech. Various techniques for improving ASR performance on children's speech are reported. A speaker normalization algorithm that combines frequency warping and model transformation is shown to reduce acoustic variability and significantly improve ASR performance for children speakers (by 25-45% under various model training and testing conditions). The use of age-dependent acoustic models further reduces word error rate by 10%. The potential of using piece-wise linear and phoneme-dependent frequency warping algorithms for reducing the variability in the acoustic feature space of children is also investigated.
---
paper_title: Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion.

paper_content:

There has been considerable research into perceptible correlates of emotional state, but a very limited amount of the literature examines the acoustic correlates and other relevant aspects of emotion effects in human speech; in addition, the vocal emotion literature is almost totally separate from the main body of speech analysis literature. A discussion of the literature describing human vocal emotion, and its principal findings, are presented. The voice parameters affected by emotion are found to be of three main types: voice quality, utterance timing, and utterance pitch contour. These parameters are described both in general and in detail for a range of specific emotions. Current speech synthesizer technology is such that many of the parameters of human speech affected by emotion could be manipulated systematically in synthetic speech to produce a simulation of vocal emotion; application of the literature to construction of a system capable of producing synthetic speech with emotion is discussed.
---
paper_title: Describing the emotional states that are expressed in speech

paper_content:

To study relations between speech and emotion, it is necessary to have methods of describing emotion. Finding appropriate methods is not straightforward, and there are difficulties associated with the most familiar. The word emotion itself is problematic: a narrow sense is often seen as "correct", but it excludes what may be key areas in relation to speech-including states where emotion is present but not full-blown, and related states (e.g., arousal, attitude). Everyday emotion words form a rich descriptive system, but it is intractable because it involves so many categories, and the relationships among them are undefined. Several alternative types of description are available. Emotion-related biological changes are well documented, although reductionist conceptions of them are problematic. Psychology offers descriptive systems based on dimensions such as evaluation (positive or negative) and level of activation, or on logical elements that can be used to define an appraisal of the situation. Adequate descriptive systems need to recognise the importance of both time course and interactions involving multiple emotions and/or deliberate control. From these conceptions of emotion come various tools and techniques for describing particular episodes. Different tools and techniques are appropriate for different purposes.
---
paper_title: Adaptive source generator compensation and enhancement for speech recognition in noisy stressful environments

paper_content:

The author describes a low-vocabulary speech recognition algorithm which provides robust performance in noisy environments with particular emphasis on characteristics due to stress. A stressed speech source generator framework is formulated to achieve robust speech parameter characterization using a morphological constrained enhancement algorithm and stressed source compensation which is unique for each source generator across a stressed speaking class. An estimated source generator class sequence allows noise parameter enhancement and stress compensation schemes to adapt to changing speech generator types. A phonetic consistency rule is also employed based on input source generator partitioning. Average recognition rates for noisy stressful speech are shown to increase from an average 36.7% for a baseline recognizer to 74.7% for the new recognition algorithm. The new algorithm is also more consistent under varying noisy conditions as demonstrated by a decrease in standard deviation of recognition from 21.1 to 11.9, and a reduction in confusable word-pairs under noisy, stressed speaking conditions. >
---
paper_title: Evaluation of acoustic correlates of speech under stress for robust speech recognition

paper_content:

Results are presented of an investigation of how speech characteristics change under varying levels of stress with specific application to improving automatic isolated-word speech recognition. The evaluation focused on five speech analysis domains: pitch, glottal source, intensity, duration, and vocal tract shaping. Goodness-of-fit statistical tests were used to ascertain the significance of parameter variation in each domain. Results from analysis of pitch and glottal source spectrum are presented. The findings suggest that such parameter information can be used reliably to aid in automatic isolated-word speech recognition in noisy stressful environments. >
---
paper_title: Segmental minimum Bayes-risk decoding for automatic speech recognition

paper_content:

Minimum Bayes-risk (MBR) speech recognizers have been shown to yield improvements over the conventional maximum a-posteriori probability (MAP) decoders through N-best list rescoring and A/sup */ search over word lattices. We present a segmental minimum Bayes-risk decoding (SMBR) framework that simplifies the implementation of MBR recognizers through the segmentation of the N-best lists or lattices over which the recognition is to be performed. This paper presents lattice cutting procedures that underly SMBR decoding. Two of these procedures are based on a risk minimization criterion while a third one is guided by word-level confidence scores. In conjunction with SMBR decoding, these lattice segmentation procedures give consistent improvements in recognition word error rate (WER) on the Switchboard corpus. We also discuss an application of risk-based lattice cutting to multiple-system SMBR decoding and show that it is related to other system combination techniques such as ROVER. This strategy combines lattices produced from multiple ASR systems and is found to give WER improvements in a Switchboard evaluation system.
---
paper_title: A framework for predicting speech recognition errors

paper_content:

Pronunciation modeling in automatic speech recognition systems has had mixed results in the past; one likely reason for poor performance is the increased confusability in the lexicon from adding new pronunciation variants. In this work, we propose a new framework for determining lexically confusable words based on inverted finite state transducers (FSTs); we also present experiments designed to test some of the implementation details of this framework. The method is evaluated by examining how well the algorithm predicts the errors in an ASR system. The model is able to generalize confusions learned from a training set to predict errors made by the speech recognizer on an unseen test set.
---
paper_title: Speech recognition, sylabification and statistical phonetics.

paper_content:

The invention relates to a condiment dispensing system for dispensing condiment and the like which includes a gas operated pump for moving condiment(s) from a condiment source to a delivery nozzle through a condiment conduit which includes a draw-back valve to prevent condiment buildup on the delivery nozzle. A gas-operated pump is controlled at the delivery nozzle to effect delivery of the condiment.
---
paper_title: Finding consensus in speech recognition: word error minimization and other applications of confusion networks

paper_content:

We describe a new framework for distilling information from word lattices to improve the accuracy of the speech recognition output and obtain a more perspicuous representation of a set of alternative hypotheses. In the standard MAP decoding approach the recognizer outputs the string of words corresponding to the path with the highest posterior probability given the acoustics and a language model. However, even given optimal models, the MAP decoder does not necessarily minimize the commonly used performance metric, word error rate (WER). We describe a method for explicitly minimizing WER by extracting word hypotheses with the highest posterior probabilities from word lattices. We change the standard problem formulation by replacing global search over a large set of sentence hypotheses with local search over a small set of word candidates. In addition to improving the accuracy of the recognizer, our method produces a new representation of a set of candidate hypotheses that specifies the sequence of word-level confusions in a compact lattice format. We study the properties of confusion networks and examine their use for other tasks, such as lattice compression, word spotting, confidence annotation, and reevaluation of recognition hypotheses using higher-level knowledge sources.
---
paper_title: Analysis of acoustic-phonetic variations in fluent speech using TIMIT

paper_content:

We propose a hierarchically structured analysis of variance (ANOVA) method to analyze, in a quantitative manner, the contributions of various identifiable factors to the overall acoustic variability exhibited in fluent speech data of TIMIT processed in the form of mel-frequency cepstral coefficients. The results of the analysis show that the greatest acoustic variability in TIMIT data is explained by the difference among distinct phonetic labels in TIMIT, followed by the phonetic context difference given a fixed phonetic label. The variability among sequential sub-segments within each TIMIT-defined phonetic segment is found to be significantly greater than the gender, dialect region, and speaker factors. Our results serve to provide useful insights to the understanding of the roles of various components of speech recognizers in contributing to the ultimate speech recognition performance.
---
paper_title: Prosodic and other cues to speech recognition failures

paper_content:

In spoken dialogue systems, it is important for the system to know how likely a speech recognition hypothesis is to be correct, so it can reject misrecognized user turns, or, in cases where many errors have occurred, change its interaction strategy or switch the caller to a human attendant. We have identified prosodic features which predict more accurately when a recognition hypothesis contains errors than the acoustic confidence scores traditionally used in automatic speech recognition in spoken dialogue systems. We describe statistical comparisons of features of correctly and incorrectly recognized turns in the TOOT train information corpus and the W99 conference registration corpus, which reveal significant prosodic differences between the two sets of turns. We then present machine learning results showing that the use of prosodic features, alone and in combination with other automatically available features, can predict more accurately whether or not a user turn was correctly recognized, when compared to the use of acoustic confidence scores alone.
---
paper_title: Error-responsive feedback mechanisms for speech recognizers

paper_content:

This thesis is about modeling, analyzing, and predicting errorful behavior in large vocabulary continuous speech recognition systems. Because today's state-of-the-art recognizers are not designed to be situated naturally in an error feedback loop, they are ill-positioned for inclusion in multi-modal interfaces, multi-media databases, and other interesting applications. I make improvements to the current approach to predicting and analyzing error behaviors, which is currently based only on the measurement of word error rate. ::: The speech recognizer's functionality is extended to include confidence annotations, which are "meta-level" markings that indicate how certain the recognizer is that it has decoded its input correctly. This is accomplished by feeding externally defined error conditions back to the recoginizer. Error feedback enables the construction of statistical models that map measurements of the recognizer's internal states and behaviors to externally defined error conditions. ::: The measuring and modeling techniques used for confidence annotation are extended to create a blame assignment system for utterances whose actual transcripts are known. Errors are classified into a set of categories, some of which are directly useful in automatic adaptation schemes while others are more suited for human interpretation. ::: This classification approach is enhanced when used in conjunction with a visual error analysis tool that was developed during the thesis project.
---
paper_title: Causal analysis of Speech Recognition failure in adverse environments

paper_content:

A common approach to measuring the impact of noise and the effectiveness of noise mitigation (NM) algorithms for Automatic Speech Recognition (ASR) systems is to compare the word error rates (WERs). However, the WER measure does not give much insight into how an NM algorithm affects phoneme-level acoustic characteristics. Such insight can help in tuning the NM parameters and may also lead to reduced research time because the impact of an NM algorithm on ASR can first be investigated on smaller corpora. In this paper, two measures, phoneme error rate (PER) and phoneme confidence score (PCS), are investigated to assess the impact of NM algorithms on the ASR performance. Experimental results using the TIMIT corpus show that both PER and PCS can help identify where the degradation from noise occurs as well as give a useful indication of how an NM algorithm may impact ASR performance. A diagnostic method based on these two measures is also proposed to assess the NM impact on ASR and help improve the NM algorithm performance.
---
paper_title: Understanding and improving speech recognition performance through the use of diagnostic tools

paper_content:

The goal of this work is to highlight aspects of an experiment other than the word error rate. When a speech recognition experiment is performed, the word error rate provides no insight into the factors responsible for the recognition errors. We begin this paper by describing an experiment which contrasts the language of conversational speech with that of text from the Wall Street Journal. The remainder of the paper is devoted to the description of a more general approach to performance diagnosis which identifies significant sources of error in a given experiment. The technique is based on the use of binary classification trees; we refer to the results of our analyses as diagnostic trees. Beyond providing understanding, diagnostic trees allow for improvements in the performance of a recognizer through the use of feedback provided by quantifying confidence in the recognition.
---
paper_title: Analysis of speaker variability

paper_content:

Analysis and modeling of speaker variability, such as gender, accent, age, speech rate, and phones realizations, are important issues in speech recognition. It is known that existing feature representations describing speaker variations can be of very high dimension. In this paper, we introduce two powerful multivariate statistical analysis methods, namely, principal component analysis (PCA) and independent component analysis (ICA), as tools for analysis of such variability and extraction of low dimensional feature representation. Our findings are the following: (1) the first two principal components correspond to the gender and accent, respectively. The result that the second component corresponding to the accent has never been reported before, to the best of our knowledge. (2) It is shown that ICA based features yield better classification performance than PCA ones. Using 2dimensional ICA representation, we achieved about 6.1% and 13.3% error rate in gender and accent classification, respectively, for 980 speakers.
---
paper_title: Theory and practice of acoustic confusability

paper_content:

In this paper we define two alternatives to the familiar perplexity statistic (hereafter lexical perplexity), which is widely applied both as a figure of merit and as an objective function for training language models. These alternatives, respectively acoustic perplexity and the synthetic acoustic word error rate, fuse information from both the language model and the acoustic model. We show how to compute these statistics by effectively synthesizing a large acoustic corpus, demonstrate their superiority (on a modest collection of models and test sets) to lexical perplexity as predictors of language model performance, and investigate their use as objective functions for training language models. We develop an efficient algorithm for training such models, and present results from a simple speech recognition experiment, in which we achieved a small reduction in word error rate by interpolating a language model trained by synthetic acoustic word error rate with a unigram model.
---
paper_title: Objective and diagnostic assessment of (isolated) word recognizers

paper_content:

The performance of speech recognizers is measured as a function of the variation of specific speech-production and speech-transmission parameters, e.g. parameters related to interspeaker and intraspeaker variation and to the effect of stress or noise. The method uses a consonant-vowel-consonant-word database with minimal-difference word sets, consisting of three groups related to initial consonants, final consonants, and vowels, By means of analysis-resynthesis the words are manipulated corresponding to the changes of the physical parameters which are observed in natural speech or between speakers under various environmental conditions. In order to define relevant parameter changes, an analysis of representative speech tokens is made. Some experimental results obtained for four commercial recognizers as well as for human listeners are given. >
---
paper_title: Human and machine consonant recognition

paper_content:

Three traditional ASR parameterizations matched with Hidden Markov Models (HMMs) are compared to humans for speaker-dependent consonant recognition using nonsense syllables degraded by highpass filtering, lowpass filtering, or additive noise. Confusion matrices were determined by recognizing the syllables using different ASR front ends, including Mel-Filter Bank (MFB) energies, Mel-Filtered Cepstral Coefficients (MFCCs), and the Ensemble Interval Histogram (EIH). In general the MFB recognition accuracy was slightly higher than the MFCC, which was higher than the EIH. For syllables degraded by lowpass and highpass filtering, automated systems trained on the degraded condition recognized the consonants as well as humans. For syllables degraded by additive speech-shaped noise, none of the automated systems recognized consonants as well as humans. The greatest advantage displayed by humans was in determining the correct voiced/unvoiced classification of consonants in noise.
---
paper_title: Speech recognition by machines and humans

paper_content:

Abstract This paper reviews past work comparing modern speech recognition systems and humans to determine how far recent dramatic advances in technology have progressed towards the goal of human-like performance. Comparisons use six modern speech corpora with vocabularies ranging from 10 to more than 65,000 words and content ranging from read isolated words to spontaneous conversations. Error rates of machines are often more than an order of magnitude greater than those of humans for quiet, wideband, read speech. Machine performance degrades further below that of humans in noise, with channel variability, and for spontaneous speech. Humans can also recognize quiet, clearly spoken nonsense syllables and nonsense sentences with little high-level grammatical information. These comparisons suggest that the human-machine performance gap can be reduced by basic research on improving low-level acoustic-phonetic modeling, on improving robustness with noise and channel variability, and on more accurately modeling spontaneous speech.
---
paper_title: Flat‐spectrum speech

paper_content:

Flat‐spectrum stimuli, consisting of many equal‐amplitude harmonics, produce timbre sensations that can depend strongly on the phase angles of the individual harmonics. For fundamental frequencies in the human pitch range, many realizable timbres have vowel‐like perceptual qualities. This observation suggests the possibility of constructing intelligible voiced speech signals that have flat‐amplitude spectra. This paper describes a successful experiment of creating several different diphthongs by judicious choice of the phase angles of a flat‐spectrum waveform. A possible explanation of the observed vowel timbres lies in the dependence of the short‐time amplitude spectra on phase changes.
---
paper_title: Speech recognition by machines and humans

paper_content:

Abstract This paper reviews past work comparing modern speech recognition systems and humans to determine how far recent dramatic advances in technology have progressed towards the goal of human-like performance. Comparisons use six modern speech corpora with vocabularies ranging from 10 to more than 65,000 words and content ranging from read isolated words to spontaneous conversations. Error rates of machines are often more than an order of magnitude greater than those of humans for quiet, wideband, read speech. Machine performance degrades further below that of humans in noise, with channel variability, and for spontaneous speech. Humans can also recognize quiet, clearly spoken nonsense syllables and nonsense sentences with little high-level grammatical information. These comparisons suggest that the human-machine performance gap can be reduced by basic research on improving low-level acoustic-phonetic modeling, on improving robustness with noise and channel variability, and on more accurately modeling spontaneous speech.
---
paper_title: On the limits of speech recognition in noise

paper_content:

We consider the performance of speech recognition in noise and focus on its sensitivity to the acoustic feature set. In particular, we examine the perceived information reduction imposed on a speech signal using a feature extraction method commonly used for automatic speech recognition. We observe that the human recognition rates on noisy digit strings drop considerably as the speech signal undergoes the typical loss of phase and loss of frequency resolution. Steps are taken to ensure that human subjects are constrained in ways similar to that of an automatic recognizer. The high correlation between the performance of the human listeners and that of our connected digit recognizer leads us to some interesting conclusions, including that typical cepstral processing is insufficient to support speech information in noise.
---
paper_title: A database for speaker-independent digit recognition

paper_content:

A large speech database has been collected for use in designing and evaluating algorithms for speaker independent recognition of connected digit sequences. This dialect balanced database consists of more than 25 thousand digit sequences spoken by over 300 men, women, and children. The data were collected in a quiet environment and digitized at 20 KHz. Formal human listening tests on this database provided certification of the labelling of the digit sequences, and also provided information about human recognition performance and the inherent recognizability of the data.
---
paper_title: Effects of phase on the perception of intervocalic stop consonants

paper_content:

Abstract Identification experiments were performed to assess the relative importance of Fourier phase versus amplitude for intervocalic stop consonant perception. In the first experiment, three types of stimuli were constructed from VCV signals: (1) Swapped stimuli, a swapped stimulus has the amplitude spectra of its consisting segments from one VCV signal and its corresponding phase spectra from another; (2) Phase-only stimuli; and (3) Amplitude-only stimuli. It was shown that the perception of intervocalic stop consonants varies from amplitude dominance to phase dominance as the Fourier analysis window size increases. The crossover lies somewhere between 192 ms and 256 ms. The influence of phase at smaller window sizes was elaborated in the second experiment. This experiment demonstrated that phonetically different signals can be constructed by combining the same short-time amplitude spectra with different phase spectra, so the short-time amplitude spectra displayed on a spectrogram cannot exclusively specify a stop consonant. In both experiments, the perception of voicing in stops was found to rely strongly on phase information while the perception of the place of articulation was mainly determined by amplitude information.
---
paper_title: Synthesizing speech from speech recognition parameters

paper_content:

The merits of different signal preprocessing schemes for speech recognizers are usually assessed purely on the basis of the resulting recognition accuracy. Such benchmarks give a good indication as to whether one preprocessing is better than another, but little knowledge is acquired about why it is better or how it could be further improved. In order to gain more insight in the preprocessing, we seek to re-synthesize speech from speech recognition features. This way, we are able to pin-point some deficiencies in our current preprocessing scheme. Additional analysis of successful new preprocessing schemes may allow us one day to identify precisely those properties that are desirable in a feature set. Next to these purely scientific aims, the re-synthesis of speech from recognition features is of interest to thin-client speech applications, and as an alternative to the classical LPC source-filter model for speech manipulation.
---
paper_title: Usefulness of Phase Spectrum in Human Speech Perception

paper_content:

Short-time Fourier transform of speech signal has two components: magnitude spectrum and phase spectrum. In this paper, relative importance of short-time magnitude and phase spectra on speech perception is investigated. Human perception experiments are conducted to measure intelligibility of speech tokens synthesized either from magnitude spectrum or phase spectrum. It is traditionally believed that magnitude spectrum plays a dominant role for shorter windows (20-30 ms); while phase spectrum is more important for longer windows (128-3500 ms). It is shown in this paper that even for shorter windows, phase spectrum can contribute to speech intelligibility as much as the magnitude spectrum if the shape of the window function is properly selected.
---
paper_title: Vocal tract length invariant features for automatic speech recognition

paper_content:

The effects of vocal tract length (VTL) variation are often approximated by linear frequency warping of short-time spectra. Based on this relationship, we present a method for generating vocal tract length invariant features. These new features are computed as translation invariant, correlation-type features in a log-frequency domain. In phoneme classification experiments, their discrimination capabilities turned out to be considerably better than for Mel-frequency cepstral coefficients (MFCCs). The best results are obtained when VTL-invariant (VTLI) features and MFCCs are combined. The superiority of the combined feature set and its resilience to VTL variations is also shown for word recognition, using the TIDIGITS corpus and the HTK recognizer
---
paper_title: Speaker adaptive modeling by vocal tract normalization

paper_content:

This paper presents methods for speaker adaptive modeling using vocal tract normalization (VTN) along with experimental tests on three databases. We propose a new training method for VTN: By using single-density acoustic models per HMM state for selecting the scale factor of the frequency axis, we avoid the problem that a mixture-density tends to learn the scale factors of the training speakers and thus cannot be used for selecting the scale factor. We show that using single Gaussian densities for selecting the scale factor in training results in lower error rates than using mixture densities. For the recognition phase, we propose an improvement of the well-known two-pass strategy: by using a non-normalized acoustic model for the first recognition pass instead of a normalized model, lower error rates are obtained. In recognition tests, this method is compared with a fast variant of VTN. The two-pass strategy is an efficient method, but it is suboptimal because the scale factor and the word sequence are determined sequentially. We found that for telephone digit string recognition this suboptimality reduces the VTN gain in recognition performance by 30% relative. In summary, on the German spontaneous speech task Verbmobil, the WSJ task and the German telephone digit string corpus SieTill, the proposed methods for VTN reduce the error rates significantly.
---
paper_title: Fundamentals Of Speech Recognition

paper_content:

1. Fundamentals of Speech Recognition. 2. The Speech Signal: Production, Perception, and Acoustic-Phonetic Characterization. 3. Signal Processing and Analysis Methods for Speech Recognition. 4. Pattern Comparison Techniques. 5. Speech Recognition System Design and Implementation Issues. 6. Theory and Implementation of Hidden Markov Models. 7. Speech Recognition Based on Connected Word Models. 8. Large Vocabulary Continuous Speech Recognition. 9. Task-Oriented Applications of Automatic Speech Recognition.
---
paper_title: An improved sub-word based speech recognizer

paper_content:

The authors describe a system for speaker-dependent speech recognition based on acoustic subword units. Several strategies for automatic generation of an acoustic lexicon are outlined. Preliminary tests have been performed on a small vocabulary. In these tests, the proposed system showed results comparable to those of whole-word-based systems. >
---
paper_title: On the automatic segmentation of speech signals

paper_content:

For large vocabulary and continuous speech recognition, the sub-word-unit-based approach is a viable alternative to the whole-word-unit-based approach. For preparing a large inventory of subword units, an automatic segmentation is preferrable to manual segmentation as it substantially reduces the work associated with the generation of templates and gives more consistent results. In this paper we discuss some methods for automatically segmenting speech into phonetic units. Three different approaches are described, one based on template matching, one based on detecting the spectral changes that occur at the boundaries between phonetic units and one based on a constrained-clustering vector quantization approach. An evaluation of the performance of the automatic segmentation methods is given.
---
paper_title: Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences

paper_content:

Several parametric representations of the acoustic signal were compared with regard to word recognition performance in a syllable-oriented continuous speech recognition system. The vocabulary included many phonetically similar monosyllabic words, therefore the emphasis was on the ability to retain phonetically significant acoustic information in the face of syntactic and duration variations. For each parameter set (based on a mel-frequency cepstrum, a linear frequency cepstrum, a linear prediction cepstrum, a linear prediction spectrum, or a set of reflection coefficients), word templates were generated using an efficient dynamic warping method, and test data were time registered with the templates. A set of ten mel-frequency cepstrum coefficients computed every 6.4 ms resulted in the best performance, namely 96.5 percent and 95.0 percent recognition with each of two speakers. The superior performance of the mel-frequency cepstrum coefficients may be attributed to the fact that they better represent the perceptually relevant aspects of the short-term speech spectrum.
---
paper_title: Perceptual linear predictive (PLP) analysis of speech

paper_content:

A new technique for the analysis of speech, the perceptual linear predictive (PLP) technique, is presented and examined. This technique uses three concepts from the psychophysics of hearing to derive an estimate of the auditory spectrum: (1) the critical-band spectral resolution, (2) the equal-loudness curve, and (3) the intensity-loudness power law. The auditory spectrum is then approximated by an autoregressive all-pole model. A 5th-order all-pole model is effective in suppressing speaker-dependent details of the auditory spectrum. In comparison with conventional linear predictive (LP) analysis, PLP analysis is more consistent with human hearing. The effective second formant F2' and the 3.5-Bark spectral-peak integration theories of vowel perception are well accounted for. PLP analysis is computationally efficient and yields a low-dimensional representation of speech. These properties are found to be useful in speaker-independent automatic-speech recognition.
---
paper_title: Product of power spectrum and group delay function for speech recognition

paper_content:

Mel-frequency cepstral coefficients (MFCCs) are the most widely used features for speech recognition. These are derived from the power spectrum of the speech signal. Recently, the cepstral features derived from the modified group delay function (MGDF) have been studied by Murthy and Gadde (Proc. ICASSP, vol.1, p.68-71, 2003) for speech recognition. In this paper, we propose to use the product of the power spectrum and the group delay function (GDF), and derive the MFCCs from the product spectrum. This spectrum combines the information from the magnitude spectrum as well as the phase spectrum. The MFCCs of the MGDF are also investigated in this paper. Results show that the cepstral features derived from the power spectrum perform better than that from the MGDF, and the product spectrum based features provide the best performance.
---
paper_title: Continuous Speech Recognition using Joint Features derived from The Modified Group Delay Function and MFCC

paper_content:

Feature extraction and selection for continuous speech recognition is a complex task. State of the art speech recognition systems use features that are derived by ignoring the Fourier transform phase. In our earlier studies we have shown the efficacy of The Modified Group Delay Feature (MODGDF) derived from the Fourier transform phase for phoneme, syllable and speaker recognition. In this paper we use the MODGDF and the popular MFCC derived from Fourier transform magnitude to compute joint features for continuous speech recognition of two Indian languages Tamil and Telugu. A novel method of segmentation of the continuous speech signal into syllable like units followed by isolated style recognition using HMMs is used. We further use an innovative technique which transforms the problem of detecting the correct string of syllabic units with maximum likelihood to finding an optimal state sequence locally. The recognition system does not use any language models. The MODGDF gave promising recognition performance for the two languages and compared well with the MFCC. Joint features derived using MODGDF and MFCC gave a 10.6% improvement for both Tamil and Telugu languages. The improvement reinforces the hypothesis that MODGDF captures complementary information to that of the MFCC and can be used along with the MFCC to capture the complete information in the speech signal at functional level and help in avoiding heavy auditory and language models.
---
paper_title: Frequency-related representation of speech.

paper_content:

Cepstral features derived from power spectrum are widely used for automatic speech recognition. Very little work, if any,hasbeendoneinspeechresearchtoexplorephase-based representations. In this paper, an attempt is made to investigate the use of phase function in the analytic signal of critical-band filtered speech for deriving a representation of frequencies present in the speech signal. Results are presented which show the validity of this approach.
---
paper_title: Communication Systems

paper_content:

This best-selling, easy to read book offers the most complete discussion on the theories and principles behind today's most advanced communications systems. Throughout, Haykin emphasizes the statistical underpinnings of communication theory in a complete and detailed manner. Readers are guided though topics ranging from pulse modulation and passband digital transmission to random processes and error-control coding. The fifth edition has also been revised to include an extensive treatment of digital communications.
---
paper_title: Efficient coding of LPC parameters by temporal decomposition

paper_content:

This paper describes a method for efficient coding of LPC log area parameters. It is now well recognized that sample-by-sample quantization of LPC parameters is not very efficient in minimizing the bit rate needed to code these parameters. Recent methods for reducing the bit rate have used vector and segment quantization methods. Much of the past work in this area has focussed on efficient coding of LPC parameters in the context of vocoders which put a ceiling on achievable speech quality. The results from these studies cannot be directly applied to synthesis of high quality speech. This paper describes a different approach to efficient coding of log area parameters. Our aim is to determine the extent to which the bit rate of LPC parameters can be reduced without sacrificing speech quality. Speech events occur generally at non-uniformly spaced time intervals. Moreover, some speech events are slow while others are fast. Uniform sampling of speech parameters is thus not efficient. We describe a non-uniform sampling and interpolation procedure for efficient coding of log area parameters. A temporal decomposition technique is used to represent the continuous variation of these parameters as a linearly-weighted sum of a number of discrete elementary components. The location and length of each component is automatically adapted to speech events. We find that each elementary component can be coded as a very low information rate signal.
---
paper_title: An inverse signal approach to computing the envelope of a real valued signal

paper_content:

We address the problem of estimating the envelope of a real valued signal, s(t), that is observed for a duration of T seconds. We model s(t) using a Fourier series, by considering periodic extensions of the signal. By using an analog of the autocorrelation method of linear prediction on the Fourier coefficients of s(t), the envelope of the signal is estimated without explicitly computing the analytic signal through Hilbert transformation. Using this method the envelope of a non-stationary signal can be computed by processing the signal through a sliding T-second window.
---
paper_title: Traps - Classifiers Of Temporal Patterns

paper_content:

The work proposes a radically di erent set of features for ASR where TempoRAl Patterns of spectral energies are used in place of the conventional spectral patterns. The approach has several inherent advantages, among them robustness to stationary or slowly varying disturbances.
---
paper_title: Robust speech recognition using the modulation spectrogram

paper_content:

Abstract The performance of present-day automatic speech recognition (ASR) systems is seriously compromised by levels of acoustic interference (such as additive noise and room reverberation) representative of real-world speaking conditions. Studies on the perception of speech by human listeners suggest that recognizer robustness might be improved by focusing on temporal structure in the speech signal that appears as low-frequency (below 16 Hz) amplitude modulations in subband channels following critical-band frequency analysis. A speech representation that emphasizes this temporal structure, the “modulation spectrogram”, has been developed. Visual displays of speech produced with the modulation spectrogram are relatively stable in the presence of high levels of background noise and reverberation. Using the modulation spectrogram as a front end for ASR provides a significant improvement in performance on highly reverberant speech. When the modulation spectrogram is used in combination with log-RASTA-PLP (log RelAtive SpecTrAl Perceptual Linear Predictive analysis) performance over a range of noisy and reverberant conditions is significantly improved, suggesting that the use of multiple representations is another promising method for improving the robustness of ASR systems.
---
paper_title: Model-based approach to envelope and positive instantaneous frequency estimation of signals with speech applications

paper_content:

An analytic signal s(t) is modeled over a T second duration by a pole-zero model by considering its periodic extensions. This type of representation is analogous to that used in discrete-time systems theory, where the periodic frequency response of a system is characterized by a finite number of poles and zeros in the z-plane. Except, in this case, the poles and zeros are located in the complex-time plane. Using this signal model, expressions are derived for the envelope, phase, and the instantaneous frequency of the signal s(t). In the special case of an analytic signal having poles and zeros in reciprocal complex conjugate locations about the unit circle in the complex-time plane, it is shown that their instantaneous frequency (IF) is always positive. This result paves the way for representing signals by positive envelopes and positive IF (PIF). An algorithm is proposed for decomposing an analytic signal into two analytic signals, one completely characterized by its envelope and the other having a posit...
---
paper_title: On the use of phase information for speech recognition

paper_content:

This study addresses the use of short-time phase spectra in automatic speech recognition (ASR). Two recent studies [1,2] have proposed two group delay based spectral representations. Here we propose three new group delay based representations and compare usefulness of all these representations in an ASR experiment. We show that two of the representations we propose perform better, contain equivalent or complementary information to that of the power spectrum and are potentially useful for improving ASR performance.
---
paper_title: A Segmental HMM for Speech Waveforms

paper_content:

We present a purely time domain approach to speech processing which identies waveform samples at the boundaries between glottal pulse periods (in voiced speech) or at the boundaries between unvoiced segments. An ecien t algorithm for inferring these boundaries is derived from a simple probabilistic generative model of speech and state of the art results are presented on pitch tracking, voiced/unvoiced detection and timescale modication.
---
paper_title: On the automatic segmentation of speech signals

paper_content:

For large vocabulary and continuous speech recognition, the sub-word-unit-based approach is a viable alternative to the whole-word-unit-based approach. For preparing a large inventory of subword units, an automatic segmentation is preferrable to manual segmentation as it substantially reduces the work associated with the generation of templates and gives more consistent results. In this paper we discuss some methods for automatically segmenting speech into phonetic units. Three different approaches are described, one based on template matching, one based on detecting the spectral changes that occur at the boundaries between phonetic units and one based on a constrained-clustering vector quantization approach. An evaluation of the performance of the automatic segmentation methods is given.
---
paper_title: Inclusion of temporal information into features for speech recognition

paper_content:

Conventional methods for incorporating temporal information into speech features apply regression to a series of successive cepstral vectors to generate differential cepstra, or apply a cosine transform to generate cepstral-time matrices. This paper aims to generalise these techniques such that a series of stacked cepstral vectors is multiplied by a temporal transform matrix to produce the final speech feature. This can made to incorporate both static and dynamic speech information. Using this method, the coding of temporal information is not restricted to regression or cosine coefficients-any suitable transform may be used. Results are presented for a variety of transforms, such as Legendre, Karhunen-Loeve, Cosine, Rectangle, where it is shown that the transform based techniques offer higher performance than conventional differential cepstrum.
---
paper_title: Coherent envelope detection for modulation filtering of speech

paper_content:

Modulation filtering, which has been previously described as several related approaches to achieve modification of speech temporal dynamics, is shown to be less effective than intended. In particular, past Hilbert envelope approaches generate distortion which spreads across frequency sub-bands and modulation rejection is far from the amount intended. The source of this distortion is analyzed and a solution, based upon coherent envelope detection in each sub-band is proposed. This coherent approach is shown to be substantially more effective than conventional incoherent approaches on speech samples.
---
paper_title: Vocal tract length invariant features for automatic speech recognition

paper_content:

The effects of vocal tract length (VTL) variation are often approximated by linear frequency warping of short-time spectra. Based on this relationship, we present a method for generating vocal tract length invariant features. These new features are computed as translation invariant, correlation-type features in a log-frequency domain. In phoneme classification experiments, their discrimination capabilities turned out to be considerably better than for Mel-frequency cepstral coefficients (MFCCs). The best results are obtained when VTL-invariant (VTLI) features and MFCCs are combined. The superiority of the combined feature set and its resilience to VTL variations is also shown for word recognition, using the TIDIGITS corpus and the HTK recognizer
---
paper_title: Cepstral domain stress compensation for robust speech recogniton

paper_content:

Automtic speech recognition algorithms generally rely on the assumption that for the distance measure used, intraword variabilities are smaller than interword variabilities so that appropriate separation in the measurement space is possible. As evidenced by degradation of recognition perforrmnce, the validity of such an assumption decreases from simple tasks to complex tasks, from cooperative talkers to casual talkers, and from laboratory talking environments to practical talking environments. This paper presents a study of talker- stress-induced intraword variability, and an algorithm that commpensates for the systematic changes observed. The study is based on Hidden Markov Models trained by speech tokens in various talking styles. The talking styles include normal speech, fast speech, loud speech, soft speech, and talking with noise injected through earphones; the styles are designed to simulate speech produced under real stressful conditions. Cepstral coefficients are used as the parameters in the Hidden Markov Models. The stress compensation algorithm compensates for the variations in the cepstral coefficients in a hypothesis-driven manner. The functional form of the compensation is shown to correspond to the equalization of spectral tilts. Preliminary experiments indicate that a substantial reduction in recognition error rate can be achieved with relatively little increase in computation and storage requirements.
---
paper_title: Scale transform in speech analysis

paper_content:

In this paper, we study the scale transform of the spectral-envelope of speech utterances by different speakers. This study is motivated by the hypothesis that the formant frequencies between different speakers are approximately related by a scaling constant for a given vowel. The scale transform has the fundamental property that the magnitude of the scale-transform of a function X(f) and its scaled version /spl radic//spl alpha/X(/spl alpha/f) are same. The methods presented here are useful in reducing variations in acoustic features. We show that the F-ratio tests indicate better separability of vowels by using scale-transform based features than mel-transform based features. The data used in the comparison of the different features consist of 200 utterances of four vowels that are extracted from the TIMIT database.
---
paper_title: A comparison of several acoustic representations for speech recognition with degraded and undegraded speech

paper_content:

Several acoustic representations have been compared in speaker-dependent and independent connected and isolated-word recognition tests with undegraded speech and with speech degraded by adding white noise and by applying a 6-dB/octave spectral tilt. The representations comprised the output of an auditory model, cepstrum coefficients derived from an FFT-based mel-scale filter bank with various weighting schemes applied to the coefficients, cepstrum coefficients augmented with measures of their rates of change with time, and sets of linear discriminant functions derived from the filter-bank output and called IMELDA. The model outperformed the cepstrum representations except in noise-free connected-word tests, where it had a high insertion rate. The best cepstrum weighting scheme was derived from within-class variances. Its behavior may explain the empirical adjustments found necessary with other schemes. IMELDA outperformed all other representations in all conditions and is computationally simple. >
---
paper_title: Improvements in children's speech recognition performance

paper_content:

There are several reasons why conventional speech recognition systems modeled on adult data fail to perform satisfactorily on children's speech input. For instance, children's vocal characteristics differ significantly from those of adults. In addition, their choices of vocabulary and sentence construction modalities usually do not conform to adult patterns. We describe comparative studies demonstrating the performance gain realized by adopting to children's acoustic and language model data to construct a children's speech recognition system.
---
paper_title: Speaker modification with LPC pole analysis

paper_content:

Speaker modification is the ability to change the perceived speaker identity of a recorded utterance. Basic to this is the capability to alter the vowel segments of speech. Not only do these segments comprise the majority of the voiced portion of speech but they are dominated by clearly defined acoustic parameters-formant frequencies and pitch. A method of altering the formant frequencies of vowel segments using LPC analysis/synthesis was investigated. Pole location modification based on statistical references provided individual control over formant frequencies and bandwidths but, in some transformations, lead to artifacts in the reconstructed speech.
---
paper_title: Speaker normalization based on frequency warping

paper_content:

In speech recognition, speaker-dependence of a speech recognition system comes from speaker-dependence of the speech feature, and the variation of vocal tract shape is the major source of inter-speaker variations of the speech feature, though there are some other sources which also contribute. In this paper, we address the approach of speaker normalization which aims at normalizing speaker's vocal tract length based on frequency warping (FWP). The FWP is implemented in the front-end preprocessing of our speech recognition system. We investigate the formant-based and ML-based FWP in linear and nonlinear warping modes, and compare them in detail. All experimental results are based on our JANUS3 large vocabulary continuous speech recognition system and the Spanish Spontaneous Scheduling Task database (SSST).
---
paper_title: Speaker normalization using efficient frequency warping procedures

paper_content:

In an effort to reduce the degradation in speech recognition performance caused by variation in vocal tract shape among speakers, a frequency warping approach to speaker normalization is investigated. A set of low complexity, maximum likelihood based frequency warping procedures have been applied to speaker normalization for a telephone based connected digit recognition task. This paper presents an efficient means for estimating a linear frequency warping factor and a simple mechanism for implementing frequency warping by modifying the filter-bank in mel-frequency cepstrum feature analysis. An experimental study comparing these techniques to other well-known techniques for reducing variability is described. The results showed that frequency warping was consistently able to reduce word error rate by 20% even for very short utterances.
---
paper_title: Speaker adaptive modeling by vocal tract normalization

paper_content:

This paper presents methods for speaker adaptive modeling using vocal tract normalization (VTN) along with experimental tests on three databases. We propose a new training method for VTN: By using single-density acoustic models per HMM state for selecting the scale factor of the frequency axis, we avoid the problem that a mixture-density tends to learn the scale factors of the training speakers and thus cannot be used for selecting the scale factor. We show that using single Gaussian densities for selecting the scale factor in training results in lower error rates than using mixture densities. For the recognition phase, we propose an improvement of the well-known two-pass strategy: by using a non-normalized acoustic model for the first recognition pass instead of a normalized model, lower error rates are obtained. In recognition tests, this method is compared with a fast variant of VTN. The two-pass strategy is an efficient method, but it is suboptimal because the scale factor and the word sequence are determined sequentially. We found that for telephone digit string recognition this suboptimality reduces the VTN gain in recognition performance by 30% relative. In summary, on the German spontaneous speech task Verbmobil, the WSJ task and the German telephone digit string corpus SieTill, the proposed methods for VTN reduce the error rates significantly.
---
paper_title: Normalization of vowels by vocal - tract length and its application to vowel identificatiom

paper_content:

A hammer formed of a solid metal body of uniform thickness having a working end and a mounting end is reinforced by an increase in cross-sectional surface area at the mounting end, and provided with a channel around the mounting end and along the walls, relieved toward the longitudinal axis of the hammer, which connect the ends, both the relievement and the channel reducing the weight of the hammer remote from the working end to locate the center of mass closer to the working end.
---
paper_title: Perceptual linear predictive (PLP) analysis of speech

paper_content:

A new technique for the analysis of speech, the perceptual linear predictive (PLP) technique, is presented and examined. This technique uses three concepts from the psychophysics of hearing to derive an estimate of the auditory spectrum: (1) the critical-band spectral resolution, (2) the equal-loudness curve, and (3) the intensity-loudness power law. The auditory spectrum is then approximated by an autoregressive all-pole model. A 5th-order all-pole model is effective in suppressing speaker-dependent details of the auditory spectrum. In comparison with conventional linear predictive (LP) analysis, PLP analysis is more consistent with human hearing. The effective second formant F2' and the 3.5-Bark spectral-peak integration theories of vowel perception are well accounted for. PLP analysis is computationally efficient and yields a low-dimensional representation of speech. These properties are found to be useful in speaker-independent automatic-speech recognition.
---
paper_title: Auditory-based acoustic distinctive features and spectral cues for automatic speech recognition using a multi-stream paradigm

paper_content:

In this paper, a multi-stream paradigm is proposed to improve the performance of automatic speech recognition (ASR) systems. Our goal in this paper is to improve the performance of the HMM-based ASR systems by exploiting some features that characterize speech sounds based on the auditory system and one based on the Fourier power spectrum. It was found that combining the classical MFCCs with some auditory-based acoustic distinctive cues and the main peaks of the spectrum of a speech signal using a multi-stream paradigm leads to an improvement in the recognition performance. The Hidden Markov Model Toolkit (HTK) was used throughout our experiments to test the use of the new multi-stream feature vector. A series of experiments on speaker-independent continuous-speech recognition have been carried out using a subset of the large read-speech corpus TIMIT. Using such multi-stream paradigm, N-mixture mono-/tri-phone models and a bigram language model, we found that the word error rate was decreased by about 4.01%.
---
paper_title: Sub-Band Based Recognition Of Noisy Speech

paper_content:

A new approach to automatic speech recognition based on independent class-conditional probability estimates in several frequency sub-bands is presented. The approach is shown to be especially applicable to environments which cause partial corruption of the frequency spectrum of the signal. Some of the issues involved in the implementation of the approach are also addressed.
---
paper_title: Robust ASR front-end using spectral-based and discriminant features: experiments on the Aurora tasks

paper_content:

This paper describes an automatic speech recognition frontend that combines low-level robust ASR feature extraction techniques, and higher-level linear and non-linear feature transformations. The low-level algorithms use data-derived filters, mean and variance normalization of the feature vectors, and dropping of noise frames. The feature vectors are then linearly transformed using Principal Components Analysis (PCA). An Artificial Neural Network (ANN) is also used to compute features that are useful for classification of speech sounds. It is trained for phoneme probability estimation on a large corpus of noisy speech. These transformations lead to two feature streams whose vectors are concatenated and then used for speech recognition. This method was tested on the set of speech corpora used for the “Aurora” evaluation. Using the feature stream generated without the ANN yields an overall 41% reduction of the error rate over Mel-Frequency Cepstral Coefficients (MFCC) reference features. Adding the ANN stream further reduces the error rate yielding a 46% reduction over the reference features.
---
paper_title: Improving word accuracy with Gabor feature extraction

paper_content:

A novel type of feature extraction for automatic speech recognition is investigated. Two-dimensional Gabor functions, with varying extents and tuned to different rates and directions of spectro-temporal modulation, are applied as filters to a spectro-temporal representation provided by mel spectra. The use of these functions is motivated by findings in neurophysiology and psychoacoustics. Data-driven parameter selection was used to obtain Gabor feature sets, the performance of which is evaluated on the Aurora 2 and 3 datasets both on their own and in combination with the Qualcomm-OGI-ICSI Aurora proposal. The Gabor features consistently provide performance improvements.
---
paper_title: Modelling asynchrony in speech using elementary single-signal decomposition

paper_content:

Although the possibility of asynchrony between different components of the speech spectrum has been acknowledged, its potential effect on automatic speech recogniser performance has only recently been studied. This paper presents the results of continuous speech recognition experiments in which such asynchrony is accommodated using a variant of HMM decomposition. The paper begins with an investigation of the effects of partitioning the speech spectrum explicitly into subbands. Asynchrony between these sub-bands is then accommodated, resulting in a significant decrease in word errors. The same decomposition technique has previously been used successfully to compensate for asynchrony between the two input streams in an audiovisual speech recognition system.
---
paper_title: Noise robust speech parameterization using multiresolution feature extraction

paper_content:

In this paper, we present a multiresolution-based feature extraction technique for speech recognition in adverse conditions. The proposed front-end algorithm uses mel cepstrum-based feature computation of subbands in order not to spread noise distortions over the entire feature space. Conventional full-band features are also augmented to the final feature vector which is fed to the recognition unit. Other novel features of the proposed front-end algorithm include emphasis of long-term spectral information combined with cepstral domain feature vector normalization and the use of the PCA transform, instead of DCT, to provide the final cepstral parameters. The proposed algorithm was experimentally evaluated in a connected digit recognition task under various noise conditions. The results obtained show that the new feature extraction algorithm improves word recognition accuracy by 41 % when compared to the performance of mel cepstrum front-end. A substantial increase in recognition accuracy was observed in all tested noise environments at all different SNRs. The good performance of the multiresolution front-end is not only due to the higher feature vector dimension, but the proposed algorithm clearly outperformed the mel cepstral front-end when the same number of HMM parameters were used in both systems. We also propose methods to reduce the computational complexity of the multiresolution front-end-based speech recognition system. Experimental results indicate the viability of the proposed techniques.
---
paper_title: Robust Speech Recognition Using A Voiced-Unvoiced Feature

paper_content:

In this paper, a voiced-unvoiced measure is used as acoustic feature for continuous speech recognition. The voiced-unvoiced measure was combined with the standard Mel Frequency Cepstral Coefficients (MFCC) using linear discriminant analysis (LDA) to choose the most relevant features. Experiments were performed on the SieTill (German digit strings recorded over telephone line) and on the SPINE (English spontaneous speech under different simulated noisy environments) corpus. The additional voiced-unvoiced measure results in improvements in word error rate (WER) of up to 11% relative to using MFCC alone with the same overall number of parameters in the system.
---
paper_title: Combining Articulatory And Acoustic Information For Speech Recognition In Noisy And Reverberant Environments

paper_content:

Robust speech recognition under varying acoustic conditions may be achieved by exploiting multiple sources of information in the speech signal. In addition to an acoustic signal representation, we use an articulatory representation consisting of pseudo-articulatory features as an additional information source. Hybrid ANN/HMM recognizers using either of these representations are evaluated on a continuous numbers recognition task (OGI Numbers95) under clean, reverberant and noisy conditions. An error analysis of preliminary recognition results shows that the different representations produce qualitatively different errors, which suggests a combination of both representations. We investigate various combination possibilities at the phoneme estimation level and show that significant improvements can been achieved under all three acoustic conditions.
---
paper_title: Multi-resolution phonetic/segmental features and models for HMM-based speech recognition

paper_content:

This paper explores the modelling of phonetic segments of speech with multi-resolution spectral/time correlates. For spectral representation a set of multi-resolution cepstral features are proposed. Cepstral features obtained from a DCT of the log energy-spectrum over the full voice-bandwidth (100-4000 Hz) are combined with higher resolution features obtained from the DCT of upper subband (say 100-2100) and lower subband (2100-4000) halves. This approach can be extended to several levels of different resolutions. For representation of the temporal structure of speech segments or phonetic units, the conventional cepstral and dynamic cepstral features representing speech at the sub-phonetic levels, are supplemented by a set of phonetic features that describe the trajectory of speech over the duration of a phonetic unit. A conditional probability model for phonetic and sub-phonetic features is considered. Experiments demonstrate that the inclusion of the segmental features result in about 10% decrease in error rates.
---
paper_title: Use of periodicity and jitter as speech recognition features

paper_content:

We investigate a class of features related to voicing parameters that indicate whether the vocal chords are vibrating. Features describing voicing characteristics of speech signals are integrated with an existing 38-dimensional feature vector consisting of first and second order time derivatives of the frame energy and of the cepstral coefficients with their first and second derivatives. HMM-based connected digit recognition experiments comparing the traditional and extended feature sets show that voicing features and spectral information are complementary and that improved speech recognition performance is obtained by combining the two sources of information.
---
paper_title: Voicing feature integration in SRI's decipher LVCSR system

paper_content:

We augment the Mel cepstral (MFCC) feature representation with voicing features from an independent front end. The voicing feature front end parameters are optimized for recognition accuracy. The voicing features computed are the normalized autocorrelation peak and a newly proposed entropy of the high-order cepstrum. We explored several alternatives to integrate the voicing features into SRI's DECIPHER system. Promising early results were obtained in a simple system concatenating the voicing features with MFCC features and optimizing the voicing feature window duration. Best results overall came from a more complex system combining a multiframe voicing feature window with the MFCC plus third differential features using linear discriminant analysis and optimizing the number of voicing feature frames. The best integration approach from the single-pass system experiments was implemented in a multi-pass system for large vocabulary testing on the Switchboard database. An average WER reduction of 2% relative was obtained on the NIST Hub-5 dev2001 and eval2002 databases.
---
paper_title: Minimum phoneme error based heteroscedastic linear discriminant analysis for speech recognition

paper_content:

We introduce a discriminative feature analysis method that seeks to minimize phoneme errors in lattice-based training frameworks. This technique, referred to as minimum phoneme error heteroscedastic linear discriminant analysis (MPE-HLDA), is shown to be more robust than traditional LDA methods in high dimensional spaces, and easy to incorporate with existing training procedures, such as HLDA-SAT and discriminative training of hidden Markov models (HMMs). Results on conversational telephone speech and broadcast news corpora also show that the recognition accuracy is improved using features selected by MPE-HLDA.
---
paper_title: Maximum likelihood modeling with Gaussian distributions for classification

paper_content:

Maximum likelihood (ML) modeling of multiclass data for classification often suffers from the following problems: (a) data insufficiency implying overtrained or unreliable models, (b) large storage requirement, (c) large computational requirement and/or (d) the ML is not discriminating between classes. Sharing parameters across classes (or constraining the parameters) clearly tends to alleviate the first three problems. We show that in some cases it can also lead to better discrimination (as evidenced by reduced misclassification error). The parameters considered are the means and variances of the Gaussians and linear transformations of the feature space (or equivalently the Gaussian means). Some constraints on the parameters are shown to lead to linear discrimination analysis (a well-known result) while others are shown to lead to optimal feature spaces (a relatively new result). Applications of some of these ideas to the speech recognition problem are also given.
---
paper_title: Maximum likelihood discriminant feature spaces

paper_content:

Linear discriminant analysis (LDA) is known to be inappropriate for the case of classes with unequal sample covariances. There has been an interest in generalizing LDA to heteroscedastic discriminant analysis (HDA) by removing the equal within-class covariance constraint. This paper presents a new approach to HDA by defining an objective function which maximizes the class discrimination in the projected subspace while ignoring the rejected dimensions. Moreover, we investigate the link between discrimination and the likelihood of the projected samples and show that HDA can be viewed as a constrained ML projection for a full covariance Gaussian model, the constraint being given by the maximization of the projected between-class scatter volume. It is shown that, under diagonal covariance Gaussian modeling constraints, applying a diagonalizing linear transformation (MLLT) to the HDA space results in increased classification accuracy even though HDA alone actually degrades the recognition performance. Experiments performed on the Switchboard and Voicemail databases show a 10%-13% relative improvement in the word error rate over standard cepstral processing.
---
paper_title: Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition

paper_content:

We present the theory for heteroscedastic discriminant analysis (HDA), a model-based generalization of linear discriminant analysis (LDA) derived in the maximum-likelihood framework to handle heteroscedastic-unequal variance-classifier models. We show how to estimate the heteroscedastic Gaussian model parameters jointly with the dimensionality reducing transform, using the EM algorithm. In doing so, we alleviate the need for an a priori ad hoc class assignment. We apply the theoretical results to the problem of speech recognition and observe word-error reduction in systems that employed both diagonal and full covariance heteroscedastic Gaussian models tested on the TI-DIGITS database.
---
paper_title: Maximizing information content in feature extraction

paper_content:

In this paper, we consider the problem of quantifying the amount of information contained in a set of features, to discriminate between various classes. We explore these ideas in the context of a speech recognition system, where an important classification sub-problem is to predict the phonetic class, given an observed acoustic feature vector. The connection between information content and speech recognition system performance is first explored in the context of various feature extraction schemes used in speech recognition applications. Subsequently, the idea of optimizing the information content to improve recognition accuracy is generalized to a linear projection of the underlying features. We show that several prior methods to compute linear transformations (such as linear/heteroscedastic discriminant analysis) can be interpreted in this general framework of maximizing the information content. Subsequently, we extend this reasoning and propose a new objective function to maximize a penalized mutual information (pMI) measure. This objective function is seen to be very well correlated with the word error rate of the final system. Finally experimental results are provided that show that the proposed pMI projection consistently outperforms other methods for a variety of cases, leading to relative improvements in the word error rate of 5%-16% over earlier methods.
---
paper_title: Introduction to Statistical Pattern Recognition

paper_content:

This completely revised second edition presents an introduction to statistical pattern recognition. Pattern recognition in general covers a wide range of problems: it is applied to engineering problems, such as character readers and wave form analysis as well as to brain modeling in biology and psychology. Statistical decision and estimation, which are the main subjects of this book, are regarded as fundamental to the study of pattern recognition. This book is appropriate as a text for introductory courses in pattern recognition and as a reference book for workers in the field. Each chapter contains computer projects as well as exercises.
---
paper_title: Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion

paper_content:

We propose an eigenvector-based heteroscedastic linear dimension reduction (LDR) technique for multiclass data. The technique is based on a heteroscedastic two-class technique which utilizes the so-called Chernoff criterion, and successfully extends the well-known linear discriminant analysis (LDA). The latter, which is based on the Fisher criterion, is incapable of dealing with heteroscedastic data in a proper way. For the two-class case, the between-class scatter is generalized so to capture differences in (co)variances. It is shown that the classical notion of between-class scatter can be associated with Euclidean distances between class means. From this viewpoint, the between-class scatter is generalized by employing the Chernoff distance measure, leading to our proposed heteroscedastic measure. Finally, using the results from the two-class case, a multiclass extension of the Chernoff criterion is proposed. This criterion combines separation information present in the class mean as well as the class covariance matrices. Extensive experiments and a comparison with similar dimension reduction techniques are presented.
---
paper_title: Acoustic space dimensionality selection and combination using the maximum entropy principle

paper_content:

We propose a discriminative approach to acoustic space dimensionality selection based on maximum entropy modelling. We form a set of constraints by composing the acoustic space with the space of phone classes, and use a continuous feature formulation of maximum entropy modelling to select an optimal feature set. The suggested approach has two steps: (1) the selection of the best acoustic space that efficiently and economically represents the acoustic data and its variability; (2) the combination of selected acoustic features in the maximum entropy framework to estimate the posterior probabilities over the phonetic labels given the acoustic input. Specific contributions of the paper include a parameter estimation algorithm (generalized improved iterative scaling) that enables the use of negative features, the parameterization of constraint functions using Gaussian mixture models, and experimental results using the TIMIT database.
---
paper_title: Improving Eigenspace-based MLLR Adaptation by Kernel PCA

paper_content:

Eigenspace-based MLLR (EMLLR) adaptation has been shown effective for fast speaker adaptation. It applies the basic idea of eigenvoice adaptation, and derives a small set of eigenmatrices using principal component analysis (PCA). The MLLR adaptation transformation of a new speaker is then a linear combination of the eigenmatrices. In this paper, we investigate the use of kernel PCA to find the eigenmatrices in the kernel-induced high dimensional feature space so as to exploit possible nonlinearity in the transformation supervector space. In addition, composite kernel is used to preserve the row information in the transformation supervector which, otherwise, will be lost during the mapping to the kernel-induced feature space. We call our new method kernel eigenspace-based MLLR (KEMLLR) adaptation. On a RM adaptation task, we find that KEMLLR adaptation may reduce the word error rate of a speaker-independent model by 11%, and outperforms MLLR and EMLLR adaptation.
---
paper_title: Eigenvoice modeling with sparse training data

paper_content:

We derive an exact solution to the problem of maximum likelihood estimation of the supervector covariance matrix used in extended MAP (or EMAP) speaker adaptation and show how it can be regarded as a new method of eigenvoice estimation. Unlike other approaches to the problem of estimating eigenvoices in situations where speaker-dependent training is not feasible, our method enables us to estimate as many eigenvoices from a given training set as there are training speakers. In the limit as the amount of training data for each speaker tends to infinity, it is equivalent to cluster adaptive training.
---
paper_title: Speaker normalization and adaptation using second-order connectionist networks

paper_content:

A method for speaker normalization and adaption using connectionist networks is developed. A speaker-specific linear transformation of observations of the speech signal is computed using second-order network units. Classification is accomplished by a multilayer feedforward network that operates on the normalized speech data. The network is adapted for a new talker by modifying the transformation parameters while leaving the classifier fixed. This is accomplished by backpropagating classification error through the classifier to the second-order transformation units. This method was evaluated for the classification of ten vowels for 76 speakers using the first two formant values of the Peterson-Barney data. The results suggest that rapid speaker adaptation resulting in high classification accuracy can be accomplished by this method. >
---
paper_title: Segmental eigenvoice with delicate eigenspace for improved speaker adaptation

paper_content:

Eigenvoice techniques have been proposed to provide rapid speaker adaptation with very limited adaptation data, but the performance may be saturated when more adaptation data become available. This is because in these techniques an eigenspace with reduced dimensionality is established by properly utilizing the a priori knowledge from the large quantity of training data. The reduced dimensionality of the eigenspace requires less adaptation data to estimate the model parameters for the new speaker, but also makes it less easy to obtain more precise models with more adaptation data. In this paper, a new segmental eigenvoice approach is proposed, in which the eigenspace can be further segmented into N subeigenspaces by properly classifying the model parameters into N clusters. These N subeigenspaces can help to construct a more delicate eigenspace and more precise models when more adaptation data are available. It will be shown that there can be at least mixture-based, model-based and feature-based segmental eigenvoice approaches. Not only improved performance can be obtained, but these different approaches can be properly integrated to offer better performance. Two further approaches leading to improved segmental eigenvoice techniques with even better performance are also proposed. The experiments were performed with both a large vocabulary and a small vocabulary recognition tasks.
---
paper_title: Eigenvoices: A compact representation of speakers in model space

paper_content:

In this article, we present a new approach to modeling speaker-dependent systems. The approach was inspired by the eigenfaces techniques used in face recognition. We build a linear vector space of low dimensionality, called eigenspace, in which speakers are located. The basis vectors of this space are called eigenvoices. Each eigenvoice models a direction of inter-speaker variability. The eigenspace is built during the training phase. Then, any speaker model can be expressed as a linear combination of eigenvoices. The benefits of this technique as set forth in this article reside in the reduction of the number of parameters that describe a model. Thereby we are able to reduce the number of parameters to estimate, as well as computation and/or storage costs. We apply the approach to speaker adaptation and speaker recognition. Some experimental results are supplied.
---
paper_title: Rapid online adaptation using speaker space model evolution

paper_content:

This paper presents a new approach to online adaptation of continuous density hidden Markov model (CDHMM) with a small amount of adaptation data based on speaker space model (SSM) evolution. The SSM which characterizes the a priori knowledge of the training speakers is effectively described in terms of the latent variable models such as the factor analysis or probabilistic principal component analysis. The SSM provides various sources of information such as the correlation information, the prior density, and the prior knowledge of the CDHMM parameters that are very useful for rapid online adaptation. We design the SSM evolution based on the quasi-Bayes estimation technique which incrementally updates the hyperparameters of the SSM and the CDHMM parameters simultaneously. In a series of speaker adaptation experiments on the continuous digit and large vocabulary recognition tasks, we demonstrate that the proposed approach not only achieves a good performance for a small amount of adaptation data but also maintains a good asymptotic convergence property as the data size increases.
---
paper_title: Speech recognition in noise using a projection-based likelihood measure for mixture density HMM's

paper_content:

In this study, a cepstral likelihood measure based on the projection operation is incorporated into a mixture density hidden Markov model (HMM) scheme to improve recognition in the presence of additive noise. The case in which the models are determined only under noise-free conditions is addressed. A background discussion and a derivation of the measure are provided. Recognition experiments are presented showing the usefulness of the proposed measure over the standard Gaussian measure (weighted Euclidean distance) for speaker independent, isolated word recognition in noise. It was found that the proposed mixture weighted projection measure significantly improved performance in several noise types, including white, jittering white, and colored noise. As an example, at an SNR of 10-dB white noise, recognition improved from only 38.4% correct using the Gaussian measure to 83.6% using the developed measure. >
---
paper_title: Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models

paper_content:

Abstract A method of speaker adaptation for continuous density hidden Markov models (HMMs) is presented. An initial speaker-independent system is adapted to improve the modelling of a new speaker by updating the HMM parameters. Statistics are gathered from the available adaptation data and used to calculate a linear regression-based transformation for the mean vectors. The transformation matrices are calculated to maximize the likelihood of the adaptation data and can be implemented using the forward–backward algorithm. By tying the transformations among a number of distributions, adaptation can be performed for distributions which are not represented in the training data. An important feature of the method is that arbitrary adaptation data can be used—no special enrolment sentences are needed. Experiments have been performed on the ARPA RM1 database using an HMM system with cross-word triphones and mixture Gaussian output distributions. Results show that adaptation can be performed using as little as 11 s of adaptation data, and that as more data is used the adaptation performance improves. For example, using 40 adaptation utterances, a 37% reduction in error from the speaker-independent system was achieved with supervised adaptation and a 32% reduction in unsupervised mode.
---
paper_title: Maximum a posteriori adaptation for large scale HMM recognizers

paper_content:

We present a framework for maximum a posteriori (MAP) adaptation of large scale HMM recognizers. First we review the standard MAP adaptation for Gaussian mixtures. We then show how MAP can be used to estimated transformations which are shared across many parameters. Finally, we combine both techniques: each of the HMM models is adapted based on an interpolation of MAP estimates obtained under varying degrees of sharing. We evaluate this algorithm for adaptation of a continuous density HMM with 96 K Gaussians and show that very satisfactory improvements can be achieved, especially for adaptation of non-native speakers of American English.
---
paper_title: Acoustic adaptation using nonlinear transformations of HMM parameters

paper_content:

Speech recognition performance degrades significantly when there is a mismatch between testing and training conditions. Linear transformation-based maximum-likelihood (ML) techniques have been proposed recently to tackle this problem. We extend this approach to use nonlinear transformations. These are implemented by multilayer perceptrons (MLPs) which transform the Gaussian means. We derive a generalized expectation-maximization (GEM) training algorithm to estimate the MLP weights. Some preliminary experimental results on nonnative speaker adaptation are presented.
---
paper_title: Comparative Experiments On Large Vocabulary Speech Recognition

paper_content:

We describe recent changes to the BYBLOS system's training and recognition algorithms and report on numerous experiments in large vocabulary speech recognition. In earlier work, we performed five key experiments that were designed to answer questions related to different training scenarios. We investigated (1) the effect of varying the number of training speakers if the total amount of training data remains constant, (2) data pooling versus model averaging for generating speaker-independent (SI) HMMs, (3) the benefit of doubling the acoustic training data, (4) SI versus SD performance when the SI training data is twelve times greater, (5) the effect of cross-domain training for both the acoustic and language models. Our recent work was focused on four specific problem areas sharing the common thread that the test condition exposes the recognizer to phenomena not observed in the training data. Here we investigated (1) words outside the vocabulary, (2) spoken language effects due to subject variability and spontaneous dictation, (3) non-native dialects of the language, and (4) new microphones not used in training. >
---
paper_title: Discriminative linear transforms for feature normalization and speaker adaptation in HMM estimation

paper_content:

Linear transforms have been used extensively for training and adaptation of HMM-based ASR systems. Recently procedures have been developed for the estimation of linear transforms under the Maximum Mutual Information (MMI) criterion. In this paper we introduce discriminative training procedures that employ linear transforms for feature normalization and for speaker adaptive training. We integrate these discriminative linear transforms into MMI estimation of HMM parameters for improvement of large vocabulary conversational speech recognition systems.
---
paper_title: Rapid discriminative acoustic model based on eigenspace mapping for fast speaker adaptation

paper_content:

It is widely believed that strong correlations exist across an utterance as a consequence of time-invariant characteristics of speaker and acoustic environments. It is verified in this paper that the first primary eigendirections of the utterance covariance matrix are speaker dependent. Based on this observation, a novel family of fast speaker adaptation algorithms entitled Eigenspace Mapping (EigMap) is proposed. The proposed algorithms are applied to continuous density Hidden Markov Model (HMM) based speech recognition. The EigMap algorithm rapidly constructs discriminative acoustic models in the test speaker's eigenspace by preserving discriminative information learned from baseline models in the directions of the test speaker's eigenspace. Moreover, the adapted models are compressed by discarding model parameters that are assumed to contain no discrimination information. The core idea of EigMap can be extended in many ways, and a family of algorithms based on EigMap is described in this paper. Unsupervised adaptation experiments show that EigMap is effective in improving baseline models using very limited amounts of adaptation data with superior performance to conventional adaptation techniques such as MLLR and block diagonal MLLR. A relative improvement of 18.4% over a baseline recognizer is achieved using EigMap with only about 4.5 s of adaptation data. Furthermore, it is also demonstrated that EigMap is additive to MLLR by encompassing important speaker dependent discriminative information. A significant relative improvement of 24.6% over baseline is observed using 4.5 s of adaptation data by combining MLLR and EigMap techniques.
---
paper_title: Maximum-likelihood nonlinear transformation for acoustic adaptation

paper_content:

In this paper, we describe an adaptation method for speech recognition systems that is based on a nonlinear transformation of the feature space. In contrast to most existing adaptation methods which assume some form of affine transformation of either the feature vectors or the acoustic models that model the feature vectors, our proposed method composes a general nonlinear transformation from two transformations, one of these being an affine transformation that combines the dimensions of the original feature space, and the other being a nonlinear transformation that is applied independently to each dimension of the previously transformed feature space leading to a general multidimensional nonlinear transformation of the original feature space. This method also differs from other affine techniques in the way the parameters of the transform are shared. In most previous methods, the parameters of the transformation are shared on the basis of the phonetic class, in our method, the parameters of the nonlinear transformation are shared not on the basis of the phonetic class, but rather on the location in the feature space. Experimental results show that the method outperforms affine methods providing up to a 25% relative improvement in the word error rate in an in-car speech recognition task.
---
paper_title: Speaker Clustering And Transformation For Speaker Adaptation In Large-Vocabulary Speech Recognition Systems

paper_content:

A speaker adaptation strategy is described that is based on finding a subset of speakers, from the training set, who are acoustically close to the test speaker, and using only the data from these speakers (rather than the complete training corpus) to re-estimate the system parameters. Further, a linear transformation is computed for every one of the selected training speakers to better map the training speaker's data to the test speaker's acoustic space. Finally, the system parameters (Gaussian means) are re-estimated specifically for the test speaker using the transformed data from the selected training speakers. Experiments showed that this scheme is capable of reducing the error rate by 10-15% with the use of as little as 3 sentences of adaptation data.
---
paper_title: Robust recognition of children's speech

paper_content:

Developmental changes in speech production introduce age-dependent spectral and temporal variability in the speech signal produced by children. Such variabilities pose challenges for robust automatic recognition of children's speech. Through an analysis of age-related acoustic characteristics of children's speech in the context of automatic speech recognition (ASR), effects such as frequency scaling of spectral envelope parameters are demonstrated. Recognition experiments using acoustic models trained from adult speech and tested against speech from children of various ages clearly show performance degradation with decreasing age. On average, the word error rates are two to five times worse for children speech than for adult speech. Various techniques for improving ASR performance on children's speech are reported. A speaker normalization algorithm that combines frequency warping and model transformation is shown to reduce acoustic variability and significantly improve ASR performance for children speakers (by 25-45% under various model training and testing conditions). The use of age-dependent acoustic models further reduces word error rate by 10%. The potential of using piece-wise linear and phoneme-dependent frequency warping algorithms for reducing the variability in the acoustic feature space of children is also investigated.
---
paper_title: A family of distortion measures based upon projection operation for robust speech recognition

paper_content:

Consideration is given to the formulation of speech similarity measures, a fundamental component in recognizer designs, that are robust to the change of ambient conditions. The authors focus on the speech cepstrum derived from linear prediction coefficients (the LPC cepstrum). By using some common models for noisy speech, they show analytically that additive white noise reduces the norm (length) of the LPC cepstral vectors. Empirical observations on the parameter histograms not only confirm the analytical results through the use of noise models but further reveal that at a given (global) signal-to-noise ratio (SNR), the norm reduction on cepstral vectors with larger norms is generally less than on vectors with smaller norms, and that lower order coefficients are more affected than higher order terms. In addition, it is found that the orientation (or direction) of the cepstral vector is less susceptible to noise perturbation than the vector norm. As a consequence of the above results, a family of distortion measures based on the projection between two cepstral vectors is proposed. The new measures have the same computational efficiency as the band-pass cepstral distortion measure. >
---
paper_title: Speaker adaptation based on MAP estimation of HMM parameters

paper_content:

A number of issues related to the application of Bayesian learning techniques to speaker adaptation are investigated. It is shown that the seed models required to construct prior densities to obtain the MAP (maximum a posteriori) estimate can be a speaker-independent (SI) model, a set of female and male models, or even a task-independent acoustic model. Speaker-adaptive training algorithms are shown to be effective in improving the performance of both speaker-dependent and speaker-independent speech recognition systems. The segmental MAP estimation formulation is used to perform adaptive acoustic modeling for speaker adaptation applications. Tested on an RM (resource management) task, it was found that supervised speaker adaptation based on two gender-dependent models gave a better result than that obtained with a single SI seed. Compared with speaker-dependent training, speaker adaptation achieved an equal or better performance with the same amount of training/adaptation data. >
---
paper_title: Vocal Tract Length Normalization for Large Vocabulary Continuous Speech Recognition

paper_content:

Abstract : Generally speaking, the speaker-dependence of a speech recognition system stems from speaker-dependent speech feature. The variation of vocal tract length and/or shape is one of the major source of inter-speaker variations. In this paper, we address several methods of vocal tract length normalization (VTLN) for large vocabulary continuous speech recognition: (1) explore the bilinear warping VTLN in frequency domain; (2) propose a speaker-specific Bark/Mel scale VTLN in Bark/Mel domain; (3) investigate adaptation of the normalization factor. Our experimental results show that the speaker-specific Bark/Mel scale VTLN is better than the piecewise/bilinear warping VTLN in frequency domain. It can reduce up to 12% word error rate for our Spanish and English spontaneous speech scheduling task database. For adaptation of the normalization factor, our experimental results show that promising result can be obtained by using not more than three utterances from a new speaker to estimate his/her normalization factor, and the unsupervised adaptation mode works as well as the supervised one. Therefore, the computational complexity of VTLN can be avoided by learning the normalization factor from very few utterances of a new speaker.
---
paper_title: VARIABILITY OF AUTOMATIC SPEECH RECOGNITION SYSTEMS USING DIFFERENT FEATURES

paper_content:

The paper describes the use of two recognizers fed by different acoustic features. The first recognizer performs Multiple Resolution Analysis (MRA) while the other recognizer computes JRASTA Perceptual Linear Prediction Coefficients (JRASTAPLP). The two recognizers use the same denoising method but perform different partitions of their acoustic spaces. Experiments with the Italian and Spanish components of the AURORA3 corpus show that the two systems provide, in a significant proportion of cases, substantially different posterior probabilities for the same phoneme in the same time interval. A decision rule is proposed when two different words are hypothesized by the two recognizers. It is based on the probability that a hypothesis is correct, given the identity of the word hypotheses that are in competition. Significant word error rate (WER) reductions have been found for the CH1 proportion of the Italian and Spanish components of the AURORA3 corpus.
---
paper_title: CREATING HIDDEN MARKOV MODELS FOR FAST SPEECH

paper_content:

PCT No. PCT/DE80/00104 Sec. 371 Date Mar. 5, 1981 Sec. 102(e) Date Mar. 5, 1981 PCT Filed Jul. 18, 1980 PCT Pub. No. WO81/00272 PCT Pub. Date Feb. 5, 1981.A prefabricated ceiling element for ceilings in buildings comprises a concrete ceiling panel (1, 1', 1'') and a steel girder (2, 2', 2'') having a T-section, the web of which has tips (2c, 2c') which are anchored in the concrete ceiling panel (1, 1', 1''). Free cutouts (2d) are located between the tips. The steel girder (2, 2', 2'') extends only over the midportion of the ceiling panel (1, 1', 1''), so that at the ends of the panel support areas and installation lanes (4) remain freely accessible.
---
paper_title: The OGI kids’ speech corpus and recognizers

paper_content:

We describe a corpus of children’s speech, called the OGI Kids’ Speech corpus, and a speakerand vocabulary-independent recognition system trained and evaluated with these data. The corpus is composed of both prompted and spontaneous speech from 1100 children from kindergarten through grade 10. The prompted speech was presented as text appearing below an animated character (Baldi) that produced accurate visible speech synchronized with recorded prompts. The speech and text consists of isolated words, sentences, and digit strings. A phonetic recognizer was trained using an HMM/ANN framework, with training data taken from intervals of speech associated with phonetic segments in the isolated words in the corpus. Phonetic segments were derived using automatic phonetic alignment. To find out how well the recognizer is able to generalize to new words not found in the training set, we performed two test-set evaluations: one using a new set of utterances from the set of 205 words spoken in isolation (similar to the data used to train the recognizer) and one using words from the prompted sentences. Results were dramatically different (97.5% for isolated vs. 37.9% for words in sentences), and we explore methods that may be used to improve the recognizer’s ability to generalize to new words.
---
paper_title: Speaking-rate dependent decoding and adaptation for spontaneous lecture speech recognition

paper_content:

This paper addresses the problem of speaking rate in large vocabulary spontaneous speech recognition. In spontaneous lecture speech, the speaking rate is generally fast and may vary a lot within a talk. We also observed different error tendencies for fast and slow speech segments. Therefore, we first present a speaking-rate dependent decoding strategy that applies the most adequate acoustic analysis, phone models and decoding parameters according to the speaking rate. Several methods are investigated and their selective application leads to accuracy improvement. We also propose to make use of speaking-rate information in speaker adaptation, in which the different adapted models are set up for fast and slow utterances. It is confirmed that the method is more effective than normal adaptation.
---
paper_title: Acoustic feature combination for robust speech recognition

paper_content:

In this paper, we consider the use of multiple acoustic features of the speech signal for robust speech recognition. We investigate the combination of various auditory based (mel frequency cepstrum coefficients, perceptual linear prediction, etc.) and articulatory based (voicedness) features. Features are combined by linear discriminant analysis and log-linear model combination based techniques. We describe the two feature combination techniques and compare the experimental results. Experiments performed on the large-vocabulary task VerbMobil II (German conversational speech) show that the accuracy of automatic speech recognition systems can be improved by the combination of different acoustic features.
---
paper_title: Towards improving ASR robustness for PSN and GSM telephone applications

paper_content:

Abstract In real-life applications, errors in the speech recognition system are mainly due to inefficient detection of speech segments, unreliable rejection of Out-Of-Vocabulary (OOV) words, and insufficient account of noise and transmission channel effects. In this paper, we review a set of techniques developed at CNET in order to increase the robustness to mismatches between training and testing conditions. These techniques are divided in two classes: preprocessing techniques and Hidden Markov Models (HMM) parameters adaptation. The results of several experiments carried out on field databases, as well as on databases collected over PSN and GSM networks are presented. The main sources of errors are analyzed. We show that a blind equalization scheme significantly improves the recognition accuracy regarding both field and GSM data. Speech detection allows a system to delimit the boundaries of the words to be recognized. We also use preprocessing techniques to increase the robustness of such detectors to noisy GSM speech. We show that spectral subtraction improves speech detection under noisy GSM conditions. Bayesian adaptation of HMM parameters produces models which are robust to field and GSM conditions. Models robust to GSM conditions can also be generated by linear regression adaptation of HMM parameters. Our experiments show an equivalent performance obtained with both Bayesian and linear regression adaptation of HMM parameters. The results obtained also prove that HMM adaptation and preprocessing techniques can be advantageously combined to improve Automatic Speech Recognition (ASR) robustness.
---
paper_title: Phone-context specific gender-dependent acoustic-models for continuous speech recognition

paper_content:

Gender dependent systems are usually created by splitting the training data into each gender and building two separate acoustic models for each gender. This method assumes that every state of a subphonetic model is uniformly dependent on the gender. We use the premise that the acoustic realizations of various sub phonetic units are dependent on gender in varying degrees across phones and more particularly context dependent. We show that this is indeed the case by using gender as a question in addition to phone context questions in the context decision trees. Using these trees we build phone specific gender dependent acoustic models and demonstrate a novel method to pick between genders during decoding based on a measure of confidence of the decoded hypothesis. An improvement of 6.3% in word error is achieved relative to a gender independent system.
---
paper_title: Foreign-accented speaker-independent speech recognition.

paper_content:

This research investigated whether acoustic-phonetic knowledge of the mother tongue of a non-native speaker can be used to adapt an existing target language phoneme HMM recognizer. For this purpose three sets of phoneme HMMs were generated, one representing the target language (German), one the mother tongue of the non-native speaker (Turkish), and the third the foreign-accented pronunciation of the target language (German spoken by Turkish speakers). The latter served as a benchmark for the tested adaptation methods. A derived Hidden Markov Model (HMM) clustering algorithm was applied on the target language phoneme HMM set using the mother tongue phoneme HMM set of the non-native speaker. Following the HMM adaptation a phoneme-level pronunciation technique was applied to generate phoneme mapping rules for the lexicon adaptation task. The results revealed a relative reduction of about 6% in WER for the adapted HMM. No further improvements were observed from the lexicon adaptation task.
---
paper_title: Extensions to phone-state decision-tree clustering: single tree and tagged clustering

paper_content:

The article describes two extensions to the "traditional" decision tree methods for clustering allophone HMM states in large vocabulary continuous speech recognition (LVCSR) systems. The first, single tree clustering, combines all allophone states of all phones into a single tree. This can be used to improve the performance for very small systems. The single tree clustering structure can also be exploited for speaker and channel adaptation and is shown to provide a 30% reduction in the error rate for an LVCSR task under matched channel conditions and a greater reduction under mismatched channel conditions. The second, tagged clustering, is a mechanism for providing additional information to the clustering procedure. The tags are labels for any of a wide variety of factors, such as stress, placed on the triphones. These tags are then accessible to the clustering process. Small improvements in the recognition performance were obtained under certain conditions. Both methods can be combined.
---
paper_title: Hidden mode HMM using Bayesian network for modeling speaking rate fluctuation

paper_content:

One of the most important issues in spontaneous speech recognition is how to cope with the degradation of recognition accuracy due to speaking rate fluctuation within an utterance. This paper proposes an acoustic model for adjusting mixture weights and transition probabilities of the HMM for each frame according to the local speaking rate. The proposed model is implemented along with variants and conventional models using the Bayesian network framework. The proposed model has a hidden variable representing variation of the "mode" of the speaking rate and its value controls the parameters of the underlying HMM. Model training and maximum probability assignment of the variables are conducted using the EM/GEM and inference algorithms for Bayesian networks. Utterances from meetings and lectures are used for evaluation where Bayesian network-based acoustic models are used to rescore the utterance hypotheses obtained from a first-pass N-best list. In the experiments, the proposed model shows consistently higher performance than conventional models.
---
paper_title: Speech recognition in car noise environments using multiple models according to noise masking levels.

paper_content:

In speech recognition for real-world applications, the performance degradation due to the mismatch introduced between training and testing environments should be overcome. In this paper, to reduce this mismatch, we provide a hybrid method of spectral subtraction and residual noise masking. We also employ multiple model approach to obtain improved robustness over various noise environments. In this approach, multiple model sets are made according to several noise masking levels and then a model set appropriate for the estimated noise level is selected automatically in recognition phase. According to speaker independent isolated word recognition experiments in car noise environments, the proposed method using model sets with only two masking levels reduces average word error rate by 60% in comparison with spectral subtraction method.
---
paper_title: Hidden Factor Dynamic Bayesian Networks for Speech Recognition

paper_content:

This paper presents a novel approach to modeling speech data by Dynamic Bayesian Networks. Instead of defining a specific set of factors that affect speech signals the factors are modeled implicitly by speech data clustering. Different data clusters correspond to different subsets of the factor values. These subsets are represented by the corresponding factor states. The factor states along with the phone states represent 2 hidden layers in the Hidden Factor Dynamic Bayesian Network (HFDBN). In this study we proved that Hidden Factor Dynamic Bayesian Networks provide a better speech recognition performance than HMMs of equal complexity. Speech recognition experiments were conducted on the speech data recorded in a moving car and demonstrated advantage of using HFDBN over HMM for clean and noisy speech data recognition.
---
paper_title: A speaker-stress resistant HMM isolated word recognizer

paper_content:

Most current speech recognition systems are sensitive to variations in speaker style, the following is the result of an effort to make a Hidden Markov Model (HMM) Isolated Word Recognizer (IWR) tolerant to such speech changes caused by speaker stress. More than an order-of-magnitude reduction of the error rate was achieved for a 105 word simulated stress database and a 0% error rate was achieved for the TI 20 isolated word database.
---
paper_title: Large vocabulary continuous speech recognition using HTK

paper_content:

HTK is a portable software toolkit for building speech recognition systems using continuous density hidden Markov models developed by the Cambridge University Speech Group. One particularly successful type of system uses mixture density tied-state triphones. We have used this technique for the 5 k/20 k word ARPA Wall Street Journal (WSJ) task. We have extended our approach from using word-internal gender independent modelling to use decision tree based state clustering, cross-word triphones and gender dependent models. Our current systems can be run with either bigram or trigram language models using a single pass dynamic network decoder. Systems based on these techniques were included in the November 1993 ARPA WSJ evaluation, and gave the lowest error rate reported on the 5 k word bigram, 5 k word trigram and 20 k word bigram "hub" tests and the second lowest error rate on the 20 k word trigram "hub" test. >
---
paper_title: HMM clustering for connected word recognition

paper_content:

The authors describe an HMM (hidden Markov model) clustering procedure and discuss its application to connected-word systems and to large-vocabulary recognition based on phonelike units. It is shown that the conventional approach of maximizing likelihood is easily implemented but does not work well in practice, as it tends to give improved models of tokens for which the initial model was generally quite good, but does not improve tokens which are poorly represented by the initial model. The authors have developed a splitting procedure which initializes each new cluster (statistical model) by splitting off all tokens in the training set which were poorly represented by the current set of models. This procedure is highly efficient and gives excellent recognition performance in connected-word tasks. In particular, for speaker-independent connected-digit recognition, using two HMM-clustered models, the recognition performance is as good as or better than previous results using 4-6 models/digit obtained from template-based clustering. >
---
paper_title: An Investigation of Modelling Aspects for Rate-Dependent Speech Recognition

paper_content:

For the modelling of speech rate variation in speech recognition many approaches have been suggested. However, the training of speech-rate dependent models has by far received most of the attention. In order to investigate problematic a spects related with the classification of the speech data whic h represents one of the major problems of these approaches extensive experiments were carried out on a German corpus of read speech. The results indicate that while the kind of the modeldriven speech-rate measure is only of minor importance a datadriven classification of the speech data significantly impro ves the performance of rate-dependent models. Further results suggest a detailed modelling of speech rate based on more general models. This means that it might be possible to model speech rate adaptation by means of a transformation based on a continuous measure.
---
paper_title: Speaker normalization using efficient frequency warping procedures

paper_content:

In an effort to reduce the degradation in speech recognition performance caused by variation in vocal tract shape among speakers, a frequency warping approach to speaker normalization is investigated. A set of low complexity, maximum likelihood based frequency warping procedures have been applied to speaker normalization for a telephone based connected digit recognition task. This paper presents an efficient means for estimating a linear frequency warping factor and a simple mechanism for implementing frequency warping by modifying the filter-bank in mel-frequency cepstrum feature analysis. An experimental study comparing these techniques to other well-known techniques for reducing variability is described. The results showed that frequency warping was consistently able to reduce word error rate by 20% even for very short utterances.
---
paper_title: MLLR-based accent model adaptation without accented data

paper_content:

A thermal recording apparatus variably controls the amount of energy supplied to heating elements in accordance with the relative length of a recording time interval between lines effected by linearly arranged heating elements constituted by a plurality of resistors. In another embodiment the heating elements are provided in a plurality of individually controllable blocks and the energy supplied to the heating elements in each block is variably controlled in accordance with the number of heating elements to be energized in that block.
---
paper_title: GDNN: a gender-dependent neural network for continuous speech recognition

paper_content:

Most parametric representations of speech are highly speaker dependent, and probability distributions suitable for a certain speaker may not perform as well for other speakers. It is desirable to incorporate constraints on analysis that rely on the same speaker producing all the frames in an utterance. Experiments for speaker consistency modeling by using a classification network to help generate gender-dependent phonetic probabilities for a statistical recognition system are reported. Results show a good classification rate for the gender classification net. Simple use of such a model to augment an existing larger network that estimates phonetic probabilities does not help speech recognition performance. When the net is properly integrated in a hidden Markov model (HMM) recognizer, it significantly improves word accuracy. >
---
paper_title: A fast method of speaker normalisation using formant estimation.

paper_content:

It has recently been shown that normalisation of vocal tract length can significantly increase recognition accuracy in speaker independent automatic speech recognition systems. An inherent difficulty with this technique is in automatically estimating the normalisation parameter from a new speaker’s speech and previous techniques have typically relied on an exhaustive search to estimate this parameter. In this paper, we present a method of normalising utterances by a linear warping of mel filter bank channels in which the normalisation parameter is estimated by fitting formant estimates to a probabilistic model. This method is fast, computationally inexpensive and requires only a limited amount of data for estimation. It generates normalisations which are close to those which would be found by an exhaustive search. The normalisation is applied to a phoneme recognition task using the TIMIT database and results show a useful improvement over an un-normalised speaker independent system.
---
paper_title: A Post-Processing System To Yield Reduced Word Error Rates: Recognizer Output Voting Error Reduction (ROVER)

paper_content:

Describes a system developed at NIST to produce a composite automatic speech recognition (ASR) system output when the outputs of multiple ASR systems are available, and for which, in many cases, the composite ASR output has a lower error rate than any of the individual systems. The system implements a "voting" or rescoring process to reconcile differences in ASR system outputs. We refer to this system as the NIST Recognizer Output Voting Error Reduction (ROVER) system. As additional knowledge sources are added to an ASR system (e.g. acoustic and language models), error rates are typically decreased. This paper describes a post-recognition process which models the output generated by multiple ASR systems as independent knowledge sources that can be combined and used to generate an output with reduced error rate. To accomplish this, the outputs of multiple of ASR systems are combined into a single, minimal-cost word transition network (WTN) via iterative applications of dynamic programming (DP) alignments. The resulting network is searched by an automatic rescoring or "voting" process that selects the output sequence with the lowest score.
---
paper_title: Speaker adaptive modeling by vocal tract normalization

paper_content:

This paper presents methods for speaker adaptive modeling using vocal tract normalization (VTN) along with experimental tests on three databases. We propose a new training method for VTN: By using single-density acoustic models per HMM state for selecting the scale factor of the frequency axis, we avoid the problem that a mixture-density tends to learn the scale factors of the training speakers and thus cannot be used for selecting the scale factor. We show that using single Gaussian densities for selecting the scale factor in training results in lower error rates than using mixture densities. For the recognition phase, we propose an improvement of the well-known two-pass strategy: by using a non-normalized acoustic model for the first recognition pass instead of a normalized model, lower error rates are obtained. In recognition tests, this method is compared with a fast variant of VTN. The two-pass strategy is an efficient method, but it is suboptimal because the scale factor and the word sequence are determined sequentially. We found that for telephone digit string recognition this suboptimality reduces the VTN gain in recognition performance by 30% relative. In summary, on the German spontaneous speech task Verbmobil, the WSJ task and the German telephone digit string corpus SieTill, the proposed methods for VTN reduce the error rates significantly.
---
paper_title: Generalized mixture of HMMs for continuous speech recognition

paper_content:

This paper presents a new technique for modeling heterogeneous data sources such as speech signals received via distinctly different channels. Such a scenario arises when an automatic speech recognition system is deployed in wireless telephony in which highly heterogeneous channels coexist and interoperate. The problem is that a simple model may become inadequate to describe accurately the diversity of the signal, resulting in an unsatisfactory recognition performance. To deal with such a problem, we propose a generalized mixture model (GMM) approach. For speech signals, in particular, we use mixtures of hidden Markov models (i.e., GMHMM, generalized mixture of HMMs). By applying discriminative training for GMHMM we obtained 1.0% word error rate for the recognition of the digits strings from the wireless database, comparing to 1.4% word error rate for the conventional HMM based discriminative technique.
---
paper_title: High-accuracy connected digit recognition for mobile applications

paper_content:

We present a connected digit recognition system with low storage and computational complexity which achieves good performance in car noise. Our system uses the TI-DIGITS database with additive car noise for training whole-word digit and background models. A digit accuracy of 96.1% is obtained on a 15-speaker database collected in a car using an open microphone with an average SNR of approximately 2 dB. There is a further error reduction of almost 35% if the top two candidate strings are considered using a traceback based N-best algorithm. The system can be implemented on a currently available fixed-point DSP chip. We show that significant performance improvements are obtained by using two-level cepstral mean subtraction (CMS), gender-dependent models and a decoding grammar constraining the possible lengths of digit strings.
---
paper_title: A study of speech recognition for children and the elderly

paper_content:

Although children and the elderly have obvious needs for voice operated interfaces, hardly anything is known about the performance of the current automatic speech recognition technology with these people. In this paper we report the results of a thorough investigation into this field using a connected digit recognizer and a major telephone speech database. One would generally assume that the recognition of speech from these people would only be a matter of having enough, sufficiently representative training data. This turns out to be true only, as long as the speakers belong to the age range 15 to approximately 70. Outside this range the error rates increase dramatically, even with balanced amounts of training data. For males, the lower limit is very sharp and can be attributed to the change of pitch frequency during puberty. For females, the lower limit is gradual and caused by the slowly changing dimensions of the vocal tract length only. For both genders, the upper limit is very gradual and can possibly be attributed to changes in the glottis area and the internal control loops of the human articulatory system. The paper presents some supporting evidence for the above assertions and gives results for various attempts to improve the performance. Recognition of children and the elderly will require much more research if we are to fully understand the characteristics of these age group on current and future speech recognition systems.
---
paper_title: Fast Model Selection Based Speaker Adaptation for Nonnative Speech

paper_content:

The problem of adapting acoustic models of native English speech to nonnative speakers is addressed from a perspective of adaptive model complexity selection. The goal is to select model complexity dynamically for each nonnative talker so as to optimize the balance between model robustness to pronunciation variations and model detailedness for discrimination of speech sounds. A maximum expected likelihood (MEL) based technique is proposed to enable reliable complexity selection when adaptation data are sparse, where expectation of log-likelihood (EL) of adaptation data is computed based on distributions of mismatch biases between model and data, and model complexity is selected to maximize EL. The MEL based complexity selection is further combined with MLLR (maximum likelihood linear regression) to enable adaptation of both complexity and parameters of acoustic models. Experiments were performed on WSJ1 data of speakers with a wide range of foreign accents. Results show that the MEL based complexity selection is feasible when using as little as one adaptation utterance, and it is able to select dynamically the proper model complexity as the adaptation data increases. Compared with the standard MLLR, the MEL+MLLR method leads to consistent and significant improvement to recognition accuracy on nonnative speakers, without performance degradation on native speakers.
---
paper_title: Towards robustness to fast speech in ASR

paper_content:

Psychoacoustic studies show that human listeners are sensitive to speaking rate variations. Automatic speech recognition (ASR) systems are even more affected by the changes in rate, as double to quadruple word recognition error rates of average speakers have been observed for fast speakers on many ASR systems. In our earlier work (see Proceedings of EUROSPEECH95, p.491-4, 1995), we studied the causes of higher error and concluded that both the acoustic-phonetic and the phonological differences are sources of higher word error rates. In this work, we have studied various measures for quantifying rate of speech (ROS) and used simple methods for estimating the speaking rate of a novel utterance using ASR technology. We have also implemented mechanisms that make our ASR system more robust to fast speech. Using our ROS estimator to identify fast sentences in the test set, our rate-dependent system has 24.5% fewer errors on the fastest sentences and 6.2% fewer errors on all sentences of the WSJ93 evaluation set relative to the baseline HMM/MLP system.
---
paper_title: Speech recognition using voice-characteristic-dependent acoustic models

paper_content:

This paper proposes a speech recognition technique based on acoustic models considering voice characteristic variations. Context-dependent acoustic models, which are typically triphone HMM, are often used in continuous speech recognition systems. This work hypothesizes that the speaker voice characteristics that humans can perceive by listening are also factors in acoustic variation for construction of acoustic models, and a tree-based clustering technique is also applied to speaker voice characteristics to construct voice-characteristic-dependent acoustic models. In speech recognition using triphone models, the neighboring phonetic context is given from the linguistic-phonetic knowledge in advance; in contrast, the voice characteristics of input speech are unknown in recognition using voice-characteristic-dependent acoustic models. This paper proposes a method of recognizing speech even under conditions where the voice characteristics of the input speech are unknown. The result of a gender-dependent speech recognition experiment shows that the proposed method achieves higher recognition performance in comparison to conventional methods.
---
paper_title: A Confidence Measure Based On Agreement Among Multiple Lvcsr Models - Correlation Between Pair Of Acoustic Models And Confidence

paper_content:

{utsuro,tetsu}@cl.ics.tut.ac.jp, {nisizaki,nakagawa}@slp.ics.tut.ac.jpABSTRACTFor many practicalapplications ofspeech recognition systems, it isquite desirable to have an estimate of conﬁdence for each hypoth-esized word. Unlike previous works on conﬁdence measures, thispaper studies features for conﬁdence measures that are extractedfrom outputs of more than one LVCSR models. More speciﬁcally,this paper experimentally evaluates the agreement among the out-puts of multiple Japanese LVCSR models, with respect to whetherit is effective as an estimate of conﬁdence for each hypothesizedword. The results of experimental evaluation show that the agree-ment between the outputs with two LVCSR models with differ-ent decoders and acoustic models can achieve quite reliable con-ﬁdence. Furthermore, among various features of acoustic modelsbased on Gaussian mixture HMMs, it is concluded that ones suchas whether or not to have short pause models, as well as differentunits in HMMs (e.g., triphone model or syllable model) are themost effective in achieving highly reliable conﬁdence.1. INTRODUCTIONSince current speech recognizers’ outputs are far from perfect andalways include a certain amount of recognition errors, it is quitedesirable to have an estimate of conﬁdence for each hypothesizedword. This is especially true for many practical applications ofspeech recognition systems such as word selection for unsuper-vised adaptation schemes, automatic weighting of additional, non-speech knowledge sources, keyword based speech understanding,and recognition error rejection – conﬁrmation in spoken dialoguesystems.Most of previous works on conﬁdence measures (e.g., [1, 2])are based on features available in a single LVCSR model. How-ever, it is well known that a voting scheme such as ROVER (Rec-ognizeroutputvoting error reduction) forcombining multiple speechrecognizers’ outputs can achieve word error reduction [3, 4]. Con-sidering the success of a simple voting scheme such as ROVER, italso seems quite possible to improve reliability of previously stud-ied features for conﬁdence measures by simply exploiting morethan one speech recognizers’ outputs. From this observation, un-like those previous works on conﬁdence measures, this paper stud-ies features for conﬁdence measures that are extracted from out-puts of more than one LVCSR models.For the purpose of estimating conﬁdence for each hypothe-sized word, it is more important to examine which combination ofexisting LVCSR models can achieve high conﬁdence and whichcombination can not, although even simple voting schemes canachieve word error reduction. Therefore, in this paper, we exper-imentally evaluate the agreement among the outputs of multipleJapanese LVCSR models, with respect to whether it is effectiveas an estimate of conﬁdence for each hypothesized word. In thisevaluation of existing Japanese LVCSR models, we concentrate onevaluating conﬁdence of the agreement among outputs with dif-ferent decoders and/or different acoustic models. The results ofexperimental evaluation show that the agreement between the out-puts with two LVCSR models with different decoders and acousticmodels can achieve quite reliable conﬁdence. Furthermore,amongvarious features of acoustic models based on Gaussian mixtureHMMs, it is concluded that ones such as whether or not to haveshort pause models, as well as different units in HMMs (e.g., tri-phone model or syllable model) are the most effective in achievinghighly reliable conﬁdence. It is also shown that it is better to com-bine various features including those most effective ones than touse one of those most effective features alone.2. SPECIFICATION OF JAPANESE LVCSR SYSTEMS2.1. DecodersAs the decoders of Japanese LVCSR systems, we use the onenamed Julius, which is provided by IPA Japanese dictation freesoftware project [5], as well as the one named SPOJUS [6], whichhas been developed in our laboratory. Both decoders are com-posed of two decoding passes, where the ﬁrst pass uses the wordbigram, and the second pass uses the word trigram. Julius is withword-trellis searches and hence has much broader search spacethan SPOJUS, which is with N-best searches.2.2. Acoustic ModelsThe acoustic models of Japanese LVCSR systems are based onGaussian mixture HMM. We evaluate phoneme-based HMMs aswell as syllable-based HMMs.2.2.1. Acoustic Models with the Decoder J
---
paper_title: Spontaneous Speech Recognition Using a Massively Parallel Decoder

paper_content:

Since spontaneous utterances include many variations, speakerand task-independent general models do not work well. This paper proposes combining cluster-based language and acoustic models based on the framework of Massively Parallel Decoder (MPD). The MPD is a parallel decoder that has a large number of decoding units, in which each unit is assigned to each combination of element models. It runs efficiently on a parallel computer, and thus the turnaround time is comparable to conventional decoders using a single model and a processor. In the experiments conducted using lecture speeches from the Corpus of Spontaneous Japanese, two types of cluster models have been investigated: lecture-based cluster models and utterancebased cluster models. It has been confirmed that utterancebased cluster models give significantly lower recognition error rate than lecture-based cluster models in both language and acoustic modeling. It has also been shown that roughly 100 decoding units are enough in terms of recognition rate, and in the best setting, 12% reduction in word error rate was obtained in comparison with the conventional decoder.
---
paper_title: A family of parallel hidden Markov models

paper_content:

Stochastic signal models represent a powerful tool for automatic speech recognition. A particular type of stochastic modeling based on first-order hidden Markov models (HMMs), has been increasingly popular, because it has a solid theoretical basis and offers practical advantages. The authors extend the standard HMM theory to parallel hidden Markov models (PHMMs). The parallel model consists of two statistically related HMMs. This configuration has mixture densities of HMM observations whose weights can be made variable depending on the probability of other HMMs being in certain states. This allows one to dynamically adapt observation statistics to acoustic contexts. Some preliminary experiments have been carried out in order to compare the PHMMs with standard HMMs and the results are presented. >
---
paper_title: Effective Acoustic Modeling for Rate-of-Speech Variation in Large Vocabulary Conversational Speech Recognition

paper_content:

We investigate several variants of speech-rate-dependent acoustic models for large-vocabulary conversational speech recognition, in the framework of combining rate-specific models in decoding to compensate for speech rate variation. We study two basic approaches to combining rate-specific models: one combines models at the pronunciation level and the other at the HMM state level. Furthermore, we investigate the influence of different numbers of rate-of-speech classes and different parameter tying schemes. Experiments on the Switchboard database, using SRI’s DECIPHER recognition system, show that rate-dependent acoustic modeling resulted in a 2% relative word error rate reduction over a rateindependent baseline, and that the pronunciation-level constraint, Gaussian sharing between rate-specific models, and a well-chosen number of rate-of-speech classes are all important for best performance.
---
paper_title: Recognition Of Non-Native German Speech With Multilingual Recognizers

paper_content:

In this study we present di erent approaches to the recognition of non-natives. With a corpus in German spoken by speakers with 56 di erent rst languages, the Strange Corpus, we perform recognition experiments with monolingual and multilingual recognizers. Among other, we compared two German recognizers, one that was trained in addition with non-native (Italian) speech and the other trained with German speakers only. We found that best performance is achieved with the recognizer trained with German including non-native speech, followed by a bilingual recognizer and an Italian recognizer trained with German and Italian natives.
---
paper_title: Towards a robust/fast continuous speech recognition system using a voiced-unvoiced decision

paper_content:

We show that the concept of voiced-unvoiced (VU) classification of speech sounds can be incorporated not only in speech analysis or speech enhancement processes, but also can be useful for recognition processes. That is, the incorporation of such a classification in a continuous speech recognition (CSR) system not only improves its performance in low SNR environments, but also limits the time and the necessary memory to carry out the process of the recognition. The proposed V-U classification of the speech sounds has two principal functions: (1) it allows the enhancement of the voiced and unvoiced parts of speech separately; (2) it limits the Viterbi (1967) search space, and consequently the process of recognition can be carried out in real time without degrading the performance of the system. We prove via experiments that such a system outperforms the baseline HTK when a V-U decision is included in both front- and far-end of the HTK-based recognizer.
---
paper_title: Hidden Markov model for Mandarin lexical tone recognition

paper_content:

A case of lexical tone recognition for Mandarin speech is discussed using a combination of vector quantization and hidden Markov modelling techniques. The observation sequence was a sequence of vectorized parameters consisting of a logarithmic pitch interval and its first derivative. The vector quantization was applied to convert the observation sequence into a symbol sequence for Hidden Markov modeling. The speech database was provided by seven male and seven female college students, with each pronouncing 72 isolated monosyllabic utterances. A probabilistic model for each of the four tones was generated. A series of tonal recognition tests were then conducted to evaluate the effects of pitch reference base, codebook size, and tonal model topology. Future consideration of Mandarin speech recognition is also discussed. >
---
paper_title: Discriminative HMM stream model for Mandarin digit string speech recognition

paper_content:

The conventional hidden Markov model (HMM) only based on the spectral features does not have a high recognition performance for connected Mandarin digits, because highly confusable syllables exist. The main problems of Mandarin digit recognition are analyzed. It is revealed that to establish the precise classification models for Mandarin digits not only features extracted from the spectrum, energy and pitch contour are necessary but also they should be used with different emphases for different digits. So each-type of feature is used to train a single-stream HMM by maximum likelihood. Then a multi-stream HMM is obtained by combining the single-stream HMMs with exponents that weigh the log-likelihood of each stream. The exponents are estimated by means of the generalized probabilistic descent algorithm according to the digit minimum classification error rate criteria. The superiority of the multi-stream HMM is demonstrated: the relative string error rate is reduced by 54.5%. And the unknown length digit string error rate and its digit error rate decrease to 4.66% and 1.31% respectively.
---
paper_title: Voicing parameter and energy based speech/non-speech detection for speech recognition in adverse conditions.

paper_content:

In adverse conditions, the speech recognition performances decrease in part due to imperfect speech/non-speech detection. In this paper, a new combination of voicing parameter and energy for speech/non-speech detection is described. This combination avoids especially the noise detections in real life very noisy environments and provides better performances for continuous speech recognition. This new speech/non-speech detection approach outperforms both noise statistical based [1] and Linear Discriminate Analysis (LDA) based [2] criteria in noisy environments and for continuous speech recognition applications.
---
paper_title: Using formant frequencies in speech recognition.

paper_content:

Formant frequencies have rarely been used as acoustic features for speech recognition, in spite of their phonetic significance. For some speech sounds one or more of the formants may be so badly defined that it is not useful to attempt a frequency measurement. Also, it is often difficult to decide which formant labels to attach to particular spectral peaks. This paper describes a new method of formant analysis which includes techniques to overcome both of the above difficulties. Using the same data and HMM model structure, results are compared between a recognizer using conventional cepstrum features and one using three formant frequencies, combined with fewer cepstrum features to represent general spectral trends. For the same total number of features, results show that including formant features can offer increased accuracy over using cepstrum features only.
---
paper_title: Hybrid HMM/BN LVCSR system integrating multiple acoustic features

paper_content:

In current HMM based speech recognition systems, it is difficult to supplement acoustic spectrum features with additional information such as pitch, gender, articulator positions, etc. On the other hand, dynamic Bayesian networks (DBN) allow for easy combination of different features and make use of conditional dependencies between them. However, lack of efficient algorithms has prevented their application in large vocabulary continuous speech recognition. The hybrid HMM/BN acoustic model, where HMM are used for modeling of temporal speech characteristics and state probability model is represented by BN, provides a trade off solution to the problem. In this paper we describe the HMM/BN acoustic model and LVCSR system built upon this model. In the HMM/BN model, in addition to speech observation variable, state BN has two more discrete variables representing speaker gender and pitch frequency. Evaluation results on WSJ database showed lower word error rate with respect to the same complexity conventional HMM acoustic model when there is enough training data to estimate reliable HMM/BN parameters.
---
paper_title: Speaker independent speech recognition using features based on glottal sound source.

paper_content:

The invention features an implantable device for the automatic delivery of an active ingredient according to an adjustable delivery profile. The device includes a housing; a reservoir operatively connected to the housing and arranged to store a solid composition including the active ingredient; an actuator arranged within the housing to move the solid composition from the reservoir to a transit area, wherein the solid composition exits the housing at the transit area; a controller that acts on the actuator to adjust movement of the solid composition out of the housing according to the adjustable delivery profile; and a power source arranged to provide energy to the actuator and the controller. The solid composition can be an elongate, solid composition comprising a drug, and up to 90% of a carrier, wherein the composition has a cross-section of less than 0.5 mm, and wherein the drug and the carrier are selected and compounded in a proportion such that the drug is immediately released from the carrier upon contact with a liquid.
---
paper_title: Pitch tracking and tone features for Mandarin speech recognition

paper_content:

Tone modeling is a critical component for Mandarin large-vocabulary continuous-speech recognition systems. This paper presents an efficient real-time pitch tracker and a set of tone features that achieve a vast 30% reduction of the character error rate (CER), compared to the non-tonal baseline. To our knowledge, this is the highest improvement from tones ever reported for Mandarin. The paper first discusses adapting a known pitch-tracking algorithm for real-time operation. Second, we study the derivation of tone features for Mandarin LVCSR. Compared to the baseline vector (F/sub 0/, /spl Delta/F/sub 0/), our best tone features lead to a 28% reduction of tone errors. Results are shown for three LVCSR databases, including the Chinese 1998 National Performance Assessment (Project 863) and the Taiwan telephony database "MAT." Performance of Western-language systems is reached, and for the "863 System Performance Test," our system achieves 1.5% CER.
---
paper_title: Speech recognition with auxiliary information

paper_content:

State-of-the-art automatic speech recognition (ASR) systems are usually based on hidden Markov models (HMMs) that emit cepstral-based features which are assumed to be piecewise stationary. While not really robust to noise, these features are also known to be very sensitive to "auxiliary" information, such as pitch, energy, rate-of-speech (ROS), etc. Attempts so far to include such auxiliary information in state-of-the-art ASR systems have often been based on simply appending these auxiliary features to the standard acoustic feature vectors. In the present paper, we investigate different approaches to incorporating this auxiliary information using dynamic Bayesian networks (DBNs) or hybrid HMM/ANNs (HMMs with artificial neural networks). These approaches are motivated by the fact that the auxiliary information is not necessarily (directly) emitted by the HMM states but, rather, carries higher-level information (e.g., speaker characteristics) that is correlated with the standard features. As implicitly done for gender modeling elsewhere, this auxiliary information then appears as a conditional variable in the emission distributions and can be hidden (except in the case of some HMM/ANNs) as its estimates become too noisy. Based on recognition experiments carried out on the OGI Numbers database (free format numbers spoken over the telephone), we show that auxiliary information that conditions the distribution of the standard features can, in certain conditions, provide more robust recognition than using auxiliary information that is appended to the standard features; this is most evident in the case of energy as an auxiliary variable in noisy speech.
---
paper_title: Evaluation of formant-like features on an automatic vowel classification task

paper_content:

Numerous attempts have been made to find low-dimensional, formant-related representations of speech signals that are suitable for automatic speech recognition. However, it is often not known how these features behave in comparison with true formants. The purpose of this study was to compare two sets of automatically extracted formant-like features, i.e., robust formants and HMM2 features, to hand-labeled formants. The robust formant features were derived by means of the split Levinson algorithm while the HMM2 features correspond to the frequency segmentation of speech signals obtained by two-dimensional hidden Markov models. Mel-frequency cepstral coefficients (MFCCs) were also included in the investigation as an example of state-of-the-art automatic speech recognition features. The feature sets were compared in terms of their performance on a vowel classification task. The speech data and hand-labeled formants that were used in this study are a subset of the American English vowels database presented in ...
---
paper_title: Recognition of syllables in a tone language

paper_content:

Speech recognition of tone languages requires detection of the tone in addition to detection of the consonants and vowels of a syllable. This paper compares three different methods based on the hidden Markov model (HMM) framework for recognition of tonal syllables in continuous speech. In joint detection, recognition is done by employing a HMM of connected tonal syllables, in which the pitch and its time derivative are included into the feature vector in addition to the phonetic features. In sequential detection, base syllables (syllables ignoring their tones) are recognized by using a HMM of connected base syllables only; the estimated syllable boundaries are then used for subsequent tone recognition in a separate HMM of tones. In linked detection, the recognition in the HMM of connected base syllables is modified to periodically take into account also tonal likelihood computed from a HMM of tones. Linked detection can provide performance that is comparable to the performance of joint detection, which is clearly superior to that of sequential detection. The computational complexity of linked detection is lower than that of joint detection for a large vocabulary task, where the number of states in the HMM of connected tonal syllables is substantially larger than that in the HMM of connected base syllables.
---
paper_title: Decision tree based tone modeling for Chinese speech recognition

paper_content:

Because of the tonal nature of Chinese languages, correct recognition of lexical tones is necessary for Chinese speech recognition. In order to incorporate tone information into Chinese speech recognition, three issues need to be addressed: (i) the representation of the syllable pitch contour as well as the tone contour; (ii) the lexical tone probability estimation; and (iii) the integration of tone probabilities into the Viterbi recognition process. In this paper we propose a robust polynomial segmental representation of the pitch contour coupled with a decision tree based tone classifier. We also propose a novel approach of integrating the decision tree tone classifier directly into a single pass recognition process. The proposed approaches were evaluated on tasks of tone classification and tonal-syllable recognition. In regard to tone classification, the robust decision tree gave a tone classification-accuracy of 89% for isolated syllables and 71.2% for the continuous speech. Moreover, by incorporating the decision tree tone classifier into the Viterbi search, the tonal-syllable recognition error rate in continuous speech was reduced by 13.54%.
---
paper_title: Pitch dependent phone modelling for HMM-based speech recognition

paper_content:

This paper proposes a novel method of incorporating pitch information into an HMM speech recognition system by exploiting the correlation between pitch and spectral parameters, e.g. cepstrum. Pitch patterns are not used explicitly; instead, spectral parameters are normalized framewise according to the pitch value. Evidence is given to show that the use of pitch information consistently improves the recognition performance. Experiments with 24 phoneme labels showed that the phoneme error rate for fast continuous speech could be improved by about 10%. Using these pitchnormalized phone models in an HMM-LR speech recognition system improved the phrase recognition accuracy for the top 5 candidates from 96% to 97.5%, i.e. the error rate was nearly halved.
---
paper_title: Modelling Auxiliary Features in Tandem Systems

paper_content:

Tandem systems transform the cepstral features into posterior probabilities of subword units using artificial neural networks (ANNs), which are processed to form input features for conventional speech recognition systems. They have been shown to perform better than conventional speech recognition systems using cepstral features. Recent studies have shown that modelling cepstral features with auxiliary sources of knowledge leads to improvement in the performance of speech recognition systems. In this paper, we study two approaches to incorporate auxiliary knowledge sources such as pitch frequency, short-term energy, etc. (referred to as auxiliary features), in a tandem-based automatic speech recognition system. In the first approach, we model the auxiliary features in the process of training an ANN, which is later used to extract tandem-features. In the second approach, we extract the tandem-features from an ANN trained with cepstral features only and then model them jointly with auxiliary features. Recognition studies conducted on a connected word recognition task under clean and noisy conditions show that the performance of the tandem system can be improved by incorporating auxiliary features.
---
paper_title: On the robust incorporation of formant features into hidden Markov models for automatic speech recognition

paper_content:

A formant analyser is interpreted probabilistically via a noisy channel model. This leads to a robust method of incorporating formant features into hidden Markov models for automatic speech recognition. Recognition equations follow trivially, and Baum-Welch style re-estimation equations are derived. Experimental results are presented which provide empirical proof of convergence, and demonstrate the effectiveness of the technique in achieving recognition performance advantages by including formant features rather than only using cepstrum features.
---
paper_title: Use of voicing features in HMM-based speech recognition

paper_content:

Abstract We investigate speech recognition features related to voicing functions that indicate whether the vocal folds are vibrating. We describe two voicing features, periodicity and jitter, and demonstrate that they are powerful voicing discriminators. The periodicity and jitter features and their first and second time derivatives are appended to a standard 38-dimensional feature vector comprising the first and second time derivatives of the frame energy and the cepstral coefficients with their first and second time derivatives. HMM-based connected-digit (CD) and large-vocabulary (LV) recognition experiments comparing the traditional and extended feature sets show that voicing features and spectral information are complementary and that improved speech recognition performance is obtained by combining the two sources of information. We further conclude that the difference in performance with and without voicing becomes more significant when minimum string error (MSE) training is used than when maximum likelihood (ML) training is used.
---
paper_title: A pattern recognition approach to voiced-unvoiced-silence classification with applications to speech recognition

paper_content:

In speech analysis, the voiced-unvoiced decision is usually performed in conjunction with pitch analysis. The linking of voiced-unvoiced (V-UV) decision to pitch analysis not only results in unnecessary complexity, but makes it difficult to classify short speech segments which are less than a few pitch periods in duration. In this paper, we describe a pattern recognition approach for deciding whether a given segment of a speech signal should be classified as voiced speech, unvoiced speech, or silence, based on measurements made on the signal. In this method, five different measurements are made on the speech segment to be classified. The measured parameters are the zero-crossing rate, the speech energy, the correlation between adjacent speech samples, the first predictor coefficient from a 12-pole linear predictive coding (LPC) analysis, and the energy in the prediction error. The speech segment is assigned to a particular class based on a minimum-distance rule obtained under the assumption that the measured parameters are distributed according to the multidimensional Gaussian probability density function. The means and covariances for the Gaussian distribution are determined from manually classified speech data included in a training set. The method has been found to provide reliable classification with speech segments as short as 10 ms and has been used for both speech analysis-synthesis and recognition applications. A simple nonlinear smoothing algorithm is described to provide a smooth 3-level contour of an utterance for use in speech recognition applications. Quantitative results and several examples illustrating the performance of the method are included in the paper.
---
paper_title: Automated lexical adaptation and speaker clustering based on pronunciation habits for non-native speech recognition

paper_content:

This paper describes a method to improve speech recognition for non-native speech in a spoken dialogue system. Based on very general rules about possible vocalic substitutions, the frequency of occurrence of each substitution in different phonetic contexts is estimated on a small set of recordings. The most frequently observed substitutions are applied to the lexicon of the recognizer. Speakers in the training set are automatically clustered according to their preferred phonetic variants, and a specific lexicon is built for each cluster. Acoustic adaptation is also performed on each cluster. Experiments show that lexical adaptation provides a relative WER reduction over acoustic adaptation alone. Lexical clustering can further reduce WER if the system can reliably select the cluster best matching each input utterance.
---
paper_title: Hidden mode HMM using Bayesian network for modeling speaking rate fluctuation

paper_content:

One of the most important issues in spontaneous speech recognition is how to cope with the degradation of recognition accuracy due to speaking rate fluctuation within an utterance. This paper proposes an acoustic model for adjusting mixture weights and transition probabilities of the HMM for each frame according to the local speaking rate. The proposed model is implemented along with variants and conventional models using the Bayesian network framework. The proposed model has a hidden variable representing variation of the "mode" of the speaking rate and its value controls the parameters of the underlying HMM. Model training and maximum probability assignment of the variables are conducted using the EM/GEM and inference algorithms for Bayesian networks. Utterances from meetings and lectures are used for evaluation where Bayesian network-based acoustic models are used to rescore the utterance hypotheses obtained from a first-pass N-best list. In the experiments, the proposed model shows consistently higher performance than conventional models.
---
paper_title: Alternate phone models for conversational speech

paper_content:

This paper investigates the use of alternate phone models for the transcription of conversational telephone speech. The focus of this work is to explore alternative ways of modeling different manners of speaking so as to better cover the observed articulatory styles and pronunciation variants. Four alternate phone sets are compared ranging from 38 to 129 units. Two of the phone sets make use of syllable-position dependent phone models. The acoustic models were trained on 2300 hours of conversational telephone speech data from the Switchboard and Fisher corpora, and experimental results are reported on the EARS Dev04 test set which contains 3 hours of speech from 36 Fisher conversations. While no one particular phone set was found to outperform the others for a majority of speakers, the best overall performance was obtained with the original 48 phone set and a reduced 38 phone set, however combining the hypotheses of the individual models reduces the word error rate from 17.5% (original phone set) to 16.8%.
---
paper_title: Language independent and language adaptive large vocabulary speech recognition

paper_content:

This paper describes the design of a multilingual speech recognizer using an LVCSR dictation database which has been collected under the project GlobalPhone. This project at the University of Karlsruhe investigates LVCSR systems in 15 languages of the world, namely Arabic, Chinese, Croatian, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish, Swedish, Tamil, and Turkish. Based on a global phoneme set we built different multilingual speech recognition systems for five of the 15 languages. Context dependent phoneme models are created data-driven by introducing questions about language and language groups to our polyphone clustering procedure. We apply the resulting multilingual models to unseen languages and present several recognition results in language independent and language adaptive setups.
---
paper_title: Off-line acoustic modelling of non-native accents.

paper_content:

This paper introduces a family of three techniques to improve non-native speaker-independent recognition if the type of accent, i.e. the mother tongue of a non-native speaker is known. These techniques permit the computation of non-native models without requiring adaptation data, that is, they can be computed o -line. The improved recognition performance of these approaches is obtained by combining speaker-independent hidden Markov models of the target language, i.e. the language to be taught, and of the source language, i.e. the speaker's native language. All three combination techniques require a mapping between the two languages to de ne which source model state combines with each target model state. A method has been developed to derive such a mapping automatically. Recognition results are given for all three techniques applied to the two cases of Spanish accented British English and Japanese accented British English. The average baseline word error rate of 28:3% can be decreased to 22:9% for the rst method, to 24:1% for the second and to 20:6% for the last method, which equals a relative improvement of 29% without adaptation.
---
paper_title: Generating Non-Native Pronunciation Variants for Lexicon Adaptation

paper_content:

Abstract Handling non-native speech in automatic speech recognition (ASR) systems is an area of increasing interest. The majority of systems are tailored to native speech only and as a consequence performance for non-native speakers often is not satisfactory. One way to approach the problem is to adapt the acoustic models to the new speaker. Another important means to improve performance for non-native speakers is to consider non-native pronunciations in the dictionary. The difficulty here lies in the generation of the non-native variants, especially if various accents are to be considered. Traditional approaches to model pronunciation variation either require phonetic expertise or extensive speech databases. They are too costly, especially if a flexible modelling of several accents is desired. We propose to exclusively use native speech databases to derive non-native pronunciation variants. We use an English phoneme recogniser to generate English pronunciations for German words and use these to train decision trees that are able to predict the respective English-accented variant from the German canonical transcription. Furthermore we combine this approach with online, incremental weighted MLLR speaker adaptation. Using the enhanced dictionary and the speaker adaptation alone improved the word error rate of the baseline system by 5.2% and 16.8%, respectively. When both methods were combined, we achieved an improvement of 18.2%.
---
paper_title: Statistical Modeling of Phonological Rules through Linguistic Hierarchies

paper_content:

Abstract This paper describes our research aimed at acquiring a generalized probability model for alternative phonetic realizations in conversational speech. For all of our experiments, we utilize the summit landmark-based speech recognition framework. The approach begins with a set of formal context-dependent phonological rules, applied to the baseforms in the recognizer’s lexicon. A large speech corpus is phonetically aligned using a forced recognition procedure. The probability model is acquired by observing specific realizations expressed in these alignments. A set of context-free rules is used to parse words into substructure, in order to generalize context-dependent probabilities to other words that share the same sub-word context. The model maps phones to sub-word units probabilistically in a finite state transducer framework, capturing phonetic predictions based on local phonemic, morphologic, and syllabic contexts. We experimented within two domains: the mercury flight reservation domain and the jupiter weather domain. The baseline system used the same set of phonological rules for lexical expansion, but with no probabilities for the alternates. We achieved 14.4% relative reduction in concept error rate for jupiter and 16.5% for mercury .
---
paper_title: Implicit modelling of pronunciation variation in automatic speech recognition

paper_content:

Modelling of pronunciation variability is an important task for the acoustic model of an automatic speech recognition system. Good pronunciation models contribute to the robustness and generic applicability of a speech recogniser. Usually pronunciation modelling is associated with a lexicon that allows to explicitly control the selection of appropriate HMMs for a particular word. However, the use of data-driven clustering techniques or specific parameter tying techniques has considerable impact on this form of model selection and the construction of a task-optimal dictionary. Most large vocabulary speech recognition systems make use of a dictionary with multiple possible pronunciation variants per word. By manual addition of pronunciation variants explicit human knowledge is used in the recognition process. For reasons of complexity the optimisation of manual entries for performance is often not feasible. In this paper a method for the stepwise reduction of the number of pronunciation variants per word to one is described. By doing so in a way consistent with the classification procedure, pronunciation variation is modelled implicitly. It is shown that the use of single pronunciation dictionaries provides similar or better word error rate performance, achieved both on Wall Street Journal and Switchboard data. The use of single pronunciation dictionaries in conjunction with Hidden Model Sequence Models as an example of an implicit pronunciation modelling technique shows further improvements.
---
paper_title: Lexical modeling of non-native speech for automatic speech recognition

paper_content:

The paper examines the recognition of non-native speech in JUPITER, a speaker-independent, spontaneous-speech conversational system. Because the non-native speech in this domain is limited and varied, speaker- and accent-specific methods are impractical. We therefore chose to model all of the non-native data with a single model. In particular, the paper describes an attempt to better model non-native lexical patterns. These patterns are incorporated by applying context-independent phonetic confusion rules, whose probabilities are estimated from training data. Using this approach, the word error rate on a non-native test set is reduced from 20.9% to 18.8%.
---
paper_title: Speech recognition for multiple non-native accent groups with speaker-group-dependent acoustic models.

paper_content:

In this paper, the recognition performance for non-native English speech with two different kinds of speaker-groupdependent acoustic models is investigated. The approaches for creating speaker groups include knowledge-based grouping of non-native speakers by their first language, and the automatic clustering of speakers. Clustering is based on speakerdependent acoustic models in speaker Eigenspace. The acoustic model for each speaker group is obtained by bootstrapping with pre-segmented speech data or adaptation of a speakerindependent native baseline model. For the decoding of a nonnative speaker’s utterance not seen during the training or adaptation phase, the selection of a model suitable to cope with the accent characteristics of that speaker is necessary. Here, ideal selection via an oracle and parallel decoding are examined. Evaluation is conducted in a hotel reservation task for five major accent groups, including German, French, Indonesian, Chinese and Japanese speakers. Recognition results with speakerdependent and an accent-independent non-native model will also be reported.
---
paper_title: Pronunciation modeling using a finite-state transducer representation

paper_content:

Abstract The MIT summit speech recognition system models pronunciation using a phonemic baseform dictionary along with rewrite rules for modeling phonological variation and multi-word reductions. Each pronunciation component is encoded within a finite-state transducer (FST) representation whose transition weights can be trained using an EM algorithm for finite-state networks. This paper explains the modeling approach we use and the details of its realization. We demonstrate the benefits and weaknesses of the approach both conceptually and empirically using the recognizer for our jupiter weather information system. Our experiments demonstrate that the use of phonological rewrite rules within our system achieves word error rate reductions between 4% and 9% over different test sets when compared against a system using no phonological rewrite rules.
---
paper_title: A study of multilingual speech recognition

paper_content:

This paper describes our work in developing multilingual (Swedish and English) speech recognition systems in the ATIS domain. The acoustic component of the multilingual systems is realized through sharing Gaussian codebooks across Swedish and English allophones. The language model (LM) components are constructed by training a statistical bigram model, with a common backoff node, on bilingual texts, and by combining two monolingual LMs into a probabilistic finite state grammar. This system uses a single decoder for Swedish and English sentences, and is capable of recognizing sentences with words from both languages. Preliminary experiments show that sharing acoustic models across the two languages has not resulted in improved performance, while sharing a backoff node at the LM component provides flexibility and ease in recognizing bilingual sentences at the expense of a slight increase in word error rate in some cases. As a by-product, the bilingual decoder also achieves good performance on language identification (LID).
---
paper_title: Multilingual speech recognition in seven languages

paper_content:

Abstract In this study we present approaches to multilingual speech recognition. We first define different approaches, namely portation, cross-lingual and simultaneous multilingual speech recognition. We will show some experiments performed in the fields of multilingual speech recognition. In recent years we have ported our recognizer to other languages than German (Italian, Slovak, Slovenian, Czech, English, Japanese). We found that some languages achieve a higher recognition performance with comparable tasks, and are thus easier for automatic speech recognition than others. Furthermore, we present experiments which show the performance of cross-lingual speech recognition of an untrained language with a recognizer trained with other languages. The substitution of phones is important for cross-lingual and simultaneous multilingual recognition. We compared results in cross-lingual recognition for different baseline systems and found that the number of shared acoustic units is very important for the performance. With simultaneous multilingual recognition, performance usually decreases compared to monolingual recognition. In few cases, like in the case of non-native speech, however, the recognition can be improved.
---
paper_title: Multiple Models for Improved Speech Recognition for Non-Native Speakers

paper_content:

Speech recognition of foreign accented speech is one of the most difficult tasks in ASR. The problem of foreign accent is addressed in this study using acoustic models of the target language phonemes (French phonemes in our case) adapted with speech data from 3 other languages: English (US and UK), German and Spanish. Recognition results obtained for 11 language groups of speakers show that error rate can be significantly reduced when standard acoustic models of phonemes are adapted using speech data from other languages. Phonological rules are also introduced into the standard phonetic description of the lexical units to account for some foreign accent pronunciation variants. It appears that using phonological rules together with foreign language adapted acoustic units provides the best recognition performance. The highest error rate reduction (40%) is obtained on English speakers.
---
paper_title: Context dependent "long units" for speech recognition.

paper_content:

It is expected that longer-than-phoneme units such as syllables or multi-phone units can deal with sources of performance degradation, such as pronunciation variation or coarticulation, better than phoneme-sized units like triphones. The possible number of contextual realizations of those “long units” (LU) is very high, causing an explosion of the number of parameters to be estimated. As the training data are limited, the usual solution is to share parameters between different units to improve parameter estimation. Another problem is how to provide a model for a unit (syllable/multiphones) that was not present during training (unseen unit). In this paper we evaluate and compare syllable and automatically derived multi-phone units. We introduce a method called “contextual factorization” to share parameters between different models and we propose a figure of merit to decide which decomposition of an unseen syllable is the most appropriate. Performance is improved comparing to a triphone based system.
---
paper_title: Using accent-specific pronunciation modelling for robust speech recognition

paper_content:

A method of modelling accent specific pronunciation variations is presented. Speech from an unseen accent group is phonetically transcribed such that pronunciation variations may be derived. These context dependent variations an clustered in a decision tree which is used as a model of the pronunciation variation associated with this new accent group. The tree is then used to build a new pronunciation dictionary for use during the recognition process. Experiments are presented for the recognition of Lancashire and Yorkshire accented speech using a recognizer trained on London and South East England speakers. The results show that the addition of accent specific pronunciations can reduce the error rate by almost 20% for cross accent recognition. It is also shown that worthwhile gains in performance can be obtained using only a small amount of accent specific data.
---
paper_title: Rate-Of-Speech Modeling For Large Vocabulary Conversational Speech Recognition

paper_content:

Variations in rate of speech (ROS) produce changes in both spectral features and word pronunciations that affect automatic speech recognition (ASR) systems. To deal with these ROS effects, we propose to use parallel, rate-specific, acoustic models: one for fast speech, the other for slow speech. Rate switching is permitted at word boundaries, to allow modeling within-sentence speech rate variation, which is common in conversational speech. Due to the parallel structure of ratespecific models and the maximum likelihood decoding method, we do not need high-quality ROS estimation before recognition, which is usually hard to achieve. In this paper, we evaluate our approach on a large-vocabulary conversational speech recognition (LVCSR) task over the telephone, with several minimal pair comparisons based on different baseline systems. Experiments show that on a development set for the 2000 Hub-5 evaluation, introducing word-level ROS-dependent models results in a 1.9% absolute win over a baseline system without multiword pronunciation modeling, and a 0.7% absolute win over a baseline system that incorporates a 4.0% absolute win from multiword pronunciation modeling. The combination of rate-dependent acoustic models with rate-dependent pronunciations obtained by using a data-driven approach is also explored and shown to produce an additional win.
---
paper_title: Language based phone model combination for ASR adaptation to foreign accent

paper_content:

The aim of this study is to improve automatic speech recognition performance when the speech material to be recognised is produced by non-native speakers and therefore prone to a foreign accent. A foreign accent generally causes a drop in recognition performance. In the this study, it is shown that adding modelling units from the native language into the phonetic pronunciation network can largely improve the recognition of non-native speakers without significantly degrading the performance of native speakers. Moreover, when some extra pronunciation variants are also added into the word phonetic transcription network, the recognition improvement becomes even larger. On the different corpora used in this paper, the error rate reduction ranged from 11% to 71%.
---
paper_title: Recognition Of Non-Native German Speech With Multilingual Recognizers

paper_content:

In this study we present di erent approaches to the recognition of non-natives. With a corpus in German spoken by speakers with 56 di erent rst languages, the Strange Corpus, we perform recognition experiments with monolingual and multilingual recognizers. Among other, we compared two German recognizers, one that was trained in addition with non-native (Italian) speech and the other trained with German speakers only. We found that best performance is achieved with the recognizer trained with German including non-native speech, followed by a bilingual recognizer and an Italian recognizer trained with German and Italian natives.
---
paper_title: Alternate phone models for conversational speech

paper_content:

This paper investigates the use of alternate phone models for the transcription of conversational telephone speech. The focus of this work is to explore alternative ways of modeling different manners of speaking so as to better cover the observed articulatory styles and pronunciation variants. Four alternate phone sets are compared ranging from 38 to 129 units. Two of the phone sets make use of syllable-position dependent phone models. The acoustic models were trained on 2300 hours of conversational telephone speech data from the Switchboard and Fisher corpora, and experimental results are reported on the EARS Dev04 test set which contains 3 hours of speech from 36 Fisher conversations. While no one particular phone set was found to outperform the others for a majority of speakers, the best overall performance was obtained with the original 48 phone set and a reduced 38 phone set, however combining the hypotheses of the individual models reduces the word error rate from 17.5% (original phone set) to 16.8%.
---
paper_title: LARGE CORPUS EXPERIMENTS FOR BROADCAST NEWS RECOGNITION

paper_content:

A manually operated brake actuator for a wheeled walker and the like. For normal braking, a handle is manually moved from a neutral first position towards a second position for rotating a cable link to progressively pull on a brake cable to engage a wheel brake. When the handle is released, it returns to the first position. When the handle is manually moved from the first position to a third position, a parking link is rotated to push on and rotate the cable link to engage the brake. The parking link is rotated to a stable position when the handle is in the third position, wherein pull on the brake cable cannot release the brake until the handle is manually moved back to the first position.
---
paper_title: An efficient repair procedure for quick transcriptions

paper_content:

We describe an efficient procedure for automatic repair of quickly transcribed (QT) speech. QT speech, typically closed captioned data from television broadcasts, usually has a significant number of deletions and misspellings, and has a characteristic absence of disfluencies such as filled pauses (for example, um, uh). Errors of these kinds often throw an acoustic model training program out of alignment and make it hard for it to resynchronize. At best the erroneous utterance is discarded and does not benefit the training procedure. At worst, it could misalign and end up sabotaging the training data. The procedure we propose in this paper aims to cleanse such quick transcriptions so that they align better with the acoustic evidence and thus provide for better acoustic models for automatic speech recognition (ASR). Results from comparing our transcripts with those from careful transcriptions on the same corpus, and from comparable state-of-the-art methods are also presented and discussed.
---
paper_title: Cross-Domain and Cross-Language Portability of Acoustic Features Estimated by Multilayer Perceptrons

paper_content:

Recent results with phone-posterior acoustic features estimated by multilayer perceptrons (MLPs) have shown that such features can effectively improve the accuracy of state-of-the-art large vocabulary speech recognition systems. MLP features are trained discriminatively to perform phone classification and are therefore, like acoustic models, tuned to a particular language and application domain. In this paper we investigate how portable such features are across domains and languages. We show that even without retraining, English-trained MLP features can provide a significant boost to recognition accuracy in new domains within the same language, as well as in entirely different languages such as Mandarin and Arabic. We also show the effectiveness of feature-level adaptation in porting MLP features to new domains.
---
paper_title: Creating Speaker Independent HMM Models for Restricted Database Using STRAIGHT-TEMPO Morphing

paper_content:

ICSLP1998: the 5th International Conference on Spoken Language Processing, November 30 - December 4, 1998, Sydney, Australia.
---

<format>
Title: Automatic Speech Recognition and Speech Variability: A Review

Section 1: Introduction
Description 1: This section provides a broad overview of the challenges and complexities in automatic speech recognition (ASR) due to speech variability, and sets the stage for the detailed exploration that follows.

Section 2: Speech variability sources
Description 2: This section identifies and discusses the various intrinsic factors that cause variability in speech, affecting ASR performance.

Section 3: Foreign and regional accents
Description 3: This section examines the impact of foreign and regional accents on ASR, and how these variances are currently addressed.

Section 4: Speaker physiology
Description 4: This section describes how physiological differences between speakers affect speech signals and highlights methods to mitigate their impact on ASR.

Section 5: Speaking style and spontaneous speech
Description 5: This section investigates the effects of different speaking styles and spontaneous speech patterns on ASR accuracy.

Section 6: Rate of speech
Description 6: This section explores how varying speech rates impact ASR performance and the techniques developed to handle these variations.

Section 7: Children speech
Description 7: This section delves into the unique challenges posed by children's speech in ASR systems and the specific approaches to address these challenges.

Section 8: Emotional state
Description 8: This section examines how emotional states influence speech signal characteristics and the resultant effects on ASR systems.

Section 9: And more...
Description 9: This section briefly touches upon other less-studied sources of speech variability, like medical conditions and physical activities.

Section 10: ASR performance analysis and diagnosis
Description 10: This section discusses methods and techniques for analyzing and diagnosing ASR performance issues, including error analysis and statistical comparisons.

Section 11: Man-machine comparison
Description 11: This section compares the performance of humans and ASR systems in recognizing speech under various conditions, identifying key areas for improvement in ASR.

Section 12: Specific methodologies
Description 12: This section reviews specific experimental methodologies to identify and address weaknesses in ASR systems, focusing on feature extraction and modeling limitations.

Section 13: ASR techniques
Description 13: This section provides an overview of techniques aimed at improving ASR robustness and accuracy, including feature extraction, compensation, and multi-stream approaches.

Section 14: Front-end techniques
Description 14: This section revisits the advancements in front-end feature extraction techniques to handle speech variability.

Section 15: Overcoming assumptions
Description 15: This section critiques the standard assumptions in ASR processing and proposes alternative methods to handle speech non-stationarity and other nuances.

Section 16: Compensation and invariance
Description 16: This section discusses techniques to achieve invariance in feature extraction and compensation for variability factors.

Section 17: Additional cues and multiple feature streams
Description 17: This section highlights the benefits of incorporating additional acoustic cues and combining multiple feature streams to enhance ASR performance.

Section 18: Dimensionality reduction and feature selection
Description 18: This section reviews methods for dimensionality reduction and feature selection to streamline feature vectors while retaining important information.

Section 19: Acoustic modeling techniques
Description 19: This section explores advanced acoustic modeling techniques, including adaptation, multi-style training, and the use of auxiliary features.

Section 20: Adaptation
Description 20: This section elaborates on model adaptation methodologies to tailor ASR systems for specific speakers or environments.

Section 21: Multiple modeling
Description 21: This section discusses the use of multiple acoustic models to handle specific conditions or variations, such as gender, age, and speaking rate.

Section 22: Auxiliary acoustic features
Description 22: This section examines the integration of auxiliary features like pitch or energy into ASR systems to improve robustness.

Section 23: Pronunciation modeling techniques
Description 23: This section covers the strategies for modeling pronunciation variations, especially under conditions of accent and spontaneous speech.

Section 24: Larger and diverse training corpora
Description 24: This section discusses the necessity and benefits of using large and diverse training data sets to improve ASR performance.

Section 25: Conclusion
Description 25: This section summarizes the key points covered in the paper, emphasizing the importance of addressing speech variability to enhance ASR systems.
</format>
