Exploiting nonacoustic sensors for speech encoding

The intelligibility of speech transmitted through low-rate coders is severely degraded when high levels of acoustic noise are present in the acoustic environment. Recent advances in nonacoustic sensors, including microwave radar, skin vibration, and bone conduction sensors, provide the exciting possibility of both glottal excitation and, more generally, vocal tract measurements that are relatively immune to acoustic disturbances and can supplement the acoustic speech waveform. We are currently investigating methods of combining the output of these sensors for use in low-rate encoding according to their capability in representing specific speech characteristics in different frequency bands. Nonacoustic sensors have the ability to reveal certain speech attributes lost in the noisy acoustic signal; for example, low-energy consonant voice bars, nasality, and glottalized excitation. By fusing nonacoustic low-frequency and pitch content with acoustic-microphone content, we have achieved significant intelligibility performance gains using the DRT across a variety of environments over the government standard 2400-bps MELPe coder. By fusing quantized high-band 4-to-8-kHz speech, requiring only an additional 116 bps, we obtain further DRT performance gains by exploiting the ear's insensitivity to fine spectral detail in this frequency region.     

AM-FM energy detection and separation in noise using multibandenergy operators

This paper develops a multiband or wavelet approach for capturing the AM-FM components of modulated signals immersed in noise. The technique utilizes the recently-popularized nonlinear energy operator Ψ(s)=(s˙)²-ss¨ to isolate the AM-FM energy, and an energy separation algorithm (ESA) to extract the instantaneous amplitudes and frequencies. It is demonstrated that the performance of the energy operator/ESA approach is vastly improved if the signal is first filtered through a bank of bandpass filters, and at each instant analyzed (via Ψ and the ESA) using the dominant local channel response. Moreover, it is found that uniform (worst-case) performance across the frequency spectrum is attained by using a constant-Q, or multiscale wavelet-like filter bank. The elementary stochastic properties of Ψ and of the ESA are developed first. The performance of Ψ and the ESA when applied to bandpass filtered versions of an AM-FM signal-plus-noise combination is then analyzed. The predicted performance is greatly improved by filtering, if the local signal frequencies occur in-band. These observations motivate the multiband energy operator and ESA approach, ensuring the in-band analysis of local AM-PM energy. In particular, the multi-bands must have the constant-Q or wavelet scaling property to ensure uniform performance across bands. The theoretical predictions and the simulation results indicate that improved practical strategies are feasible for tracking and identifying AM-FM components in signals possessing pattern coherencies manifested as local concentrations of frequencies     

Peak-to-RMS reduction of speech based on a sinusoidal model

A sinusoidal-based analysis/synthesis system is used to apply a radar design solution to the problem of dispersing the phase of a speech waveform. Unlike conventional methods of phase dispersion, this solution technique adapts dynamically to the pitch and spectral characteristics of the speech, while maintaining the original spectral envelope. The solution can also be used to drive the sine-wave amplitude modification for amplitude compression, and is coupled to the desired shaping of the speech spectrum. The proposed dispersion solution, when integrated with amplitude compression, results in a significant reduction in the peak-to-RMS (root-mean-square) ratio of the speech waveform with acceptable loss in quality. Application of a real-time prototype sine-wave preprocessor to AM radio broadcasting is described     

Spectral Representations of Nonmodal Phonation

Regions of nonmodal phonation, which exhibit deviations from uniform glottal-pulse periods and amplitudes, occur often in speech and convey information about linguistic content, speaker identity, and vocal health. Some aspects of these deviations are random, including small perturbations, known as jitter and shimmer, as well as more significant aperiodicities. Other aspects are deterministic, including repeating patterns of fluctuations such as diplophonia and triplophonia. These deviations are often the source of misinterpretation of the spectrum. In this paper, we introduce a general signal-processing framework for interpreting the effects of both stochastic and deterministic aspects of nonmodality on the short-time spectrum. As an example, we show that the spectrum is sensitive to even small perturbations in the timing and amplitudes of glottal pulses. In addition, we illustrate important characteristics that can arise in the spectrum, including apparent shifting of the harmonics and the appearance of multiple pitches. For stochastic perturbations, we arrive at a formulation of the power-spectral density as the sum of a low-pass line spectrum and a high-pass noise floor. Our findings are relevant to a number of speech-processing areas including linear-prediction analysis, sinusoidal analysis-synthesis, spectrally derived features, and the analysis of disordered voices.     

Energy separation in signal modulations with application to speechanalysis

An efficient solution to the fundamental problem of estimating the time-varying amplitude envelope and instantaneous frequency of a real-valued signal that has both an AM and FM structure is provided. Nonlinear combinations of instantaneous signal outputs from the energy operator are used to separate its output energy product into its AM and FM components. The theoretical analysis is done first for continuous-time signals. Then several efficient algorithms are developed and compared for estimating the amplitude envelope and instantaneous frequency of discrete-time AM-FM signals. These energy separation algorithms are used to search for modulations in speech resonances, which are modeled using AM-FM signals to account for time-varying amplitude envelopes and instantaneous frequencies. The experimental results provide evidence that bandpass-filtered speech signals around speech formants contain amplitude and frequency modulations within a pitch period     

Estimation of modulation based on FM-to-AM transduction:two-sinusoid case

A method is described for estimating the amplitude modulation (AM) and the frequency modulation (FM) of the components of a signal that consists of two AM-FM sinusoids. The approach is based on the transduction of FM to AM that occurs whenever a signal of varying frequency passes through a filter with a nonflat frequency response. The objective is to separate the AM and FM of the sinusoids from the amplitude envelopes of the output of two transduction filters, where the AM and FM are nonlinearly combined in the amplitude envelopes. The current scheme is first refined for AM-FM estimation of a single AM-FM sinusoid by iteratively inverting the AM and FM estimates to reduce error introduced in transduction. The transduction filter pair is designed relying on both a time- and frequency-domain characterization of transduction error. The approach is then extended to the case of two AM-FM sinusoids by essentially reducing the problem to two single-component AM-FM estimation problems. By exploiting the beating in the amplitude envelope of each filter output due to the two-sinusoidal input, a closed-form solution is obtained. This solution is also improved upon by iterative refinement. The AM-FM estimation methods are evaluated through an error analysis and are illustrated for a wide range of AM-FM signals     

Shape invariant time-scale and pitch modification of speech

The simplified linear model of speech production predicts that when the rate of articulation is changed, the resulting waveform takes on the appearance of the original, except for a change in the time scale. A time-scale modification system that preserves this shape-invariance property during voicing is developed. This is done using a version of the sinusoidal analysis-synthesis system that models and independently modifies the phase contributions of the vocal tract and vocal cord excitation. An important property of the system is its ability to perform time-varying rates of change. Extensions of the method are applied to fixed and time-varying pitch modification of speech. The sine-wave analysis-synthesis system also allows for shape-invariant joint time-scale and pitch modification, and allows for the adjustment of the time scale and pitch according to speech characteristics such as the degree of voicing     

Far-echo cancellation in the presence of frequency offset [fullduplex modem]

A design is presented for a full-duplex echo-cancelling data modem based on a combined adaptive reference echo canceller and adaptive channel equalizer. The adaptive reference algorithm has the advantage that interference to the echo canceller caused by the far-end signal can be eliminated by subtracting an estimate of the far-end signal based on receiver decisions. This technique provides a novel approach for full-duplex far-echo cancellation in which the far echo can be cancelled in spite of carrier-frequency offset. To estimate the frequency offset, the system uses a separate receiver structure for the far echo which provides equalization of the far echo channel and tracks the frequency offset in the far echo. The feasibility of the echo-cancelling algorithms is demonstrated by computer simulation with realistic channel distortions and with 4800-b/s data transmission, at which rate the frequency offset in the far echo becomes important