Survey of Sparse and Non-Sparse Methods in Source Separation
---
paper_title: Infomax and maximum likelihood for blind source separation

paper_content:

Algorithms for the blind separation of sources can be derived from several different principles. This article shows that the infomax (information-maximization) principle is equivalent to the maximum likelihood. The application of the infomax principle to source separation consists of maximizing an output entropy.
---
paper_title: BSS: a new approach for watermark attack

paper_content:

Digital watermarking is an enabling technology to prove ownership on copyrighted material, detect originators of illegally made copies, monitor the usage of copyrighted multimedia data, and analyze the spread spectrum of data over networks and servers. Most watermarking methods for images and video can be viewed as a communications problem in which the watermark must be transmitted and received through a watermark channel. This channel includes distortions resulting from attacks and interference from the original digital data. It is well accepted that an effective watermarking scheme may be described as the secure, imperceptible, robust communication of information by direct embedding in, and retrieval from, digital data. For verifying the security and robustness of watermarking algorithms, specific attacks have to be applied to test them. By using a theoretical approach based on random processes, signal processes, and communication theory, we propose a stochastic formulation of a new watermarking attack using a blind source separation-based concept. The proposed attack considers the watermarking channel as a "black-box". A host image was passed through the "black-box", which includes the watermarking embedding process, and then the watermarked image was produced The watermarked image is viewed as linear mixtures of unknown source signals, and we attempt to recover sources from their linear mixtures without resorting to any prior knowledge by using blind source separation theory.
---
paper_title: Analyzing and Visualizing Single-Trial Event-Related Potentials

paper_content:

Event-related potentials (ERPs), are portions of electroencephalographic (EEG) recordings that are both time- and phase-locked to experimental events. ERPs are usually averaged to increase their signal/noise ratio relative to non-phase locked EEG activity, regardless of the fact that response activity in single epochs may vary widely in time course and scalp distribution. This study applies a linear decomposition tool, Independent Component Analysis (ICA) [1], to multichannel single-trial EEG records to derive spatial filters that decompose single-trial EEG epochs into a sum of temporally independent and spatially fixed components arising from distinct or overlapping brain or extra-brain networks. Our results on normal and autistic subjects show that ICA can separate artifactual, stimulus-locked, response-locked, and. nonevent related background EEG activities into separate components, allowing (1) removal of pervasive artifacts of all types from single-trial EEG records, and (2) identification of both stimulus- and response-locked EEG components. Second, this study proposes a new visualization tool, the 'ERP image', for investigating variability in latencies and amplitudes of event-evoked responses in spontaneous EEG or MEG records. We show that sorting single-trial ERP epochs in order of reaction time and plotting the potentials in 2-D clearly reveals underlying patterns of response variability linked to performance. These analysis and visualization tools appear broadly applicable to electrophyiological research on both normal and clinical populations.
---
paper_title: Progress in blind separation of magnetoencephalographic data

paper_content:

The match between the physics of MEG and the assumptions of the most well developed blind source separation (BSS) algorithms (unknown instantaneous linear mixing process, many sensors compared to expected recoverable sources, large data limit) have tempted researchers to apply these algorithms to MEG data. We review some of these efforts, with particular emphasis on our own work.
---
paper_title: Independent component approach to the analysis of EEG and MEG recordings

paper_content:

Multichannel recordings of the electromagnetic fields emerging from neural currents in the brain generate large amounts of data. Suitable feature extraction methods are, therefore, useful to facilitate the representation and interpretation of the data. Recently developed independent component analysis (ICA) has been shown to be an efficient tool for artifact identification and extraction from electroencephalographic (EEG) and magnetoencephalographic (MEG) recordings. In addition, ICA has been applied to the analysis of brain signals evoked by sensory stimuli. This paper reviews our recent results in this field.
---
paper_title: Learning of sparse auditory receptive fields

paper_content:

It is largely unknown how the properties of the auditory system relate to the properties of natural sounds. Here, we analyze representations of simulated neurons that have optimally sparse activity in response to spectro-temporal speech data. These representations share important properties with the auditory neurons determined in electrophysiological experiments.
---
paper_title: An Application of the Principle of Maximum Information Preservation to Linear Systems

paper_content:

This paper addresses the problem of determining the weights for a set of linear filters (model "cells") so as to maximize the ensemble-averaged information that the cells' output values jointly convey about their input values, given the statistical properties of the ensemble of input vectors. The quantity that is maximized is the Shannon information rate, or equivalently the average mutual information between input and output. Several models for the role of processing noise are analyzed, and the biological motivation for considering them is described. For simple models in which nearby input signal values (in space or time) are correlated, the cells resulting from this optimization process include center-surround cells and cells sensitive to temporal variations in input signal.
---
paper_title: Blind separation of disjoint orthogonal signals: demixing N sources from 2 mixtures

paper_content:

We present a novel method for blind separation of any number of sources using only two mixtures. The method applies when sources are (W-)disjoint orthogonal, that is, when the supports of the (windowed) Fourier transform of any two signals in the mixture are disjoint sets. We show that, for anechoic mixtures of attenuated and delayed sources, the method allows one to estimate the mixing parameters by clustering ratios of the time-frequency representations of the mixtures. The estimates of the mixing parameters are then used to partition the time-frequency representation of one mixture to recover the original sources. The technique is valid even in the case when the number of sources is larger than the number of mixtures. The general results are verified on both speech and wireless signals.
---
paper_title: An Information-Maximization Approach to Blind Separation and Blind Deconvolution

paper_content:

We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximization has extra properties not found in the linear case (Linsker 1989). The nonlinearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalization of principal components analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to 10 speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information maximization provides a unifying framework for problems in "blind" signal processing.
---
paper_title: Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture

paper_content:

Abstract The separation of independent sources from an array of sensors is a classical but difficult problem in signal processing. Based on some biological observations, an adaptive algorithm is proposed to separate simultaneously all the unknown independent sources. The adaptive rule, which constitutes an independence test using non-linear functions, is the main original point of this blind identification procedure. Moreover, a new concept, that of INdependent Components Analysis (INCA), more powerful than the classical Principal Components Analysis (in decision tasks) emerges from this work.
---
paper_title: A First Application of Independent Component Analysis to Extracting Structure from Stock Returns

paper_content:

This paper discusses the application of a modern signal processing technique known as independentcomponent analysis (ICA) or blind source separation to multivariate financial time series such as aportfolio of stocks. The key idea of ICA is to linearly map the observed multivariate time series into a newspace of statistically independent components (ICs). This can be viewed as a factorization of the portfoliosince joint probabilities become simple products in the coordinate system of the ICs.We apply ICA to three years of daily returns of the 28 largest Japanese stocks and compare the results withthose obtained using principal component analysis. The results indicate that the estimated ICs fall into twocategories, (i) infrequent but large shocks (responsible for the major changes in the stock prices), and (ii)frequent smaller fluctuations (contributing little to the overall level of the stocks). We show that the overallstock price can be reconstructed surprisingly well by using a small number of thresholded weighted ICs.In contrast, when using shocks derived from principal components instead of independent components, thereconstructed price is less similar to the original one. Independent component analysis is a potentially powerfulmethod of analyzing and understanding driving mechanisms in financial markets. There are furtherpromising applications to risk management since ICA focuses on higher-order statistics.
---
paper_title: Sparse Coding In The Primate Cortex

paper_content:

A central goal in the study of cortical function is to understand how states of the environment are represented by firing patterns of cortical neurons. Electrophysiological recordings from single cells have revealed a remarkably close relationship among stimuli, neural activity, and perceptual states. The nature of this relationship and interpretations of experimental results are fiercely debated. Is sensory information represented by the activity of single, individually meaningful cells, or is it only the global activity pattern across a whole cell population that corresponds to interpretable states? There are now strong theoretical reasons and experimental evidence suggesting that the brain adopts a compromise between these extremes which is often referred to as sparse coding.
---
paper_title: Blind separation of speech mixtures via time-frequency masking

paper_content:

Binary time-frequency masks are powerful tools for the separation of sources from a single mixture. Perfect demixing via binary time-frequency masks is possible provided the time-frequency representations of the sources do not overlap: a condition we call W-disjoint orthogonality. We introduce here the concept of approximate W-disjoint orthogonality and present experimental results demonstrating the level of approximate W-disjoint orthogonality of speech in mixtures of various orders. The results demonstrate that there exist ideal binary time-frequency masks that can separate several speech signals from one mixture. While determining these masks blindly from just one mixture is an open problem, we show that we can approximate the ideal masks in the case where two anechoic mixtures are provided. Motivated by the maximum likelihood mixing parameter estimators, we define a power weighted two-dimensional (2-D) histogram constructed from the ratio of the time-frequency representations of the mixtures that is shown to have one peak for each source with peak location corresponding to the relative attenuation and delay mixing parameters. The histogram is used to create time-frequency masks that partition one of the mixtures into the original sources. Experimental results on speech mixtures verify the technique. Example demixing results can be found online at http://alum.mit.edu/www/rickard/bss.html.
---
paper_title: Removing Electroencephalographic Artifacts by Blind Source Separation

paper_content:

Eye movements, eye blinks, cardiac signals, muscle noise, and line noise present serious problems for electroencephalographic (EEG) interpretation and analysis when rejecting contaminated EEG segments results in an unacceptable data loss. Many methods have been proposed to remove artifacts from EEG recordings, especially those arising from eye movements and blinks. Often regression in the time or frequency domain is performed on parallel EEG and electrooculographic (EOG) recordings to derive parameters characterizing the appearance and spread of EOG artifacts in the EEG channels. Because EEG and ocular activity mix bidirectionally, regressing out eye artifacts inevitably involves subtracting relevant EEG signals from each record as well. Regression methods become even more problematic when a good regressing channel is not available for each artifact source, as in the case of muscle artifacts. Use of principal component analysis (PCA) has been proposed to remove eye artifacts from multichannel EEG. However, PCA cannot completely separate eye artifacts from brain signals, especially when they have comparable amplitudes. Here, we propose a new and generally applicable method for removing a wide variety of artifacts from EEG records based on blind source separation by independent component analysis (ICA). Our results on EEG data collected from normal and autistic subjects show that ICA can effectively detect, separate, and remove contamination from a wide variety of artifactual sources in EEG records with results comparing favorably with those obtained using regression and PCA methods. ICA can also be used to analyze blink-related brain activity.
---
paper_title: Adaptive separation of acoustic sources for anechoic conditions: A constrained frequency domain approach

paper_content:

Blind source separation represents a signal processing technique with a large potential for noise reduction. However, its application in modern digital hearing aids poses high demands with respect to computational efficiency and speed of adaptation towards the desired solution. In this paper, an algorithm is presented which fulfills these goals under the idealized assumption that the superposition of sources in rooms can be approximated as a superposition under anechoic conditions. Specifically, attenuation, the signals' finite propagation speed, and diffuse noise are accounted for, whereas reflections and reverberation are considered as negligible effects. This approximation is referred to as the 'free field' assumption. Starting from a general blind source separation algorithm for Fourier transformed speech signals, the free field assumption is incorporated into the framework, yielding a simple, fast and adaptive algorithm that is able to track moving sources. Implementation details are given which were found to be indispensable for fast and robust signal separation. Performance is evaluated both by simulations and experimentally, including separation of a moving and a fixed speaker in a recorded real anechoic environment. The potential benefits and shortcomings of this algorithm are discussed with regard to its inclusion into the signal processing framework of digital hearing aids for real reverberant acoustic situations.
---
paper_title: Real-Time Sound Source Localization and Separation for Robot Audition

paper_content:

Robot audition in the real world should cope with environment noises and reverberation and motor noises caused by the robot’s own movements. This paper presents the active direction-pass filter (ADPF) to separate sounds originating from the specified direction with a pair of microphones. The ADPF is implemented by hierarchical integration of visual and auditory processing with hypothetical reasoning on interaural phase difference (IPD) and interaural intensity difference (IID) for each subband. In creating hypotheses, the reference data of IPD and IID is calculated by the auditory epipolar geometry on demand. Since the performance of the ADPF depends on the direction, the ADPF controls the direction by motor movement. The human tracking and sound source separation based on the ADPF is implemented on an upper-torso humanoid and runs in real-time with 4 PCs connected over Gigabit ethernet. The signal-to-noise ratio (SNR) of each sound separated by the ADPF from a mixture of two speeches with the same loudness is improved to about 10 dB from 0 dB.
---
paper_title: Independent component analysis , A new concept ? *

paper_content:

Abstract The independent component analysis (ICA) of a random vector consists of searching for a linear transformation that minimizes the statistical dependence between its components. In order to define suitable search criteria, the expansion of mutual information is utilized as a function of cumulants of increasing orders. An efficient algorithm is proposed, which allows the computation of the ICA of a data matrix within a polynomial time. The concept of ICA may actually be seen as an extension of the principal component analysis (PCA), which can only impose independence up to the second order and, consequently, defines directions that are orthogonal. Potential applications of ICA include data analysis and compression, Bayesian detection, localization of sources, and blind identification and deconvolution.
---
paper_title: A fast fixed - point algorithm for independent component analysis

paper_content:

An apparatus for hydrolytic degradation of plastics in which plastic material is deposited into a tubular housing via a feed hopper. An elongated screw shaft has a first section in the form of a high pitch screw thread disposed below the feed hopper to receive and advance the material to a second section. The second section of the screw shaft is in the form of a lower pitch thread for compressing the plastic material and transferring it to a longer, third section in the form of kneading discs, from which material passes through an outlet nozzle section to a cyclone separator where trapped gases and liquid may be withdrawn. The tubular housing is vented upstream of the feed hopper and a water inlet pipe is disposed adjacent to the second section of the screw shaft, downstream of the feed hopper. The outlet nozzle section is provided with pressure measuring and regulating means and a liquid level measuring and regulating device.
---
paper_title: Real-Time Time-Frequency Based Blind Source Separation

paper_content:

We present a real-time version of the DUET algorithm for the blind separation of any number of sources using only two mixtures. The method applies when sources are Wdisjoint orthogonal, that is, when the supports of the windowed Fourier transform of any two signals in the mixture are disjoint sets, an assumption which is justified in the Appendix. The online algorithm is a Maximum Likelihood (ML) based gradient search method that is used to track the mixing parameters. The estimates of the mixing parameters are then used to partition the time-frequency representation of the mixtures to recover the original sources. The technique is valid even in the case when the number of sources is larger than the number of mixtures. The method was tested on mixtures generated from different voices and noises recorded from varying angles in both anechoic and echoic rooms. In total, over 1500 mixtures were tested. The average SNR gain of the demixing was 15 dB for anechoic room mixtures and 5 dB for echoic office mixtures. The algorithm runs 5 times faster than real time on a 750MHz laptop computer. Sample sound files can be found here:
---
paper_title: Independent component analysis of noninvasively recorded cortical magnetic DC-fields in humans

paper_content:

We apply a recently developed multivariate statistical data analysis technique-so called blind source separation (BSS) by independent component analysis-to process magnetoencephalogram recordings of near-DC fields. The extraction of near-DC fields from MEG recordings has great relevance for medical applications since slowly varying DC-phenomena have been found, e.g., in cerebral anoxia and spreading depression in animals. Comparing several BSS approaches, it turns out that an algorithm based on temporal decorrelation successfully extracted a DC-component which was induced in the auditory cortex by presentation of music. The task is challenging because of the limited amount of available data and the corruption by outliers, which makes it an interesting real-world testbed for studying the robustness of ICA methods.
---
paper_title: Analysis of fMRI Data by Blind Separation Into Independent Spatial Components

paper_content:

r r Abstract: Current analytical techniques applied to functional magnetic resonance imaging (fMRI) data require a priori knowledge or specific assumptions about the time courses of processes contributing to the measured signals. Here we describe a new method for analyzing fMRI data based on the independent component analysis (ICA) algorithm of Bell and Sejnowski ((1995): Neural Comput 7:1129-1159). We decomposed eight fMRI data sets from 4 normal subjects performing Stroop color-naming, the Brown and Peterson word/number task, and control tasks into spatially independent components. Each component consisted of voxel values at fixed three-dimensional locations (a component ''map''), and a unique associated time course of activation. Given data from 144 time points collected during a 6-min trial, ICA extracted an equal number of spatially independent components. In all eight trials, ICA derived one and only one component with a time course closely matching the time course of 40-sec alternations between experimental and control tasks. The regions of maximum activity in these consistently task-related components generally overlapped active regions detected by standard correlational analysis, but included frontal regions not detected by correlation. Time courses of other ICA components were transiently task-related, quasiperiodic, or slowly varying. By utilizing higher-order statistics to enforce successively stricter criteria for spatial independence between component maps, both the ICA algorithm and a related fourth-order decomposition technique (Comon (1994): Signal Processing 36:11-20) were superior to principal component analysis (PCA) in determining the spatial and temporal extent of task-related activation. For each subject, the time courses and active regions of the task-related ICA components were consistent across trials and were robust to the addition of simulated noise. Simulated movement artifact and simulated task-related activations added to actual fMRI data were clearly separated by the algorithm. ICA can be used to distinguish between nontask-related signal components, movements, and other artifacts, as well as consistently or transiently task-related fMRI activations, based on only weak
---
paper_title: Second order blind equalization in multiple input multiple output FIR systems: a weighted least squares approach

paper_content:

Multipath propagation appears to be a typical limitation in mobile digital communications where it leads to severe intersymbol interference. The classical techniques to overcome this problem use either periodically sent training sequences or blind approaches. This paper addresses the blind identification of FIR multiple input multiple output (MIMO) transfer functions in the case where the number of inputs is strictly less than the number of outputs. We consider a second order identification providing signals extraction up to a regular instantaneous mixture matrix. A novel method of second order channel estimation is presented. Basically exploiting the simultaneous MA and AR nature of the observations, it displays however a considerable improvement as compared to the original linear prediction approach. Performance study and comparison with the existing approach is provided by computer simulations.
---
paper_title: Notes on Cost Functions and Estimators for `Bussgang' Adaptive Blind Equalization

paper_content:

The aim of this paper is to present some remarks on the cost functions for blind channel equalization by ‘Bussgang’ algorithms recently discussed in the literature; also, some possible associated non-Bayesian estimators are considered with details, and the effects of the choice of such estimators in relation to ‘Bussgang’ filtering are investigated.
---
paper_title: Blind signal separation of convolutive mixtures

paper_content:

A novel algorithm is described for the blind separation of signals which have been mixed in a convolutive manner. It involves an initial process of strong decorrelation and spectral equalisation, based entirely on second order statistics. This is followed by the identification of a hidden paraunitary matrix which necessitates the use of higher (fourth) order statistics. The hidden paraunitary matrix is built up as a carefully chosen sequence of elementary paraunitary matrices.
---
paper_title: Convolutive Acoustic Mixtures Approximation to an Instantaneous Model Using a Stereo Boundary Microphone Configuration

paper_content:

In this work it is demonstrated that, taking into account the conditions of a convolutive mixture of acoustic signals, it is possible to configure a mixture system whose separation model accomplishes the conditions of an a instantaneous mixture. This system is achievable by using stereo boundary microphones; this type of coincident microphones can be used to uniform delays of the propagation channels and to reduce the number of reflections that characterize the impulse response of the system. By means of coincident boundary microphones techniques, instantaneous BSS algorithms are applicable, thus providing optimal results with less computational cost. This system is validated in both anechoic and reverberant chambers.
---
paper_title: Blind separation of convolutive mixtures using second and fourth order moments

paper_content:

This paper deals with the problem of blind identification of p-non Gaussian inputs, q-outputs AR systems in the special case where p<q. In this case, the identification problem is degenerated, therefore as the classical Levinson (Robinson) algorithm cannot be applied, we use the Inouye's (1983) method. As this procedure assumes that the AR model is normalized, it is necessary to split the problem in two parts: first, we estimate the convolutive mixture by means of linear prediction, and second, we estimate the instantaneous mixture. The first one requires second order moments and the second one, high order statistics. Numerical simulations are presented to show the influence of the conditioning of the instantaneous mixture matrix in the identification problem in presence of white Gaussian noise.
---
paper_title: Blind separation of convolved mixtures in the frequency domain

paper_content:

Abstract In this paper we employ information theoretic algorithms, previously used for separating instantaneous mixtures of sources, for separating convolved mixing in the frequency domain. It is observed that convolved mixing in the time domain corresponds to instantaneous mixing in the frequency domain. Such mixing can be inverted using simpler and more robust algorithms than the ones recently developed. Advantages of this approach are improved efficiency and better convergence features.
---
paper_title: An Approach to Blind Source Separation Based on Temporal Structure of Speech Signals

paper_content:

Abstract In this paper, we introduce a new technique for blind source separation of speech signals. We focus on the temporal structure of the signals. The idea is to apply the decorrelation method proposed by Molgedey and Schuster in the time–frequency domain. Since we are applying separation algorithm on each frequency separately, we have to solve the amplitude and permutation ambiguity properly to reconstruct the separated signals. For solving the amplitude ambiguity, we use the matrix inversion and for the permutation ambiguity, we introduce a method based on the temporal structure of speech signals. We show some results of experiments with both artificially controlled data and speech data recorded in the real environment.
---
paper_title: Multirate Systems and Filter Banks

paper_content:

1. Introduction 2. Review of Discrete-Time Systems 3. Review of Digital Filters 4. Fundamentals of Multirate Systems 5. Maximally Decimated Filter Banks 6. Paraunitary Perfect Reconstruction Filter Banks 7. Linear Phase Perfect Reconstruction QMF Banks 8. Cosine Modulated Filter Banks 9. Finite Word Length Effects 10. Multirate Filter Bank Theory and Related Topics 11. The Wavelet Transform and Relation to Multirate Filter Banks 12. Multidimensional Multirate Systems 13. Review of Discrete-Time Multi-Input Multi-Output LTI Systems 14. Paraunitary and Lossless Systems Appendices Bibliography Index
---
paper_title: Blind separation of multiple speakers in a multipath environment

paper_content:

We relate information theoretic blind learning methods (infomax) and Bussgang blind equalization methods. The multipath extension of blind source separation methods can be seen in the frequency domain using FIR matrix algebra (matrices of finite impulse response filters). Three forms of Bussgang algorithms are given. The blind serial update method of Cardoso and Laheld (1994) is related to the infomax objective of Bell and Sejnowski (1995). The application emphasis is on speech separation. We demonstrate the robustness and power of the new techniques by blindly separating speech signals recorded in a multipath environment.
---
paper_title: Adaptive separation of acoustic sources for anechoic conditions: A constrained frequency domain approach

paper_content:

Blind source separation represents a signal processing technique with a large potential for noise reduction. However, its application in modern digital hearing aids poses high demands with respect to computational efficiency and speed of adaptation towards the desired solution. In this paper, an algorithm is presented which fulfills these goals under the idealized assumption that the superposition of sources in rooms can be approximated as a superposition under anechoic conditions. Specifically, attenuation, the signals' finite propagation speed, and diffuse noise are accounted for, whereas reflections and reverberation are considered as negligible effects. This approximation is referred to as the 'free field' assumption. Starting from a general blind source separation algorithm for Fourier transformed speech signals, the free field assumption is incorporated into the framework, yielding a simple, fast and adaptive algorithm that is able to track moving sources. Implementation details are given which were found to be indispensable for fast and robust signal separation. Performance is evaluated both by simulations and experimentally, including separation of a moving and a fixed speaker in a recorded real anechoic environment. The potential benefits and shortcomings of this algorithm are discussed with regard to its inclusion into the signal processing framework of digital hearing aids for real reverberant acoustic situations.
---
paper_title: Monaural Source Separation Using Spectral Cues

paper_content:

The acoustic environment poses at least two important challenges. First, animals must localise sound sources using a variety of binaural and monaural cues; and second they must separate sources into distinct auditory streams (the “cocktail party problem”). Binaural cues include intra-aural intensity and phase disparity. The primary monaural cue is the spectral filtering introduced by the head and pinnae via the head-related transfer function (HRTF), which imposes different linear filters upon sources arising at different spatial locations.
---
paper_title: Sound source segregation based on estimating incident angle of each frequency component of input signals acquired by multiple microphones

paper_content:

We have developed a method of segregating desired speech from concurrent sounds received by two microphones. In this method, which we call SAFIA, signals received by two microphones are analyzed by discrete Fourier transformation. For each frequency component, differences in the amplitude and phase between channels are calculated. These differences are used to select frequency components of the signal that come from the desired direction and to reconstruct these components as the desired source signal. To clarify the effect of frequency resolution on the proposed method, we conducted three experiments. First, we analyzed the relationship between frequency resolition and the power spectrum’s cumulative distribution. We found that the speech-signal power was concentrated on specific frequency components when the frequency resolution was about 10-Hz. Second, we determined whether a given frequency resolution decreased the overlap between the frequency components of two speech signals. A 10-Hz frequency resolution minimized the overlap. Third, we analyzed the relationship between sound quality and frequency resolution through subjective tests. The best frequency resolution in terms of sound quality corresponded to the frequency resolutions that concentrated the speech signal power on specific frequency components and that minimized the degree of overlap. Finally, we demonstrated that this method improved the signal-to-noise ratio by over 18dB.
---
paper_title: Blind separation of disjoint orthogonal signals: demixing N sources from 2 mixtures

paper_content:

We present a novel method for blind separation of any number of sources using only two mixtures. The method applies when sources are (W-)disjoint orthogonal, that is, when the supports of the (windowed) Fourier transform of any two signals in the mixture are disjoint sets. We show that, for anechoic mixtures of attenuated and delayed sources, the method allows one to estimate the mixing parameters by clustering ratios of the time-frequency representations of the mixtures. The estimates of the mixing parameters are then used to partition the time-frequency representation of one mixture to recover the original sources. The technique is valid even in the case when the number of sources is larger than the number of mixtures. The general results are verified on both speech and wireless signals.
---
paper_title: Underdetermined blind source separation using sparse representations

paper_content:

The scope of this work is the separation of N sources from M linear mixtures when the underlying system is underdetermined, that is, when Mi N. If the input distribution is sparse the mixing matrix can be estimated either by external optimization or by clustering and, given the mixing matrix, a minimal l1 norm representation of the sources can be obtained by solving a low-dimensional linear programming problem for each of the data points. Yet, when the signals per se do not satisfy this assumption, sparsity can still be achieved by realizing the separation in a sparser transformed domain. The approach is illustrated here for M = 2. In this case we estimate both the number of sources and the mixing matrix by the maxima of a potential function along the circle of unit length, and we obtain the minimal l1 norm representation of each data point by a linear combination of the pair of basis vectors that enclose it. Several experiments with music and speech signals show that their time-domain representation is not sparse enough. Yet, excellent results were obtained using their short-time Fourier transform, including the separation of up to six sources from two mixtures. ? 2001 Elsevier Science B.V. All rights reserved.
---
paper_title: REAL-TIME BINAURAL BLIND SOURCE SEPARATION

paper_content:

Binaural blind source separation algorithm for noisy mixtures is proposed. We consider ambient background noise signals and nonstationary target source signals. The proposed blind source separation combines signal estimation from noisy observations with source identication through mixing parameter estimation. The sparseness property of target source signals enables the proposed noisy, underdetermined, binaural blind source separation. A minimum mean square error estimator in frequency domain is implemented to estimate the signal spectra from noisy observations. The K-means clustering algorithm is utilized to identify the sources. With the help of calculating the signal absence probability for each frequency bin, noises are effectively eliminated from the target source signals and the mixing parameter estimation becomes more accurate in noisy environments.
---
paper_title: Blind separation of speech mixtures via time-frequency masking

paper_content:

Binary time-frequency masks are powerful tools for the separation of sources from a single mixture. Perfect demixing via binary time-frequency masks is possible provided the time-frequency representations of the sources do not overlap: a condition we call W-disjoint orthogonality. We introduce here the concept of approximate W-disjoint orthogonality and present experimental results demonstrating the level of approximate W-disjoint orthogonality of speech in mixtures of various orders. The results demonstrate that there exist ideal binary time-frequency masks that can separate several speech signals from one mixture. While determining these masks blindly from just one mixture is an open problem, we show that we can approximate the ideal masks in the case where two anechoic mixtures are provided. Motivated by the maximum likelihood mixing parameter estimators, we define a power weighted two-dimensional (2-D) histogram constructed from the ratio of the time-frequency representations of the mixtures that is shown to have one peak for each source with peak location corresponding to the relative attenuation and delay mixing parameters. The histogram is used to create time-frequency masks that partition one of the mixtures into the original sources. Experimental results on speech mixtures verify the technique. Example demixing results can be found online at http://alum.mit.edu/www/rickard/bss.html.
---
paper_title: Real-Time Time-Frequency Based Blind Source Separation

paper_content:

We present a real-time version of the DUET algorithm for the blind separation of any number of sources using only two mixtures. The method applies when sources are Wdisjoint orthogonal, that is, when the supports of the windowed Fourier transform of any two signals in the mixture are disjoint sets, an assumption which is justified in the Appendix. The online algorithm is a Maximum Likelihood (ML) based gradient search method that is used to track the mixing parameters. The estimates of the mixing parameters are then used to partition the time-frequency representation of the mixtures to recover the original sources. The technique is valid even in the case when the number of sources is larger than the number of mixtures. The method was tested on mixtures generated from different voices and noises recorded from varying angles in both anechoic and echoic rooms. In total, over 1500 mixtures were tested. The average SNR gain of the demixing was 15 dB for anechoic room mixtures and 5 dB for echoic office mixtures. The algorithm runs 5 times faster than real time on a 750MHz laptop computer. Sample sound files can be found here:
---
paper_title: A Geometric Algorithm for Overcomplete Linear ICA

paper_content:

Abstract Geometric algorithms for linear square independent component analysis (ICA) have recently received some attention due to their pictorial description and their relative ease of implementation. The geometric approach to ICA was proposed first by Puntonet and Prieto (Neural Process. Lett. 2 (1995), Signal Processing 46 (1995) 267) in order to separate linear mixtures. We generalize these algorithms to overcomplete cases with more sources than sensors. With geometric ICA we get an efficient method for the matrix-recovery step in the framework of a two-step approach to the source separation problem. The second step—source-recovery—uses a maximum-likelihood approach. There we prove that the shortest-path algorithm as proposed by Bofill and Zibulevsky (in: P. Pajunen, J. Karhunen (Eds.), Independent Component Analysis and Blind Signal Separation (Proceedings of ICA’2000), 2000, pp. 87–92) indeed solves the maximum-likelihood conditions.
---
paper_title: Hard-LOST: Modified k-Means for Oriented Lines

paper_content:

Robust clustering of data into linear subspaces is a ::: common problem. Here we treat clustering into one-dimensional ::: subspaces that cross the origin. This problem arises in blind source ::: separation, where the subspaces correspond directly to columns of a ::: mixing matrix. We present an algorithm that identifies these subspaces ::: using a modified k-means procedure, where line orientations ::: and distances from a line replace the cluster centres and distance ::: from cluster centres of conventional k-means. This method, combined ::: with a transformation into a sparse domain and an L1-norm ::: optimisation, constitutes a blind source separation algorithm for the ::: under-determined case.
---
paper_title: On the approximate W-disjoint orthogonality of speech

paper_content:

It is possible to blindly separate an arbitrary number of sources given just two anechoic mixtures provided the time-frequency representations of the sources do not overlap, a condition which we call W-disjoint orthogonality. We define a power weighted two-dimensional histogram constructed from the ratio of the time-frequency representations of the mixtures which is shown to have one peak for each source with: peak location corresponding to the relative amplitude and delay mixing parameters. All of the time-frequency points which yield estimates in a given peak are exactly all the non-zero magnitude components of one of the sources. We introduce the concept of approximate W-disjoint orthogonality, present experimental results demonstrating the level of approximate W-disjoint orthogonality of speech in mixtures of various order, and show that even with imperfect W-disjoint orthogonality the histogram can be used to determine the mixing parameters and separate sources. Example demixing results can be found online: http://www.princeton.edu/∼srickard/bss.html
---
paper_title: An Information-Theoretic Analysis of Hard and Soft Assignment Methods for Clustering

paper_content:

Assignment methods are at the heart of many algorithms for unsupervised learning and clustering -- in particular, the well-known K-means and Expectation-Maximizatian (EM) algorithms. In this work, we study several different methods of assignment, including the "hard" assignments used by K-means and the "soft" assignments used by EM. While it is known that K-means minimizes the distortion on the data and EM maximizes the likelihood, little is known about the systematic differences of behavior between the two algorithms. Here we shed light on these differences via an information-theoretic analysis. The cornerstone of our results is a simple decomposition of the expected distortion, showing that K-means (and its extension for inferring general parametric densities from unlabeled sample data) must implicitly manage a trade-off between how similar the data assigned to each cluster are, and how the data are balanced among the clusters. How well the data are balanced is measured by the entropy of the partition defined by the hard assignments. In addition to letting us predict and verify systematic differences between K-means and EM on specific examples, the decomposition allows us to give a rather general argument showing that K-means will consistently find densities with less "overlap" than EM. We also study a third natural assignment method that we call posterior assignment, that is close in spirit to the soft assignments of EM, but leads to a surprisingly different algorithm.
---
paper_title: Maximum likelihood from incomplete data via the EM algorithm

paper_content:

Vibratory power unit for vibrating conveyers and screens comprising an asynchronous polyphase motor, at least one pair of associated unbalanced masses disposed on the shaft of said motor, with the first mass of a pair of said unbalanced masses being rigidly fastened to said shaft and with said second mass of said pair being movably arranged relative to said first mass, means for controlling and regulating the conveying rate during conveyer operation by varying the rotational speed of said motor between predetermined minimum and maximum values, said second mass being movably outwardly by centrifugal force against the pressure of spring means, said spring means being prestressed in such a manner that said second mass is, at rotational motor speeds lower than said minimum speed, held in its initial position, and at motor speeds between said lower and upper values in positions which are radially offset with respect to the axis of said motor to an extent depending on the value of said rotational motor speed.
---
paper_title: Clustering approach to square and non-square blind source separation

paper_content:

Recently, a number of heuristic techniques, mostly based on topographic maps, have been introduced in order to overcome (some) of the limitations of the blind source separation (BSS) algorithms that are rooted in the theory of independent component analysis. Here, we introduce a new heuristic that relies on the tendency of the mixture samples to cluster around the source directions in mixture space. We use linear mixtures of speech signals and consider BSS problems with not only square but also non-square mixing matrices (less mixtures than sources).
---
paper_title: Underdetermined blind source separation using sparse representations

paper_content:

The scope of this work is the separation of N sources from M linear mixtures when the underlying system is underdetermined, that is, when Mi N. If the input distribution is sparse the mixing matrix can be estimated either by external optimization or by clustering and, given the mixing matrix, a minimal l1 norm representation of the sources can be obtained by solving a low-dimensional linear programming problem for each of the data points. Yet, when the signals per se do not satisfy this assumption, sparsity can still be achieved by realizing the separation in a sparser transformed domain. The approach is illustrated here for M = 2. In this case we estimate both the number of sources and the mixing matrix by the maxima of a potential function along the circle of unit length, and we obtain the minimal l1 norm representation of each data point by a linear combination of the pair of basis vectors that enclose it. Several experiments with music and speech signals show that their time-domain representation is not sparse enough. Yet, excellent results were obtained using their short-time Fourier transform, including the separation of up to six sources from two mixtures. ? 2001 Elsevier Science B.V. All rights reserved.
---
paper_title: Kernel-Based Equiprobabilistic Topographic Map Formation

paper_content:

We introduce a new unsupervised competitive learning rule, the kernel-based maximum entropy learning rule (kMER), which performs equiprobabilistic topographic map formation in regular, fixed-topology lattices, for use with nonparametric density estimation as well as nonparametric regression analysis. The receptive fields of the formal neurons are overlapping radially symmetric kernels, compatible with radial basis functions (RBFs); but unlike other learning schemes, the radii of these kernels do not have to be chosen in an ad hoc manner: the radii are adapted to the local input density, together with the weight vectors that define the kernel centers, so as to produce maps of which the neurons have an equal probability to be active (equiprobabilistic maps). Both an “online” and a “batch” version of the learning rule are introduced, which are applied to nonparametric density estimation and regression, respectively. The application envisaged is blind source separation (BSS) from nonlinear, noisy mixtures.
---
paper_title: Sound Source Separation: Azimuth Discrimination and Resynthesis

paper_content:

In this paper we present a novel sound source separation algorithm which requires no prior knowledge, no learning, assisted or otherwise, and performs the task of separation based purely on azimuth discrimination within the stereo field. The algorithm exploits the use of the pan pot as a means to achieve image localisation within stereophonic recordings. As such, only an interaural intensity difference exists between left and right channels for a single source. We use gain scaling and phase cancellation techniques to expose frequency dependent nulls across the azimuth domain, from which source separation and resynthesis is carried out. We present results obtained from real recordings, and show that for musical recordings, the algorithm improves upon the output quality of current source separation schemes.
---
paper_title: Feature extraction approach to blind source separation

paper_content:

Local independent component analysis is formulated as a task involving the extraction of local geometric structure in the joint distribution. Because the geometrical structure of statistical independence is not well captured by statistical descriptions such as moments and cumulants, we use feature detection tools from image analysis to locate the local independent component coordinate system. The resulting approach to source separation can be implemented in real time using conventional image analysis hardware. The generality of this approach is demonstrated by blind source separation of multi-modal sources, and the pseudo-separation of three sources from two mixtures.
---
paper_title: Soft-LOST: EM on a Mixture of Oriented Lines

paper_content:

Robust clustering of data into overlapping linear subspaces is a common problem. Here we consider one-dimensional subspaces that cross the origin. This problem arises in blind source separation, where the subspaces correspond directly to columns of a mixing matrix. We present an algorithm that identifies these subspaces using an EM procedure, where the E-step calculates posterior probabilities assigning data points to lines and M-step repositions the lines to match the points assigned to them. This method, combined with a transformation into a sparse domain and an L1-norm optimisation, constitutes a blind source separation algorithm for the under-determined case.
---
paper_title: Eigen-structure of the fourth-order cumulant tensor with application to the blind source separation problem

paper_content:

Higher-order multivariate statistics are addressed. A theoretical discussion is followed by an original application. A special (index-free) tensor formalism to express fourth-order multivariate statistics is proposed. A quadricovariance tensor which contains the fourth-order joint cumulants is defined. In this formalism, the quadricovariance tensor and its eigen-matrices are natural fourth-order generalizations of second-order covariance and eigenvectors, allowing direct extension of many standard second-order method to fourth-order. The idea of an eigen-matrix is then proposed as a solution to the blind source separation problem. The task is to separate a mixture of N independent non-Gaussian signals received on an array of sensors when no information is available about propagation conditions or array geometry (blind situation). This can be achieved only by resorting to higher-order information. Quadricovariance eigen-matrices give a direct solution to this problem. >
---
paper_title: Networks for the separation of sources that are superimposed and delayed

paper_content:

We have created new networks to unmix signals which have been mixed either with time delays or via filtering. We first show that a subset of the Herault-Jutten learning rules fulfills a principle of minimum output power. We then apply this principle to extensions of the Herault-Jutten network which have delays in the feedback path. Our networks perform well on real speech and music signals that have been mixed using time delays or filtering.
---
paper_title: Adaptive noise cancelling: Principles and applications

paper_content:

This paper describes the concept of adaptive noise cancelling, an alternative method of estimating signals corrupted by additive noise or interference. The method uses a "primary" input containing the corrupted signal and a "reference" input containing noise correlated in some unknown way with the primary noise. The reference input is adaptively filtered and subtracted from the primary input to obtain the signal estimate. Adaptive filtering before subtraction allows the treatment of inputs that are deterministic or stochastic, stationary or time variable. Wiener solutions are developed to describe asymptotic adaptive performance and output signal-to-noise ratio for stationary stochastic inputs, including single and multiple reference inputs. These solutions show that when the reference input is free of signal and certain other conditions are met noise in the primary input can be essentiany eliminated without signal distortion. It is further shown that in treating periodic interference the adaptive noise canceller acts as a notch filter with narrow bandwidth, infinite null, and the capability of tracking the exact frequency of the interference; in this case the canceller behaves as a linear, time-invariant system, with the adaptive filter converging on a dynamic rather than a static solution. Experimental results are presented that illustrate the usefulness of the adaptive noise cancelling technique in a variety of practical applications. These applications include the cancelling of various forms of periodic interference in electrocardiography, the cancelling of periodic interference in speech signals, and the cancelling of broad-band interference in the side-lobes of an antenna array. In further experiments it is shown that a sine wave and Gaussian noise can be separated by using a reference input that is a delayed version of the primary input. Suggested applications include the elimination of tape hum or turntable rumble during the playback of recorded broad-band signals and the automatic detection of very-low-level periodic signals masked by broad-band noise.
---
paper_title: Blind Signal Separation: Statistical Principles

paper_content:

Blind signal separation (BSS) and independent component analysis (ICA) are emerging techniques of array processing and data analysis that aim to recover unobserved signals or "sources" from observed mixtures (typically, the output of an array of sensors), exploiting only the assumption of mutual independence between the signals. The weakness of the assumptions makes it a powerful approach, but it requires us to venture beyond familiar second order statistics, The objectives of this paper are to review some of the approaches that have been developed to address this problem, to illustrate how they stem from basic principles, and to show how they relate to each other.
---
paper_title: An Information-Maximization Approach to Blind Separation and Blind Deconvolution

paper_content:

We derive a new self-organizing learning algorithm that maximizes the information transferred in a network of nonlinear units. The algorithm does not assume any knowledge of the input distributions, and is defined here for the zero-noise limit. Under these conditions, information maximization has extra properties not found in the linear case (Linsker 1989). The nonlinearities in the transfer function are able to pick up higher-order moments of the input distributions and perform something akin to true redundancy reduction between units in the output representation. This enables the network to separate statistically independent components in the inputs: a higher-order generalization of principal components analysis. We apply the network to the source separation (or cocktail party) problem, successfully separating unknown mixtures of up to 10 speakers. We also show that a variant on the network architecture is able to perform blind deconvolution (cancellation of unknown echoes and reverberation in a speech signal). Finally, we derive dependencies of information transfer on time delays. We suggest that information maximization provides a unifying framework for problems in "blind" signal processing.
---
paper_title: source separation via generalized eigenvalue decomposition

paper_content:

In this short note we highlight the fact that linear blind source separation can be formulated as a generalized eigenvalue decomposition under the assumptions of non-Gaussian, non-stationary, or non-white independent sources. The solution for the unmixing matrix is given by the generalized eigenvectors that simultaneously diagonalize the covariance matrix of the observations and an additional symmetric matrix whose form depends upon the particular assumptions. The method critically determines the mixture coefficients and is therefore not robust to estimation errors. However it provides a rather general and unified solution that summarizes the conditions for successful blind source separation. To demonstrate the method, which can be implemented in two lines of matlab code, we present results for artificial mixtures of speech and real mixtures of electroencephalography (EEG) data, showing that the same sources are recovered under the various assumptions.
---
paper_title: Independent component analysis , A new concept ? *

paper_content:

Abstract The independent component analysis (ICA) of a random vector consists of searching for a linear transformation that minimizes the statistical dependence between its components. In order to define suitable search criteria, the expansion of mutual information is utilized as a function of cumulants of increasing orders. An efficient algorithm is proposed, which allows the computation of the ICA of a data matrix within a polynomial time. The concept of ICA may actually be seen as an extension of the principal component analysis (PCA), which can only impose independence up to the second order and, consequently, defines directions that are orthogonal. Potential applications of ICA include data analysis and compression, Bayesian detection, localization of sources, and blind identification and deconvolution.
---
paper_title: A fast fixed - point algorithm for independent component analysis

paper_content:

An apparatus for hydrolytic degradation of plastics in which plastic material is deposited into a tubular housing via a feed hopper. An elongated screw shaft has a first section in the form of a high pitch screw thread disposed below the feed hopper to receive and advance the material to a second section. The second section of the screw shaft is in the form of a lower pitch thread for compressing the plastic material and transferring it to a longer, third section in the form of kneading discs, from which material passes through an outlet nozzle section to a cyclone separator where trapped gases and liquid may be withdrawn. The tubular housing is vented upstream of the feed hopper and a water inlet pipe is disposed adjacent to the second section of the screw shaft, downstream of the feed hopper. The outlet nozzle section is provided with pressure measuring and regulating means and a liquid level measuring and regulating device.
---
paper_title: Underdetermined Blind Source Separation Using A Probabilistic Source Sparsity Model

paper_content:

Blind source separation consists of recovering n source signals from m measurements that are an unknown function of the sources. In solving the underdetermined (m < n) linear problem three stages can be identified: to represent the signals in an appropriate domain, to estimate the mixing matrix, and to invert the linear problem to estimate the sources. As a consequence of having more degrees of freedom than constraints, the inverse problem has an infinite number of solutions. To choose the “best” solution, additional constraints have to be imposed on the basis of some performance criterion or previous knowledge. In this communication we present a method that choose the “best” demixing matrix in a sample by sample basis by using some previous knowledge of the statistics of the sources. The behaviour of the estimator is compared to the global pseudo inverse approach and with other local heuristic methods by means of Montecarlo simulations.
---
paper_title: Atomic Decomposition by Basis Pursuit

paper_content:

The time-frequency and time-scale communities have recently developed a large number of overcomplete waveform dictionaries --- stationary wavelets, wavelet packets, cosine packets, chirplets, and warplets, to name a few. Decomposition into overcomplete systems is not unique, and several methods for decomposition have been proposed, including the method of frames (MOF), Matching pursuit (MP), and, for special dictionaries, the best orthogonal basis (BOB). ::: Basis Pursuit (BP) is a principle for decomposing a signal into an "optimal" superposition of dictionary elements, where optimal means having the smallest l1 norm of coefficients among all such decompositions. We give examples exhibiting several advantages over MOF, MP, and BOB, including better sparsity and superresolution. BP has interesting relations to ideas in areas as diverse as ill-posed problems, in abstract harmonic analysis, total variation denoising, and multiscale edge denoising. ::: BP in highly overcomplete dictionaries leads to large-scale optimization problems. With signals of length 8192 and a wavelet packet dictionary, one gets an equivalent linear program of size 8192 by 212,992. Such problems can be attacked successfully only because of recent advances in linear programming by interior-point methods. We obtain reasonable success with a primal-dual logarithmic barrier method and conjugate-gradient solver.
---
paper_title: Underdetermined blind source separation in a time-varying environment

paper_content:

The problem of estimating n source signals from m measurements that are an unknown mixture of the sources is known as blind source separation. In the underdetermined —less measurements than sources— linear case, the solution process can be conveniently divided in three stages: represent the signals in a sparse domain, find the mixing matrix, and estimate the sources. In this paper we adhere to that approach and parametrize the performance of these stages as a function of the sparsity of the signals. To find the mixing matrix and track its variations in the dynamic case a nonparametric maximum-likelihood approach based on Parzen windowing is presented. To invert the underdetermined linear problem we present an estimator that chooses the “best” demixing matrix in a sample by sample basis by using some previous knowledge of the statistics of the sources. The results are validated by Montecarlo simulations.
---
paper_title: On the Minimum ℓ 1 -Norm Signal Recovery in Underdetermined Source Separation

paper_content:

This paper studied the minimum l1-norm signal recovery in underdetermined source separation, which is a problem of separating n sources blindly from m linear mixtures for n>m. Based on our previous result of submatrix representation and decision regions, we describe the property of the minimum l1-norm sequence from the viewpoint of source separation, and discuss how to construct it geometrically from the observed sequence and the mixing matrix, and the unstability for a perturbation of mixing matrix.
---
paper_title: Feature extraction approach to blind source separation

paper_content:

Local independent component analysis is formulated as a task involving the extraction of local geometric structure in the joint distribution. Because the geometrical structure of statistical independence is not well captured by statistical descriptions such as moments and cumulants, we use feature detection tools from image analysis to locate the local independent component coordinate system. The resulting approach to source separation can be implemented in real time using conventional image analysis hardware. The generality of this approach is demonstrated by blind source separation of multi-modal sources, and the pseudo-separation of three sources from two mixtures.
---
paper_title: On the approximate W-disjoint orthogonality of speech

paper_content:

It is possible to blindly separate an arbitrary number of sources given just two anechoic mixtures provided the time-frequency representations of the sources do not overlap, a condition which we call W-disjoint orthogonality. We define a power weighted two-dimensional histogram constructed from the ratio of the time-frequency representations of the mixtures which is shown to have one peak for each source with: peak location corresponding to the relative amplitude and delay mixing parameters. All of the time-frequency points which yield estimates in a given peak are exactly all the non-zero magnitude components of one of the sources. We introduce the concept of approximate W-disjoint orthogonality, present experimental results demonstrating the level of approximate W-disjoint orthogonality of speech in mixtures of various order, and show that even with imperfect W-disjoint orthogonality the histogram can be used to determine the mixing parameters and separate sources. Example demixing results can be found online: http://www.princeton.edu/∼srickard/bss.html
---
paper_title: Blind separation of disjoint orthogonal signals: demixing N sources from 2 mixtures

paper_content:

We present a novel method for blind separation of any number of sources using only two mixtures. The method applies when sources are (W-)disjoint orthogonal, that is, when the supports of the (windowed) Fourier transform of any two signals in the mixture are disjoint sets. We show that, for anechoic mixtures of attenuated and delayed sources, the method allows one to estimate the mixing parameters by clustering ratios of the time-frequency representations of the mixtures. The estimates of the mixing parameters are then used to partition the time-frequency representation of one mixture to recover the original sources. The technique is valid even in the case when the number of sources is larger than the number of mixtures. The general results are verified on both speech and wireless signals.
---
paper_title: Underdetermined blind separation of delayed sound sources in the frequency domain

paper_content:

Abstract This paper focuses on the blind separation of n sources from m mixtures when the underlying system is underdetermined (i.e., when m n ), as applied to mixtures with only attenuations and delays (i.e., no reverberation). The separation is realized in the frequency domain where, at least for speech and music signals, the representation is sparse . The procedure is organized in three stages. First, the matrix of relative attenuations is inferred by angular clustering of the magnitude of the input, yielding a coarse partition of the data into their nearest sources. Second, for each partition, the differential delay is inferred by shifting the sensor channels and scattering the real and imaginary components until the cluster reappears. And third, given the attenuation and delay matrices, the sources are inferred at each frequency bin by assuming that the magnitude of the spectral coefficients is Laplacian distributed, which leads to the minimization of the sum of magnitudes, subject to the mixing equations. The resulting problem is an instance of second-order cone programming. The approach is experimentally illustrated for m =2 sensors and n up to six speech and music sources, using synthetic mixtures simulating real acoustic scenarios without reverberation.
---
paper_title: Blind separation of speech mixtures via time-frequency masking

paper_content:

Binary time-frequency masks are powerful tools for the separation of sources from a single mixture. Perfect demixing via binary time-frequency masks is possible provided the time-frequency representations of the sources do not overlap: a condition we call W-disjoint orthogonality. We introduce here the concept of approximate W-disjoint orthogonality and present experimental results demonstrating the level of approximate W-disjoint orthogonality of speech in mixtures of various orders. The results demonstrate that there exist ideal binary time-frequency masks that can separate several speech signals from one mixture. While determining these masks blindly from just one mixture is an open problem, we show that we can approximate the ideal masks in the case where two anechoic mixtures are provided. Motivated by the maximum likelihood mixing parameter estimators, we define a power weighted two-dimensional (2-D) histogram constructed from the ratio of the time-frequency representations of the mixtures that is shown to have one peak for each source with peak location corresponding to the relative attenuation and delay mixing parameters. The histogram is used to create time-frequency masks that partition one of the mixtures into the original sources. Experimental results on speech mixtures verify the technique. Example demixing results can be found online at http://alum.mit.edu/www/rickard/bss.html.
---
paper_title: Second order blind equalization in multiple input multiple output FIR systems: a weighted least squares approach

paper_content:

Multipath propagation appears to be a typical limitation in mobile digital communications where it leads to severe intersymbol interference. The classical techniques to overcome this problem use either periodically sent training sequences or blind approaches. This paper addresses the blind identification of FIR multiple input multiple output (MIMO) transfer functions in the case where the number of inputs is strictly less than the number of outputs. We consider a second order identification providing signals extraction up to a regular instantaneous mixture matrix. A novel method of second order channel estimation is presented. Basically exploiting the simultaneous MA and AR nature of the observations, it displays however a considerable improvement as compared to the original linear prediction approach. Performance study and comparison with the existing approach is provided by computer simulations.
---
paper_title: Blind separation of convolutive mixtures using second and fourth order moments

paper_content:

This paper deals with the problem of blind identification of p-non Gaussian inputs, q-outputs AR systems in the special case where p<q. In this case, the identification problem is degenerated, therefore as the classical Levinson (Robinson) algorithm cannot be applied, we use the Inouye's (1983) method. As this procedure assumes that the AR model is normalized, it is necessary to split the problem in two parts: first, we estimate the convolutive mixture by means of linear prediction, and second, we estimate the instantaneous mixture. The first one requires second order moments and the second one, high order statistics. Numerical simulations are presented to show the influence of the conditioning of the instantaneous mixture matrix in the identification problem in presence of white Gaussian noise.
---
paper_title: Auditory Scene Analysis: The Perceptual Organization of Sound

paper_content:

Auditory Scene Analysis addresses the problem of hearing complex auditory environments, using a series of creative analogies to describe the process required of the human auditory system as it analyzes mixtures of sounds to recover descriptions of individual sounds. In a unified and comprehensive way, Bregman establishes a theoretical framework that integrates his findings with an unusually wide range of previous research in psychoacoustics, speech perception, music theory and composition, and computer modeling.
---
paper_title: Non-negative sparse coding

paper_content:

Non-negative sparse coding is a method for decomposing multivariate data into non-negative sparse components. We briefly describe the motivation behind this type of data representation and its relation to standard sparse coding and non-negative matrix factorization. We then give a simple yet efficient multiplicative algorithm for finding the optimal values of the hidden components. In addition, we show how the basis vectors can be learned from the observed data. Simulations demonstrate the effectiveness of the proposed method.
---
paper_title: When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts?

paper_content:

We interpret non-negative matrix factorization geometrically, as the problem of finding a simplicial cone which contains a cloud of data points and which is contained in the positive orthant. We show that under certain conditions, basically requiring that some of the data are spread across the faces of the positive orthant, there is a unique such simplicial cone. We give examples of synthetic image articulation databases which obey these conditions; these require separated support and factorial sampling. For such databases there is a generative model in terms of 'parts' and NMF correctly identifies the 'parts'. We show that our theoretical results are predictive of the performance of published NMF code, by running the published algorithms on one of our synthetic image articulation databases.
---
paper_title: Algorithms for Non-negative Matrix Factorization

paper_content:

Non-negative matrix factorization (NMF) has previously been shown to be a useful decomposition for multivariate data. Two different multiplicative algorithms for NMF are analyzed. They differ only slightly in the multiplicative factor used in the update rules. One algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence. The monotonic convergence of both algorithms can be proven using an auxiliary function analogous to that used for proving convergence of the Expectation-Maximization algorithm. The algorithms can also be interpreted as diagonally rescaled gradient descent, where the rescaling factor is optimally chosen to ensure convergence.
---
paper_title: Unmixing Hyperspectral Data

paper_content:

In hyperspectral imagery one pixel typically consists of a mixture of the reflectance spectra of several materials, where the mixture coefficients correspond to the abundances of the constituting materials. We assume linear combinations of reflectance spectra with some additive normal sensor noise and derive a probabilistic MAP framework for analyzing hyperspectral data. As the material reflectance characteristics are not know a priori, we face the problem of unsupervised linear unmixing. The incorporation of different prior information (e.g. positivity and normalization of the abundances) naturally leads to a family of interesting algorithms, for example in the noise-free case yielding an algorithm that can be understood as constrained independent component analysis (ICA). Simulations underline the usefulness of our theory.
---
paper_title: Non-negative Matrix Factor Deconvolution; Extraction of Multiple Sound Sources from Monophonic Inputs

paper_content:

In this paper we present an extension to the Non-Negative Matrix Factorization algorithm which is capable of identifying components with temporal structure. We demonstrate the use of this algorithm in the magnitude spectrum domain, where we employ it to perform extraction of multiple sound objects from a single channel auditory scene.
---
paper_title: Independent Component Analysis: Principles and Practice

paper_content:

1. Introduction Stephen Roberts and Richard Everson 2. Fast ICA by a fixed-point algorithm that maximizes non-Gaussianity Aapo Hyvarinen 3. ICA, graphical models and variational methods Hagai Attias 4. Nonlinear independent component analysis Juha Karhunen 5. Separation of non-stationary natural signals Lucas Parra and Clay Spence 6. Separation of non-stationary sources: algorithms and performance Jean-Francois Cardoso and Dinh-Tuan Pham 7. Blind source separation by sparse decomposition in a signal dictionary Michael Zibulevsky, Barak Pearlmutter, Pau Bofill and Pavel Kisilev 8. Ensemble learning for blind source separation James Miskin and David MacKay 9. Image processing methods using ICA mixture models Te-Won Lee and Michael S. Lewicki 10. Latent class and trait models for data classification and visualisation Mark Girolami 11. Particle filters for non-stationary ICA Richard Everson and Stephen Roberts 12. ICA: model order selection and dynamic source models William Penny, Stephen Roberts and Richard Everson.
---
paper_title: Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications

paper_content:

From the Publisher: ::: With solid theoretical foundations and numerous potential applications, Blind Signal Processing (BSP) is one of the hottest emerging areas in Signal Processing. This volume unifies and extends the theories of adaptive blind signal and image processing and provides practical and efficient algorithms for blind source separation, Independent, Principal, Minor Component Analysis, and Multichannel Blind Deconvolution (MBD) and Equalization. Containing over 1400 references and mathematical expressions Adaptive Blind Signal and Image Processing delivers an unprecedented collection of useful techniques for adaptive blind signal/image separation, extraction, decomposition and filtering of multi-variable signals and data. ::: Offers a broad coverage of blind signal processing techniques and algorithms both from a theoretical and practical point of viewPresents more than 50 simple algorithms that can be easily modified to suit the reader's specific real world problemsProvides a guide to fundamental mathematics of multi-input, multi-output and multi-sensory systemsIncludes illustrative worked examples, computer simulations, tables, detailed graphs and conceptual models within self contained chapters to assist self studyAccompanying CD-ROM features an electronic, interactive version of the book with fully coloured figures and text. C and MATLAB user-friendly software packages are also provided ::: MATLAB is a registered trademark of The MathWorks, Inc. ::: By providing a detailed introduction to BSP, as well as presenting new results and recent developments, this informative and inspiring work will appeal to researchers, postgraduate students, engineers and scientists working in biomedical engineering, communications, electronics, computer science, optimisations, finance, geophysics and neural networks.
---

Title: Survey of Sparse and Non-Sparse Methods in Source Separation
Section 1: Introduction
Description 1: Write about the basics of source separation and blind source separation, including the dimensionality and environmental assumptions involved, and the evolution of BSS techniques, focusing on the contribution of sparse methods.

Section 2: Mixing Models
Description 2: Describe the various generative models for blind source separation, defining the differences between instantaneous, anechoic, and echoic mixing processes.

Section 3: Instantaneous Mixing
Description 3: Discuss the specifics of instantaneous mixing, including simple examples, scatter plots, and how traditional and sparse approaches differ in this context.

Section 4: Anechoic Mixing
Description 4: Explain anechoic mixing, including its complexities, assumptions, relevant algorithms, and transformations used for separating sources in anechoic environments.

Section 5: Echoic Mixing
Description 5: Explore echoic mixing models, the use of FIR filters, and techniques for estimating and inverting the mixing process for source separation in echoic scenarios.

Section 6: Mixing Parameters Estimation
Description 6: Detail the first step in the staged algorithm approach, estimating the mixing matrix, and compare sparse and non-sparse methods, including clustering techniques and eigenvector decomposition.

Section 7: Sparse Methods
Description 7: Provide a detailed overview of methods leveraging sparsity, techniques for identifying line orientations, and specific algorithms like DUET and clustering-based approaches.

Section 8: Non-Sparse Methods
Description 8: Examine methods that do not assume sparsity, focusing on statistical properties of observations and techniques like BS-Infomax and SOBI algorithms for estimating the mixing process.

Section 9: Separation Techniques
Description 9: Present various separation techniques used after estimating the mixing matrix, including linear transformations, L1-norm minimization, Time-Frequency Masking, and more.

Section 10: Joint Estimation
Description 10: Discuss methods that concurrently estimate sources and mixing processes by defining the problem as an optimization problem and approaches such as Non-negative Matrix Factorisation.

Section 11: Non-Negative Matrix Factorisation
Description 11: Delve into the theory and application of NMF in source separation, including geometric interpretations, conditions for correctness, and extensions like Non-Negative Sparse Coding.

Section 12: NMF Algorithms
Description 12: Describe specific algorithms for NMF, detailing cost functions, constraints for ensuring solution accuracy, and practical applications in source separation.

Section 13: Further Reading
Description 13: Recommend books and resources that provide additional insights into ICA techniques, blind source separation, blind deconvolution, the cocktail party problem, and machine learning principles.

Section 14: Conclusion
Description 14: Summarize the challenges and advancements in blind source separation, the staged and joint estimation approaches, the benefits of the sparse assumption, and the potential of sparse representations in practical signal processing algorithms.