Machine learning in acoustics: a review.
---
paper_title: Some methods for classification and analysis of multivariate observations

paper_content:

The main purpose of this paper is to describe a process for partitioning an N-dimensional population into k sets on the basis of a sample. The process, which is called 'k-means,' appears to give partitions which are reasonably efficient in the sense of within-class variance. That is, if p is the probability mass function for the population, S = {S1, S2, * *, Sk} is a partition of EN, and ui, i = 1, 2, * , k, is the conditional mean of p over the set Si, then W2(S) = ff=ISi f z u42 dp(z) tends to be low for the partitions S generated by the method. We say 'tends to be low,' primarily because of intuitive considerations, corroborated to some extent by mathematical analysis and practical computational experience. Also, the k-means procedure is easily programmed and is computationally economical, so that it is feasible to process very large samples on a digital computer. Possible applications include methods for similarity grouping, nonlinear prediction, approximating multivariate distributions, and nonparametric tests for independence among several variables. In addition to suggesting practical classification methods, the study of k-means has proved to be theoretically interesting. The k-means concept represents a generalization of the ordinary sample mean, and one is naturally led to study the pertinent asymptotic behavior, the object being to establish some sort of law of large numbers for the k-means. This problem is sufficiently interesting, in fact, for us to devote a good portion of this paper to it. The k-means are defined in section 2.1, and the main results which have been obtained on the asymptotic behavior are given there. The rest of section 2 is devoted to the proofs of these results. Section 3 describes several specific possible applications, and reports some preliminary results from computer experiments conducted to explore the possibilities inherent in the k-means idea. The extension to general metric spaces is indicated briefly in section 4. The original point of departure for the work described here was a series of problems in optimal classification (MacQueen [9]) which represented special
---
paper_title: Source localization in an ocean waveguide using supervised machine learning

paper_content:

Source localization in ocean acoustics is posed as a machine learning problem in which data-driven methods learn source ranges directly from observed acoustic data. The pressure received by a vertical linear array is preprocessed by constructing a normalized sample covariance matrix and used as the input for three machine learning methods: feed-forward neural networks (FNN), support vector machines (SVM), and random forests (RF). The range estimation problem is solved both as a classification problem and as a regression problem by these three machine learning algorithms. The results of range estimation for the Noise09 experiment are compared for FNN, SVM, RF, and conventional matched-field processing and demonstrate the potential of machine learning for underwater source localization.
---
paper_title: Statistics of natural reverberation enable perceptual separation of sound and space

paper_content:

Abstract ::: In everyday listening, sound reaches our ears directly from a source as well as indirectly via reflections known as reverberation. Reverberation profoundly distorts the sound from a source, yet humans can both identify sound sources and distinguish environments from the resulting sound, via mechanisms that remain unclear. The core computational challenge is that the acoustic signatures of the source and environment are combined in a single signal received by the ear. Here we ask whether our recognition of sound sources and spaces reflects an ability to separate their effects and whether any such separation is enabled by statistical regularities of real-world reverberation. To first determine whether such statistical regularities exist, we measured impulse responses (IRs) of 271 spaces sampled from the distribution encountered by humans during daily life. The sampled spaces were diverse, but their IRs were tightly constrained, exhibiting exponential decay at frequency-dependent rates: Mid frequencies reverberated longest whereas higher and lower frequencies decayed more rapidly, presumably due to absorptive properties of materials and air. To test whether humans leverage these regularities, we manipulated IR decay characteristics in simulated reverberant audio. Listeners could discriminate sound sources and environments from these signals, but their abilities degraded when reverberation characteristics deviated from those of real-world environments. Subjectively, atypical IRs were mistaken for sound sources. The results suggest the brain separates sound into contributions from the source and the environment, constrained by a prior on natural reverberation. This separation process may contribute to robust recognition while providing information about spaces around us.
---
paper_title: Machine learning: Trends, perspectives, and prospects

paper_content:

Machine learning addresses the question of how to build computers that improve automatically through experience. It is one of today's most rapidly growing technical fields, lying at the intersection of computer science and statistics, and at the core of artificial intelligence and data science. Recent progress in machine learning has been driven both by the development of new learning algorithms and theory and by the ongoing explosion in the availability of online data and low-cost computation. The adoption of data-intensive machine-learning methods can be found throughout science, technology and commerce, leading to more evidence-based decision-making across many walks of life, including health care, manufacturing, education, financial modeling, policing, and marketing.
---
paper_title: Robust Ocean Acoustic Localization With Sparse Bayesian Learning

paper_content:

Matched field processing (MFP) compares the measures to the modeled pressure fields received at an array of sensors to localize a source in an ocean waveguide. Typically, there are only a few sources when compared to the number of candidate source locations or range-depth cells. We use sparse Bayesian learning (SBL) to learn a common sparsity profile corresponding to the location of present sources. SBL performance is compared to traditional processing in simulations and using experimental ocean acoustic data. Specifically, we localize a quiet source in the presence of a surface interferer in a shallow water environment. This multi-frequency scenario requires adaptive processing and includes modest environmental and sensor position mismatch in the MFP model. The noise process changes likely with time and is modeled as a non-stationary Gaussian process, meaning that the noise variance changes across snapshots. The adaptive SBL algorithm models the complex source amplitudes as random quantities, providing robustness to amplitude and phase errors in the model. This is demonstrated with experimental data, where SBL exhibits improved source localization performance when compared to the white noise gain constraint (–3 dB) and Bartlett processors.
---
paper_title: Parameter estimation using multifrequency range‐dependent acoustic data in shallow water

paper_content:

The estimation of all forward model parameters—geometric, geoacoustic, and ocean sound speed—by the inversion of acoustic field observations is considered. The data was taken at a mildly range‐dependent shallow water site in the Mediterranean Sea. The inversion is based on data from a vertical array and carried out using information at multiple frequencies. Global optimization using a directed Monte Carlo search based on genetic algorithms and the Bartlett objective function is used. All geometric parameters are well determined, a range‐dependent geoacoustic model is determined, and the ocean sound speed is estimated. Comparisons of the observed pressure field as a function of depth and the predicted field show good agreement. The use of observations at multiple frequencies provides considerable stability for the estimated parameters. Optimization of only geometric and geoacoustic parameters in a range‐independent environment is found to be satisfactory at the lower frequencies (165–175 Hz), but for the h...
---
paper_title: Deep learning

paper_content:

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
---
paper_title: Machine Learning: A Probabilistic Perspective

paper_content:

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.
---
paper_title: Sparse Modeling for Image and Vision Processing

paper_content:

In recent years, a large amount of multi-disciplinary research has been conducted on sparse models and their applications. In statistics and machine learning, the sparsity principle is used to perform model selection---that is, automatically selecting a simple model among a large collection of them. In signal processing, sparse coding consists of representing data with linear combinations of a few dictionary elements. Subsequently, the corresponding tools have been widely adopted by several scientific communities such as neuroscience, bioinformatics, or computer vision. The goal of this monograph is to offer a self-contained view of sparse modeling for visual recognition and image processing. More specifically, we focus on applications where the dictionary is learned and adapted to data, yielding a compact representation that has been successful in various contexts.
---
paper_title: Speech Processing in Modern Communication: Challenges and Perspectives

paper_content:

Modern communication devices, such as mobile phones, teleconferencing systems, VoIP, etc., are often used in noisy and reverberant environments. Therefore, signals picked up by the microphones from telecommunication devices contain not only the desired near-end speech signal, but also interferences such as the background noise, far-end echoes produced by the loudspeaker, and reverberations of the desired source. These interferences degrade the fidelity and intelligibility of the near-end speech in human-to-human telecommunications and decrease the performance of human-to-machine interfaces (i.e., automatic speech recognition systems). The proposed book deals with the fundamental challenges of speech processing in modern communication, including speech enhancement, interference suppression, acoustic echo cancellation, relative transfer function identification, source localization, dereverberation, and beamforming in reverberant environments. Enhancement of speech signals is necessary whenever the source signal is corrupted by noise. In highly non-stationary noise environments, noise transients, and interferences may be extremely annoying. Acoustic echo cancellation is used to eliminate the acoustic coupling between the loudspeaker and the microphone of a communication device. Identification of the relative transfer function between sensors in response to a desired speech signal enables to derive a reference noise signal for suppressing directional or coherent noise sources. Source localization, dereverberation, and beamforming in reverberant environments further enable to increase the intelligibility of the near-end speech signal.
---
paper_title: Some methods for classification and analysis of multivariate observations

paper_content:

The main purpose of this paper is to describe a process for partitioning an N-dimensional population into k sets on the basis of a sample. The process, which is called 'k-means,' appears to give partitions which are reasonably efficient in the sense of within-class variance. That is, if p is the probability mass function for the population, S = {S1, S2, * *, Sk} is a partition of EN, and ui, i = 1, 2, * , k, is the conditional mean of p over the set Si, then W2(S) = ff=ISi f z u42 dp(z) tends to be low for the partitions S generated by the method. We say 'tends to be low,' primarily because of intuitive considerations, corroborated to some extent by mathematical analysis and practical computational experience. Also, the k-means procedure is easily programmed and is computationally economical, so that it is feasible to process very large samples on a digital computer. Possible applications include methods for similarity grouping, nonlinear prediction, approximating multivariate distributions, and nonparametric tests for independence among several variables. In addition to suggesting practical classification methods, the study of k-means has proved to be theoretically interesting. The k-means concept represents a generalization of the ordinary sample mean, and one is naturally led to study the pertinent asymptotic behavior, the object being to establish some sort of law of large numbers for the k-means. This problem is sufficiently interesting, in fact, for us to devote a good portion of this paper to it. The k-means are defined in section 2.1, and the main results which have been obtained on the asymptotic behavior are given there. The rest of section 2 is devoted to the proofs of these results. Section 3 describes several specific possible applications, and reports some preliminary results from computer experiments conducted to explore the possibilities inherent in the k-means idea. The extension to general metric spaces is indicated briefly in section 4. The original point of departure for the work described here was a series of problems in optimal classification (MacQueen [9]) which represented special
---
paper_title: A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection

paper_content:

We review accuracy estimation methods and compare the two most common methods crossvalidation and bootstrap. Recent experimental results on artificial data and theoretical re cults in restricted settings have shown that for selecting a good classifier from a set of classifiers (model selection), ten-fold cross-validation may be better than the more expensive leaveone-out cross-validation. We report on a largescale experiment--over half a million runs of C4.5 and a Naive-Bayes algorithm--to estimate the effects of different parameters on these algrithms on real-world datasets. For crossvalidation we vary the number of folds and whether the folds are stratified or not, for bootstrap, we vary the number of bootstrap samples. Our results indicate that for real-word datasets similar to ours, The best method to use for model selection is ten fold stratified cross validation even if computation power allows using more folds.
---
paper_title: Bayesian Data Analysis

paper_content:

FUNDAMENTALS OF BAYESIAN INFERENCE Probability and Inference Single-Parameter Models Introduction to Multiparameter Models Asymptotics and Connections to Non-Bayesian Approaches Hierarchical Models FUNDAMENTALS OF BAYESIAN DATA ANALYSIS Model Checking Evaluating, Comparing, and Expanding Models Modeling Accounting for Data Collection Decision Analysis ADVANCED COMPUTATION Introduction to Bayesian Computation Basics of Markov Chain Simulation Computationally Efficient Markov Chain Simulation Modal and Distributional Approximations REGRESSION MODELS Introduction to Regression Models Hierarchical Linear Models Generalized Linear Models Models for Robust Inference Models for Missing Data NONLINEAR AND NONPARAMETRIC MODELS Parametric Nonlinear Models Basic Function Models Gaussian Process Models Finite Mixture Models Dirichlet Process Models APPENDICES A: Standard Probability Distributions B: Outline of Proofs of Asymptotic Theorems C: Computation in R and Stan Bibliographic Notes and Exercises appear at the end of each chapter.
---
paper_title: Source localization in an ocean waveguide using supervised machine learning

paper_content:

Source localization in ocean acoustics is posed as a machine learning problem in which data-driven methods learn source ranges directly from observed acoustic data. The pressure received by a vertical linear array is preprocessed by constructing a normalized sample covariance matrix and used as the input for three machine learning methods: feed-forward neural networks (FNN), support vector machines (SVM), and random forests (RF). The range estimation problem is solved both as a classification problem and as a regression problem by these three machine learning algorithms. The results of range estimation for the Noise09 experiment are compared for FNN, SVM, RF, and conventional matched-field processing and demonstrate the potential of machine learning for underwater source localization.
---
paper_title: Multipath Broadband Localization, Bathymetry, and Sediment Inversion

paper_content:

Transmission of linearly frequency modulated pulses generates receptions at a vertical line array that can be cross correlated with the source signal to provide estimates of the oceanic waveguide impulse response. For short ranges, distinct path arrivals can be identified including the direct, surface reflection, bottom reflection, and sediment reflection. Accurate estimation of arrival times of such paths is tightly related to successful inversion for source location and water column depth and sound speed and, subsequently, estimation of sediment sound speed and thickness. To achieve accurate estimation, particle filtering is applied to the received time series at 16 phones combined with a simple cross-correlation method. Using linearization, arrival time probability density functions are connected to the geometry and water column sound-speed parameters, providing point estimates as well as probability densities. These are then employed in sediment sound speed and thickness estimation. The results, obtained from the application of the method to data collected during the Seabed Characterization Experiment, are consistent with prior information on the site.
---
paper_title: Multi-frequency sparse Bayesian learning for robust matched field processing

paper_content:

The multi-snapshot, multi-frequency sparse Bayesian learning (SBL) processor is derived and its performance compared to the Bartlett, minimum variance distortionless response, and white noise constraint processors for the matched field processing application. The two-source model and data scenario of interest includes realistic mismatch implemented in the form of array tilt and data snapshots not exactly corresponding to the range-depth grid of the replica vectors. Results demonstrate that SBL behaves similar to an adaptive processor when localizing a weaker source in the presence of a stronger source, is robust to mismatch, and exhibits improved localization performance when compared to the other processors. Unlike the basis or matching pursuit methods, SBL automatically determines sparsity and its solution can be interpreted as an ambiguity surface. Because of its computational efficiency and performance, SBL is practical for applications requiring adaptive and robust processing.
---
paper_title: Probabilistic machine learning and artificial intelligence

paper_content:

How can a machine learn from experience? Probabilistic modelling provides a framework for understanding what learning is, and has therefore emerged as one of the principal theoretical and practical approaches for designing machines that learn from data acquired through experience. The probabilistic framework, which describes how to represent and manipulate uncertainty about models and predictions, has a central role in scientific data analysis, machine learning, robotics, cognitive science and artificial intelligence. This Review provides an introduction to this framework, and discusses some of the state-of-the-art advances in the field, namely, probabilistic programming, Bayesian optimization, data compression and automatic model discovery.
---
paper_title: Robust Ocean Acoustic Localization With Sparse Bayesian Learning

paper_content:

Matched field processing (MFP) compares the measures to the modeled pressure fields received at an array of sensors to localize a source in an ocean waveguide. Typically, there are only a few sources when compared to the number of candidate source locations or range-depth cells. We use sparse Bayesian learning (SBL) to learn a common sparsity profile corresponding to the location of present sources. SBL performance is compared to traditional processing in simulations and using experimental ocean acoustic data. Specifically, we localize a quiet source in the presence of a surface interferer in a shallow water environment. This multi-frequency scenario requires adaptive processing and includes modest environmental and sensor position mismatch in the MFP model. The noise process changes likely with time and is modeled as a non-stationary Gaussian process, meaning that the noise variance changes across snapshots. The adaptive SBL algorithm models the complex source amplitudes as random quantities, providing robustness to amplitude and phase errors in the model. This is demonstrated with experimental data, where SBL exhibits improved source localization performance when compared to the white noise gain constraint (–3 dB) and Bartlett processors.
---
paper_title: Sparse Bayesian learning with multiple dictionaries

paper_content:

Abstract Sparse Bayesian learning (SBL) has emerged as a fast and competitive method to perform sparse processing. The SBL algorithm, which is developed using a Bayesian framework, iteratively solves a non-convex optimization problem using fixed point updates. It provides comparable performance and is significantly faster than convex optimization techniques used in sparse processing. We propose a multi-dictionary SBL algorithm that simultaneously can process observations generated by different underlying dictionaries sharing the same sparsity profile. Two algorithms are proposed and corresponding fixed point update equations are derived. Noise variances are estimated using stochastic maximum likelihood. The multi-dictionary SBL has many practical applications. We demonstrate this using direction-of-arrival (DOA) estimation. The first example uses the proposed multi-dictionary SBL to process multi-frequency observations. We show how spatial aliasing can be avoided while processing multi-frequency observations using SBL. SWellEx-96 experimental data demonstrates qualitatively these advantages. In the second example we show how data corrupted with heteroscedastic noise can be processed using multi-dictionary SBL with data pre-whitening.
---
paper_title: Inverse Methods for Atmospheric Sounding: Theory and Practice

paper_content:

Remote sounding of the atmosphere has proved to be a fruitful method of obtaining global information about the atmospheres of the earth and planets. This book treats the inverse problem of remote sounding comprehensively, and discusses a wide range of retrieval methods for extracting atmospheric parameters of interest from the quantities such as thermal emission that can be measured remotely. Inverse theory is treated in depth from an estimation-theory point of view, but practical questions are also emphasized, for example designing observing systems to obtain the maximum quantity of information, efficient numerical implementation of algorithms for processing of large quantities of data, error analysis and approaches to the validation of the resulting retrievals, The book is targeted at both graduate students and working scientists.
---
paper_title: Parameter Estimation and Inverse Problems

paper_content:

"Parameter Estimation and Inverse Problems, 2/e" provides geoscience students and professionals with answers to common questions like how one can derive a physical model from a finite set of observations containing errors, and how one may determine the quality of such a model. This book takes on these fundamental and challenging problems, introducing students and professionals to the broad range of approaches that lie in the realm of inverse theory. The authors present both the underlying theory and practical algorithms for solving inverse problems. The authors' treatment is appropriate for geoscience graduate students and advanced undergraduates with a basic working knowledge of calculus, linear algebra, and statistics. "Parameter Estimation and Inverse Problems, 2/e" introduces readers to both Classical and Bayesian approaches to linear and nonlinear problems with particular attention paid to computational, mathematical, and statistical issues related to their application to geophysical problems. The textbook includes Appendices covering essential linear algebra, statistics, and notation in the context of the subject. This book includes a companion website that features computational examples (including all examples contained in the textbook) and useful subroutines using MATLAB. It: includes appendices for review of needed concepts in linear, statistics, and vector calculus; features a companion website that contains comprehensive MATLAB code for all examples, which readers can reproduce, experiment with, and modify; offers an online instructor's guide that helps professors teach, customize exercises, and select homework problems; and, is accessible to students and professionals without a highly specialized mathematical background.
---
paper_title: Sparse Modeling for Image and Vision Processing

paper_content:

In recent years, a large amount of multi-disciplinary research has been conducted on sparse models and their applications. In statistics and machine learning, the sparsity principle is used to perform model selection---that is, automatically selecting a simple model among a large collection of them. In signal processing, sparse coding consists of representing data with linear combinations of a few dictionary elements. Subsequently, the corresponding tools have been widely adopted by several scientific communities such as neuroscience, bioinformatics, or computer vision. The goal of this monograph is to offer a self-contained view of sparse modeling for visual recognition and image processing. More specifically, we focus on applications where the dictionary is learned and adapted to data, yielding a compact representation that has been successful in various contexts.
---
paper_title: Regression Shrinkage and Selection Via the Lasso

paper_content:

SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described.
---
paper_title: Sparse Modeling for Image and Vision Processing

paper_content:

In recent years, a large amount of multi-disciplinary research has been conducted on sparse models and their applications. In statistics and machine learning, the sparsity principle is used to perform model selection---that is, automatically selecting a simple model among a large collection of them. In signal processing, sparse coding consists of representing data with linear combinations of a few dictionary elements. Subsequently, the corresponding tools have been widely adopted by several scientific communities such as neuroscience, bioinformatics, or computer vision. The goal of this monograph is to offer a self-contained view of sparse modeling for visual recognition and image processing. More specifically, we focus on applications where the dictionary is learned and adapted to data, yielding a compact representation that has been successful in various contexts.
---
paper_title: Machine Learning: A Probabilistic Perspective

paper_content:

Today's Web-enabled deluge of electronic data calls for automated methods of data analysis. Machine learning provides these, developing methods that can automatically detect patterns in data and then use the uncovered patterns to predict future data. This textbook offers a comprehensive and self-contained introduction to the field of machine learning, based on a unified, probabilistic approach. The coverage combines breadth and depth, offering necessary background material on such topics as probability, optimization, and linear algebra as well as discussion of recent developments in the field, including conditional random fields, L1 regularization, and deep learning. The book is written in an informal, accessible style, complete with pseudo-code for the most important algorithms. All topics are copiously illustrated with color images and worked examples drawn from such application domains as biology, text processing, computer vision, and robotics. Rather than providing a cookbook of different heuristic methods, the book stresses a principled model-based approach, often using the language of graphical models to specify models in a concise and intuitive way. Almost all the models described have been implemented in a MATLAB software package--PMTK (probabilistic modeling toolkit)--that is freely available online. The book is suitable for upper-level undergraduates with an introductory-level college math background and beginning graduate students.
---
paper_title: Multipath Broadband Localization, Bathymetry, and Sediment Inversion

paper_content:

Transmission of linearly frequency modulated pulses generates receptions at a vertical line array that can be cross correlated with the source signal to provide estimates of the oceanic waveguide impulse response. For short ranges, distinct path arrivals can be identified including the direct, surface reflection, bottom reflection, and sediment reflection. Accurate estimation of arrival times of such paths is tightly related to successful inversion for source location and water column depth and sound speed and, subsequently, estimation of sediment sound speed and thickness. To achieve accurate estimation, particle filtering is applied to the received time series at 16 phones combined with a simple cross-correlation method. Using linearization, arrival time probability density functions are connected to the geometry and water column sound-speed parameters, providing point estimates as well as probability densities. These are then employed in sediment sound speed and thickness estimation. The results, obtained from the application of the method to data collected during the Seabed Characterization Experiment, are consistent with prior information on the site.
---
paper_title: Multi-frequency sparse Bayesian learning for robust matched field processing

paper_content:

The multi-snapshot, multi-frequency sparse Bayesian learning (SBL) processor is derived and its performance compared to the Bartlett, minimum variance distortionless response, and white noise constraint processors for the matched field processing application. The two-source model and data scenario of interest includes realistic mismatch implemented in the form of array tilt and data snapshots not exactly corresponding to the range-depth grid of the replica vectors. Results demonstrate that SBL behaves similar to an adaptive processor when localizing a weaker source in the presence of a stronger source, is robust to mismatch, and exhibits improved localization performance when compared to the other processors. Unlike the basis or matching pursuit methods, SBL automatically determines sparsity and its solution can be interpreted as an ambiguity surface. Because of its computational efficiency and performance, SBL is practical for applications requiring adaptive and robust processing.
---
paper_title: Multilayer feedforward networks are universal approximators

paper_content:

Abstract This paper rigorously establishes that standard multilayer feedforward networks with as few as one hidden layer using arbitrary squashing functions are capable of approximating any Borel measurable function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available. In this sense, multilayer feedforward networks are a class of universal approximators.
---
paper_title: Dictionary Learning Algorithms for Sparse Representation

paper_content:

Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial "25 words or less"), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations.Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).
---
paper_title: Some methods for classification and analysis of multivariate observations

paper_content:

The main purpose of this paper is to describe a process for partitioning an N-dimensional population into k sets on the basis of a sample. The process, which is called 'k-means,' appears to give partitions which are reasonably efficient in the sense of within-class variance. That is, if p is the probability mass function for the population, S = {S1, S2, * *, Sk} is a partition of EN, and ui, i = 1, 2, * , k, is the conditional mean of p over the set Si, then W2(S) = ff=ISi f z u42 dp(z) tends to be low for the partitions S generated by the method. We say 'tends to be low,' primarily because of intuitive considerations, corroborated to some extent by mathematical analysis and practical computational experience. Also, the k-means procedure is easily programmed and is computationally economical, so that it is feasible to process very large samples on a digital computer. Possible applications include methods for similarity grouping, nonlinear prediction, approximating multivariate distributions, and nonparametric tests for independence among several variables. In addition to suggesting practical classification methods, the study of k-means has proved to be theoretically interesting. The k-means concept represents a generalization of the ordinary sample mean, and one is naturally led to study the pertinent asymptotic behavior, the object being to establish some sort of law of large numbers for the k-means. This problem is sufficiently interesting, in fact, for us to devote a good portion of this paper to it. The k-means are defined in section 2.1, and the main results which have been obtained on the asymptotic behavior are given there. The rest of section 2 is devoted to the proofs of these results. Section 3 describes several specific possible applications, and reports some preliminary results from computer experiments conducted to explore the possibilities inherent in the k-means idea. The extension to general metric spaces is indicated briefly in section 4. The original point of departure for the work described here was a series of problems in optimal classification (MacQueen [9]) which represented special
---
paper_title: Sparse Modeling for Image and Vision Processing

paper_content:

In recent years, a large amount of multi-disciplinary research has been conducted on sparse models and their applications. In statistics and machine learning, the sparsity principle is used to perform model selection---that is, automatically selecting a simple model among a large collection of them. In signal processing, sparse coding consists of representing data with linear combinations of a few dictionary elements. Subsequently, the corresponding tools have been widely adopted by several scientific communities such as neuroscience, bioinformatics, or computer vision. The goal of this monograph is to offer a self-contained view of sparse modeling for visual recognition and image processing. More specifically, we focus on applications where the dictionary is learned and adapted to data, yielding a compact representation that has been successful in various contexts.
---
paper_title: Non-negative matrix factorization with sparseness constraints

paper_content:

Non-negative matrix factorization (NMF) is a recently developed technique for finding parts-based, linear representations of non-negative data. Although it has successfully been applied in several applications, it does not always result in parts-based representations. In this paper, we show how explicitly incorporating the notion of `sparseness' improves the found decompositions. Additionally, we provide complete MATLAB code both for standard NMF and for our extension. Our hope is that this will further the application of these methods to solving novel data-analysis problems.
---
paper_title: Algorithms for Non-negative Matrix Factorization

paper_content:

Non-negative matrix factorization (NMF) has previously been shown to be a useful decomposition for multivariate data. Two different multiplicative algorithms for NMF are analyzed. They differ only slightly in the multiplicative factor used in the update rules. One algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence. The monotonic convergence of both algorithms can be proven using an auxiliary function analogous to that used for proving convergence of the Expectation-Maximization algorithm. The algorithms can also be interpreted as diagonally rescaled gradient descent, where the rescaling factor is optimally chosen to ensure convergence.
---
paper_title: Dictionary learning of sound speed profiles

paper_content:

To provide constraints on their inversion, ocean sound speed profiles (SSPs) often are modeled using empirical orthogonal functions (EOFs). However, this regularization, which uses the leading order EOFs with a minimum-energy constraint on their coefficients, often yields low resolution SSP estimates. In this paper, it is shown that dictionary learning, a form of unsupervised machine learning, can improve SSP resolution by generating a dictionary of shape functions for sparse processing (e.g. compressive sensing) that optimally compress SSPs; both minimizing the reconstruction error and the number of coefficients. These learned dictionaries (LDs) are not constrained to be orthogonal and thus, fit the given signals such that each signal example is approximated using few LD entries. Here, LDs describing SSP observations from the HF-97 experiment and the South China Sea are generated using the K-SVD algorithm. These LDs better explain SSP variability and require fewer coefficients than EOFs, describing much of the variability with one coefficient. Thus, LDs improve the resolution of SSP estimates with negligible computational burden.
---
paper_title: Vector quantization and signal compression

paper_content:

1 Introduction.- 1.1 Signals, Coding, and Compression.- 1.2 Optimality.- 1.3 How to Use this Book.- 1.4 Related Reading.- I Basic Tools.- 2 Random Processes and Linear Systems.- 2.1 Introduction.- 2.2 Probability.- 2.3 Random Variables and Vectors.- 2.4 Random Processes.- 2.5 Expectation.- 2.6 Linear Systems.- 2.7 Stationary and Ergodic Properties.- 2.8 Useful Processes.- 2.9 Problems.- 3 Sampling.- 3.1 Introduction.- 3.2 Periodic Sampling.- 3.3 Noise in Sampling.- 3.4 Practical Sampling Schemes.- 3.5 Sampling Jitter.- 3.6 Multidimensional Sampling.- 3.7 Problems.- 4 Linear Prediction.- 4.1 Introduction.- 4.2 Elementary Estimation Theory.- 4.3 Finite-Memory Linear Prediction.- 4.4 Forward and Backward Prediction.- 4.5 The Levinson-Durbin Algorithm.- 4.6 Linear Predictor Design from Empirical Data.- 4.7 Minimum Delay Property.- 4.8 Predictability and Determinism.- 4.9 Infinite Memory Linear Prediction.- 4.10 Simulation of Random Processes.- 4.11 Problems.- II Scalar Coding.- 5 Scalar Quantization I.- 5.1 Introduction.- 5.2 Structure of a Quantizer.- 5.3 Measuring Quantizer Performance.- 5.4 The Uniform Quantizer.- 5.5 Nonuniform Quantization and Companding.- 5.6 High Resolution: General Case.- 5.7 Problems.- 6 Scalar Quantization II.- 6.1 Introduction.- 6.2 Conditions for Optimality.- 6.3 High Resolution Optimal Companding.- 6.4 Quantizer Design Algorithms.- 6.5 Implementation.- 6.6 Problems.- 7 Predictive Quantization.- 7.1 Introduction.- 7.2 Difference Quantization.- 7.3 Closed-Loop Predictive Quantization.- 7.4 Delta Modulation.- 7.5 Problems.- 8 Bit Allocation and Transform Coding.- 8.1 Introduction.- 8.2 The Problem of Bit Allocation.- 8.3 Optimal Bit Allocation Results.- 8.4 Integer Constrained Allocation Techniques.- 8.5 Transform Coding.- 8.6 Karhunen-Loeve Transform.- 8.7 Performance Gain of Transform Coding.- 8.8 Other Transforms.- 8.9 Sub-band Coding.- 8.10 Problems.- 9 Entropy Coding.- 9.1 Introduction.- 9.2 Variable-Length Scalar Noiseless Coding.- 9.3 Prefix Codes.- 9.4 Huffman Coding.- 9.5 Vector Entropy Coding.- 9.6 Arithmetic Coding.- 9.7 Universal and Adaptive Entropy Coding.- 9.8 Ziv-Lempel Coding.- 9.9 Quantization and Entropy Coding.- 9.10 Problems.- III Vector Coding.- 10 Vector Quantization I.- 10.1 Introduction.- 10.2 Structural Properties and Characterization.- 10.3 Measuring Vector Quantizer Performance.- 10.4 Nearest Neighbor Quantizers.- 10.5 Lattice Vector Quantizers.- 10.6 High Resolution Distortion Approximations.- 10.7 Problems.- 11 Vector Quantization II.- 11.1 Introduction.- 11.2 Optimality Conditions for VQ.- 11.3 Vector Quantizer Design.- 11.4 Design Examples.- 11.5 Problems.- 12 Constrained Vector Quantization.- 12.1 Introduction.- 12.2 Complexity and Storage Limitations.- 12.3 Structurally Constrained VQ.- 12.4 Tree-Structured VQ.- 12.5 Classified VQ.- 12.6 Transform VQ.- 12.7 Product Code Techniques.- 12.8 Partitioned VQ.- 12.9 Mean-Removed VQ.- 12.10 Shape-Gain VQ.- 12.11 Multistage VQ.- 12.12 Constrained Storage VQ.- 12.13 Hierarchical and Multiresolution VQ.- 12.14 Nonlinear Interpolative VQ.- 12.15 Lattice Codebook VQ.- 12.16 Fast Nearest Neighbor Encoding.- 12.17 Problems.- 13 Predictive Vector Quantization.- 13.1 Introduction.- 13.2 Predictive Vector Quantization.- 13.3 Vector Linear Prediction.- 13.4 Predictor Design from Empirical Data.- 13.5 Nonlinear Vector Prediction.- 13.6 Design Examples.- 13.7 Problems.- 14 Finite-State Vector Quantization.- 14.1 Recursive Vector Quantizers.- 14.2 Finite-State Vector Quantizers.- 14.3 Labeled-States and Labeled-Transitions.- 14.4 Encoder/Decoder Design.- 14.5 Next-State Function Design.- 14.6 Design Examples.- 14.7 Problems.- 15 Tree and Trellis Encoding.- 15.1 Delayed Decision Encoder.- 15.2 Tree and Trellis Coding.- 15.3 Decoder Design.- 15.4 Predictive Trellis Encoders.- 15.5 Other Design Techniques.- 15.6 Problems.- 16 Adaptive Vector Quantization.- 16.1 Introduction.- 16.2 Mean Adaptation.- 16.3 Gain-Adaptive Vector Quantization.- 16.4 Switched Codebook Adaptation.- 16.5 Adaptive Bit Allocation.- 16.6 Address VQ.- 16.7 Progressive Code Vector Updating.- 16.8 Adaptive Codebook Generation.- 16.9 Vector Excitation Coding.- 16.10 Problems.- 17 Variable Rate Vector Quantization.- 17.1 Variable Rate Coding.- 17.2 Variable Dimension VQ.- 17.3 Alternative Approaches to Variable Rate VQ.- 17.4 Pruned Tree-Structured VQ.- 17.5 The Generalized BFOS Algorithm.- 17.6 Pruned Tree-Structured VQ.- 17.7 Entropy Coded VQ.- 17.8 Greedy Tree Growing.- 17.9 Design Examples.- 17.10 Bit Allocation Revisited.- 17.11 Design Algorithms.- 17.12 Problems.
---
paper_title: Compressive acoustic sound speed profile estimation

paper_content:

Ocean acoustic sound speed profile (SSP) estimation requires the inversion of acoustic fields using limited observations. Compressive sensing (CS) asserts that certain underdetermined problems can be solved in high resolution, provided their solutions are sparse. Here, CS is used to estimate SSPs in a range-independent shallow ocean by inverting a non-linear acoustic propagation model. It is shown that SSPs can be estimated using CS to resolve fine-scale structure.
---
paper_title: Compressed sensing

paper_content:

Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0<ples1. The N most important coefficients in that expansion allow reconstruction with lscr2 error O(N1/2-1p/). It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients. Moreover, a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing. The nonadaptive measurements have the character of "random" linear combinations of basis/frame elements. Our results use the notions of optimal recovery, of n-widths, and information-based complexity. We estimate the Gel'fand n-widths of lscrp balls in high-dimensional Euclidean space in the case 0<ples1, and give a criterion identifying near- optimal subspaces for Gel'fand n-widths. We show that "most" subspaces are near-optimal, and show that convex optimization (Basis Pursuit) is a near-optimal way to extract information derived from these near-optimal subspaces
---
paper_title: GradientBased Learning Applied to Document Recognition

paper_content:

Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure. Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of Graph Transformer Networks. A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.
---
paper_title: Vector quantization and signal compression

paper_content:

1 Introduction.- 1.1 Signals, Coding, and Compression.- 1.2 Optimality.- 1.3 How to Use this Book.- 1.4 Related Reading.- I Basic Tools.- 2 Random Processes and Linear Systems.- 2.1 Introduction.- 2.2 Probability.- 2.3 Random Variables and Vectors.- 2.4 Random Processes.- 2.5 Expectation.- 2.6 Linear Systems.- 2.7 Stationary and Ergodic Properties.- 2.8 Useful Processes.- 2.9 Problems.- 3 Sampling.- 3.1 Introduction.- 3.2 Periodic Sampling.- 3.3 Noise in Sampling.- 3.4 Practical Sampling Schemes.- 3.5 Sampling Jitter.- 3.6 Multidimensional Sampling.- 3.7 Problems.- 4 Linear Prediction.- 4.1 Introduction.- 4.2 Elementary Estimation Theory.- 4.3 Finite-Memory Linear Prediction.- 4.4 Forward and Backward Prediction.- 4.5 The Levinson-Durbin Algorithm.- 4.6 Linear Predictor Design from Empirical Data.- 4.7 Minimum Delay Property.- 4.8 Predictability and Determinism.- 4.9 Infinite Memory Linear Prediction.- 4.10 Simulation of Random Processes.- 4.11 Problems.- II Scalar Coding.- 5 Scalar Quantization I.- 5.1 Introduction.- 5.2 Structure of a Quantizer.- 5.3 Measuring Quantizer Performance.- 5.4 The Uniform Quantizer.- 5.5 Nonuniform Quantization and Companding.- 5.6 High Resolution: General Case.- 5.7 Problems.- 6 Scalar Quantization II.- 6.1 Introduction.- 6.2 Conditions for Optimality.- 6.3 High Resolution Optimal Companding.- 6.4 Quantizer Design Algorithms.- 6.5 Implementation.- 6.6 Problems.- 7 Predictive Quantization.- 7.1 Introduction.- 7.2 Difference Quantization.- 7.3 Closed-Loop Predictive Quantization.- 7.4 Delta Modulation.- 7.5 Problems.- 8 Bit Allocation and Transform Coding.- 8.1 Introduction.- 8.2 The Problem of Bit Allocation.- 8.3 Optimal Bit Allocation Results.- 8.4 Integer Constrained Allocation Techniques.- 8.5 Transform Coding.- 8.6 Karhunen-Loeve Transform.- 8.7 Performance Gain of Transform Coding.- 8.8 Other Transforms.- 8.9 Sub-band Coding.- 8.10 Problems.- 9 Entropy Coding.- 9.1 Introduction.- 9.2 Variable-Length Scalar Noiseless Coding.- 9.3 Prefix Codes.- 9.4 Huffman Coding.- 9.5 Vector Entropy Coding.- 9.6 Arithmetic Coding.- 9.7 Universal and Adaptive Entropy Coding.- 9.8 Ziv-Lempel Coding.- 9.9 Quantization and Entropy Coding.- 9.10 Problems.- III Vector Coding.- 10 Vector Quantization I.- 10.1 Introduction.- 10.2 Structural Properties and Characterization.- 10.3 Measuring Vector Quantizer Performance.- 10.4 Nearest Neighbor Quantizers.- 10.5 Lattice Vector Quantizers.- 10.6 High Resolution Distortion Approximations.- 10.7 Problems.- 11 Vector Quantization II.- 11.1 Introduction.- 11.2 Optimality Conditions for VQ.- 11.3 Vector Quantizer Design.- 11.4 Design Examples.- 11.5 Problems.- 12 Constrained Vector Quantization.- 12.1 Introduction.- 12.2 Complexity and Storage Limitations.- 12.3 Structurally Constrained VQ.- 12.4 Tree-Structured VQ.- 12.5 Classified VQ.- 12.6 Transform VQ.- 12.7 Product Code Techniques.- 12.8 Partitioned VQ.- 12.9 Mean-Removed VQ.- 12.10 Shape-Gain VQ.- 12.11 Multistage VQ.- 12.12 Constrained Storage VQ.- 12.13 Hierarchical and Multiresolution VQ.- 12.14 Nonlinear Interpolative VQ.- 12.15 Lattice Codebook VQ.- 12.16 Fast Nearest Neighbor Encoding.- 12.17 Problems.- 13 Predictive Vector Quantization.- 13.1 Introduction.- 13.2 Predictive Vector Quantization.- 13.3 Vector Linear Prediction.- 13.4 Predictor Design from Empirical Data.- 13.5 Nonlinear Vector Prediction.- 13.6 Design Examples.- 13.7 Problems.- 14 Finite-State Vector Quantization.- 14.1 Recursive Vector Quantizers.- 14.2 Finite-State Vector Quantizers.- 14.3 Labeled-States and Labeled-Transitions.- 14.4 Encoder/Decoder Design.- 14.5 Next-State Function Design.- 14.6 Design Examples.- 14.7 Problems.- 15 Tree and Trellis Encoding.- 15.1 Delayed Decision Encoder.- 15.2 Tree and Trellis Coding.- 15.3 Decoder Design.- 15.4 Predictive Trellis Encoders.- 15.5 Other Design Techniques.- 15.6 Problems.- 16 Adaptive Vector Quantization.- 16.1 Introduction.- 16.2 Mean Adaptation.- 16.3 Gain-Adaptive Vector Quantization.- 16.4 Switched Codebook Adaptation.- 16.5 Adaptive Bit Allocation.- 16.6 Address VQ.- 16.7 Progressive Code Vector Updating.- 16.8 Adaptive Codebook Generation.- 16.9 Vector Excitation Coding.- 16.10 Problems.- 17 Variable Rate Vector Quantization.- 17.1 Variable Rate Coding.- 17.2 Variable Dimension VQ.- 17.3 Alternative Approaches to Variable Rate VQ.- 17.4 Pruned Tree-Structured VQ.- 17.5 The Generalized BFOS Algorithm.- 17.6 Pruned Tree-Structured VQ.- 17.7 Entropy Coded VQ.- 17.8 Greedy Tree Growing.- 17.9 Design Examples.- 17.10 Bit Allocation Revisited.- 17.11 Design Algorithms.- 17.12 Problems.
---
paper_title: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition

paper_content:

We describe a recursive algorithm to compute representations of functions with respect to nonorthogonal and possibly overcomplete dictionaries of elementary building blocks e.g. affine (wavelet) frames. We propose a modification to the matching pursuit algorithm of Mallat and Zhang (1992) that maintains full backward orthogonality of the residual (error) at every step and thereby leads to improved convergence. We refer to this modified algorithm as orthogonal matching pursuit (OMP). It is shown that all additional computation required for the OMP algorithm may be performed recursively. >
---
paper_title: Vector quantization and signal compression

paper_content:

1 Introduction.- 1.1 Signals, Coding, and Compression.- 1.2 Optimality.- 1.3 How to Use this Book.- 1.4 Related Reading.- I Basic Tools.- 2 Random Processes and Linear Systems.- 2.1 Introduction.- 2.2 Probability.- 2.3 Random Variables and Vectors.- 2.4 Random Processes.- 2.5 Expectation.- 2.6 Linear Systems.- 2.7 Stationary and Ergodic Properties.- 2.8 Useful Processes.- 2.9 Problems.- 3 Sampling.- 3.1 Introduction.- 3.2 Periodic Sampling.- 3.3 Noise in Sampling.- 3.4 Practical Sampling Schemes.- 3.5 Sampling Jitter.- 3.6 Multidimensional Sampling.- 3.7 Problems.- 4 Linear Prediction.- 4.1 Introduction.- 4.2 Elementary Estimation Theory.- 4.3 Finite-Memory Linear Prediction.- 4.4 Forward and Backward Prediction.- 4.5 The Levinson-Durbin Algorithm.- 4.6 Linear Predictor Design from Empirical Data.- 4.7 Minimum Delay Property.- 4.8 Predictability and Determinism.- 4.9 Infinite Memory Linear Prediction.- 4.10 Simulation of Random Processes.- 4.11 Problems.- II Scalar Coding.- 5 Scalar Quantization I.- 5.1 Introduction.- 5.2 Structure of a Quantizer.- 5.3 Measuring Quantizer Performance.- 5.4 The Uniform Quantizer.- 5.5 Nonuniform Quantization and Companding.- 5.6 High Resolution: General Case.- 5.7 Problems.- 6 Scalar Quantization II.- 6.1 Introduction.- 6.2 Conditions for Optimality.- 6.3 High Resolution Optimal Companding.- 6.4 Quantizer Design Algorithms.- 6.5 Implementation.- 6.6 Problems.- 7 Predictive Quantization.- 7.1 Introduction.- 7.2 Difference Quantization.- 7.3 Closed-Loop Predictive Quantization.- 7.4 Delta Modulation.- 7.5 Problems.- 8 Bit Allocation and Transform Coding.- 8.1 Introduction.- 8.2 The Problem of Bit Allocation.- 8.3 Optimal Bit Allocation Results.- 8.4 Integer Constrained Allocation Techniques.- 8.5 Transform Coding.- 8.6 Karhunen-Loeve Transform.- 8.7 Performance Gain of Transform Coding.- 8.8 Other Transforms.- 8.9 Sub-band Coding.- 8.10 Problems.- 9 Entropy Coding.- 9.1 Introduction.- 9.2 Variable-Length Scalar Noiseless Coding.- 9.3 Prefix Codes.- 9.4 Huffman Coding.- 9.5 Vector Entropy Coding.- 9.6 Arithmetic Coding.- 9.7 Universal and Adaptive Entropy Coding.- 9.8 Ziv-Lempel Coding.- 9.9 Quantization and Entropy Coding.- 9.10 Problems.- III Vector Coding.- 10 Vector Quantization I.- 10.1 Introduction.- 10.2 Structural Properties and Characterization.- 10.3 Measuring Vector Quantizer Performance.- 10.4 Nearest Neighbor Quantizers.- 10.5 Lattice Vector Quantizers.- 10.6 High Resolution Distortion Approximations.- 10.7 Problems.- 11 Vector Quantization II.- 11.1 Introduction.- 11.2 Optimality Conditions for VQ.- 11.3 Vector Quantizer Design.- 11.4 Design Examples.- 11.5 Problems.- 12 Constrained Vector Quantization.- 12.1 Introduction.- 12.2 Complexity and Storage Limitations.- 12.3 Structurally Constrained VQ.- 12.4 Tree-Structured VQ.- 12.5 Classified VQ.- 12.6 Transform VQ.- 12.7 Product Code Techniques.- 12.8 Partitioned VQ.- 12.9 Mean-Removed VQ.- 12.10 Shape-Gain VQ.- 12.11 Multistage VQ.- 12.12 Constrained Storage VQ.- 12.13 Hierarchical and Multiresolution VQ.- 12.14 Nonlinear Interpolative VQ.- 12.15 Lattice Codebook VQ.- 12.16 Fast Nearest Neighbor Encoding.- 12.17 Problems.- 13 Predictive Vector Quantization.- 13.1 Introduction.- 13.2 Predictive Vector Quantization.- 13.3 Vector Linear Prediction.- 13.4 Predictor Design from Empirical Data.- 13.5 Nonlinear Vector Prediction.- 13.6 Design Examples.- 13.7 Problems.- 14 Finite-State Vector Quantization.- 14.1 Recursive Vector Quantizers.- 14.2 Finite-State Vector Quantizers.- 14.3 Labeled-States and Labeled-Transitions.- 14.4 Encoder/Decoder Design.- 14.5 Next-State Function Design.- 14.6 Design Examples.- 14.7 Problems.- 15 Tree and Trellis Encoding.- 15.1 Delayed Decision Encoder.- 15.2 Tree and Trellis Coding.- 15.3 Decoder Design.- 15.4 Predictive Trellis Encoders.- 15.5 Other Design Techniques.- 15.6 Problems.- 16 Adaptive Vector Quantization.- 16.1 Introduction.- 16.2 Mean Adaptation.- 16.3 Gain-Adaptive Vector Quantization.- 16.4 Switched Codebook Adaptation.- 16.5 Adaptive Bit Allocation.- 16.6 Address VQ.- 16.7 Progressive Code Vector Updating.- 16.8 Adaptive Codebook Generation.- 16.9 Vector Excitation Coding.- 16.10 Problems.- 17 Variable Rate Vector Quantization.- 17.1 Variable Rate Coding.- 17.2 Variable Dimension VQ.- 17.3 Alternative Approaches to Variable Rate VQ.- 17.4 Pruned Tree-Structured VQ.- 17.5 The Generalized BFOS Algorithm.- 17.6 Pruned Tree-Structured VQ.- 17.7 Entropy Coded VQ.- 17.8 Greedy Tree Growing.- 17.9 Design Examples.- 17.10 Bit Allocation Revisited.- 17.11 Design Algorithms.- 17.12 Problems.
---
paper_title: A theory for multiresolution signal decomposition: the wavelet representation

paper_content:

Multiresolution representations are effective for analyzing the information content of images. The properties of the operator which approximates a signal at a given resolution were studied. It is shown that the difference of information between the approximation of a signal at the resolutions 2/sup j+1/ and 2/sup j/ (where j is an integer) can be extracted by decomposing this signal on a wavelet orthonormal basis of L/sup 2/(R/sup n/), the vector space of measurable, square-integrable n-dimensional functions. In L/sup 2/(R), a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi (x). This decomposition defines an orthogonal multiresolution representation called a wavelet representation. It is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters. Wavelet representation lies between the spatial and Fourier domains. For images, the wavelet representation differentiates several spatial orientations. The application of this representation to data compression in image coding, texture discrimination and fractal analysis is discussed. >
---
paper_title: Deep Learning: Methods and Applications

paper_content:

This monograph provides an overview of general deep learning methodology and its applications to a variety of signal and information processing tasks. The application areas are chosen with the following three criteria in mind: (1) expertise or knowledge of the authors; (2) the application areas that have already been transformed by the successful use of deep learning technology, such as speech recognition and computer vision; and (3) the application areas that have the potential to be impacted significantly by deep learning and that have been experiencing research growth, including natural language and text processing, information retrieval, and multimodal information processing empowered by multi-task deep learning.
---
paper_title: Broadband DOA estimation using Convolutional neural networks trained with noise signals

paper_content:

A convolution neural network (CNN) based classification method for broadband DOA estimation is proposed, where the phase component of the short-time Fourier transform coefficients of the received microphone signals are directly fed into the CNN and the features required for DOA estimation are learned during training. Since only the phase component of the input is used, the CNN can be trained with synthesized noise signals, thereby making the preparation of the training data set easier compared to using speech signals. Through experimental evaluation, the ability of the proposed noise trained CNN framework to generalize to speech sources is demonstrated. In addition, the robustness of the system to noise, small perturbations in microphone positions, as well as its ability to adapt to different acoustic conditions is investigated using experiments with simulated and real data.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: MatConvNet: Convolutional Neural Networks for MATLAB

paper_content:

MatConvNet is an implementation of Convolutional Neural Networks (CNNs) for MATLAB. The toolbox is designed with an emphasis on simplicity and flexibility. It exposes the building blocks of CNNs as easy-to-use MATLAB functions, providing routines for computing linear convolutions with filter banks, feature pooling, and many more. In this manner, MatConvNet allows fast prototyping of new CNN architectures; at the same time, it supports efficient computation on CPU and GPU allowing to train complex models on large datasets such as ImageNet ILSVRC. This document provides an overview of CNNs and how they are implemented in MatConvNet and gives the technical details of each computational block in the toolbox.
---
paper_title: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

paper_content:

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on the learnable activation and advanced initialization, we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66% [33]). To our knowledge, our result is the first to surpass the reported human-level performance (5.1%, [26]) on this dataset.
---
paper_title: Understanding the difficulty of training deep feedforward neural networks

paper_content:

Whereas before 2006 it appears that deep multilayer neural networks were not successfully trained, since then several algorithms have been shown to successfully train them, with experimental results showing the superiority of deeper vs less deep architectures. All these experimental results were obtained with new initialization or training mechanisms. Our objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future. We first observe the influence of the non-linear activations functions. We find that the logistic sigmoid activation is unsuited for deep networks with random initialization because of its mean value, which can drive especially the top hidden layer into saturation. Surprisingly, we find that saturated units can move out of saturation by themselves, albeit slowly, and explaining the plateaus sometimes seen when training neural networks. We find that a new non-linearity that saturates less can often be beneficial. Finally, we study how activations and gradients vary across layers and during training, with the idea that training may be more difficult when the singular values of the Jacobian associated with each layer are far from 1. Based on these considerations, we propose a new initialization scheme that brings substantially faster convergence. 1 Deep Neural Networks Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features. They include Appearing in Proceedings of the 13 International Conference on Artificial Intelligence and Statistics (AISTATS) 2010, Chia Laguna Resort, Sardinia, Italy. Volume 9 of JMLR: WC Weston et al., 2008). Much attention has recently been devoted to them (see (Bengio, 2009) for a review), because of their theoretical appeal, inspiration from biology and human cognition, and because of empirical success in vision (Ranzato et al., 2007; Larochelle et al., 2007; Vincent et al., 2008) and natural language processing (NLP) (Collobert & Weston, 2008; Mnih & Hinton, 2009). Theoretical results reviewed and discussed by Bengio (2009), suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one may need deep architectures. Most of the recent experimental results with deep architecture are obtained with models that can be turned into deep supervised neural networks, but with initialization or training schemes different from the classical feedforward neural networks (Rumelhart et al., 1986). Why are these new algorithms working so much better than the standard random initialization and gradient-based optimization of a supervised training criterion? Part of the answer may be found in recent analyses of the effect of unsupervised pretraining (Erhan et al., 2009), showing that it acts as a regularizer that initializes the parameters in a “better” basin of attraction of the optimization procedure, corresponding to an apparent local minimum associated with better generalization. But earlier work (Bengio et al., 2007) had shown that even a purely supervised but greedy layer-wise procedure would give better results. So here instead of focusing on what unsupervised pre-training or semi-supervised criteria bring to deep architectures, we focus on analyzing what may be going wrong with good old (but deep) multilayer neural networks. Our analysis is driven by investigative experiments to monitor activations (watching for saturation of hidden units) and gradients, across layers and across training iterations. We also evaluate the effects on these of choices of activation function (with the idea that it might affect saturation) and initialization procedure (since unsupervised pretraining is a particular form of initialization and it has a drastic impact).
---
paper_title: A Fast Learning Algorithm for Deep Belief Nets

paper_content:

We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.
---
paper_title: Greedy layer-wise training of deep networks

paper_content:

Complexity theory of circuits strongly suggests that deep architectures can be much more efficient (sometimes exponentially) than shallow architectures, in terms of computational elements required to represent some functions. Deep multi-layer neural networks have many levels of non-linearities allowing them to compactly represent highly non-linear and highly-varying functions. However, until recently it was not clear how to train such deep networks, since gradient-based optimization starting from random initialization appears to often get stuck in poor solutions. Hinton et al. recently introduced a greedy layer-wise unsupervised learning algorithm for Deep Belief Networks (DBN), a generative model with many layers of hidden causal variables. In the context of the above optimization problem, we study this algorithm empirically and explore variants to better understand its success and extend it to cases where the inputs are continuous or where the structure of the input distribution is not revealing enough about the variable to be predicted in a supervised task. Our experiments also confirm the hypothesis that the greedy layer-wise unsupervised training strategy mostly helps the optimization, by initializing weights in a region near a good local minimum, giving rise to internal distributed representations that are high-level abstractions of the input, bringing better generalization.
---
paper_title: Adaptive subgradient methods for online learning and stochastic optimization

paper_content:

We present a new family of subgradient methods that dynamically incorporate knowledge of the geometry of the data observed in earlier iterations to perform more informative gradient-based learning. Metaphorically, the adaptation allows us to find needles in haystacks in the form of very predictive but rarely seen features. Our paradigm stems from recent advances in stochastic optimization and online learning which employ proximal functions to control the gradient steps of the algorithm. We describe and analyze an apparatus for adaptively modifying the proximal function, which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal function that can be chosen in hindsight. We give several efficient algorithms for empirical risk minimization problems with common and important regularization functions and domain constraints. We experimentally study our theoretical analysis and show that adaptive subgradient methods outperform state-of-the-art, yet non-adaptive, subgradient algorithms.
---
paper_title: Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

paper_content:

Rectified activation units (rectifiers) are essential for state-of-the-art neural networks. In this work, we study rectifier neural networks for image classification from two aspects. First, we propose a Parametric Rectified Linear Unit (PReLU) that generalizes the traditional rectified unit. PReLU improves model fitting with nearly zero extra computational cost and little overfitting risk. Second, we derive a robust initialization method that particularly considers the rectifier nonlinearities. This method enables us to train extremely deep rectified models directly from scratch and to investigate deeper or wider network architectures. Based on the learnable activation and advanced initialization, we achieve 4.94% top-5 test error on the ImageNet 2012 classification dataset. This is a 26% relative improvement over the ILSVRC 2014 winner (GoogLeNet, 6.66% [33]). To our knowledge, our result is the first to surpass the reported human-level performance (5.1%, [26]) on this dataset.
---
paper_title: Dropout: a simple way to prevent neural networks from overfitting

paper_content:

Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
---
paper_title: Understanding the difficulty of training deep feedforward neural networks

paper_content:

Whereas before 2006 it appears that deep multilayer neural networks were not successfully trained, since then several algorithms have been shown to successfully train them, with experimental results showing the superiority of deeper vs less deep architectures. All these experimental results were obtained with new initialization or training mechanisms. Our objective here is to understand better why standard gradient descent from random initialization is doing so poorly with deep neural networks, to better understand these recent relative successes and help design better algorithms in the future. We first observe the influence of the non-linear activations functions. We find that the logistic sigmoid activation is unsuited for deep networks with random initialization because of its mean value, which can drive especially the top hidden layer into saturation. Surprisingly, we find that saturated units can move out of saturation by themselves, albeit slowly, and explaining the plateaus sometimes seen when training neural networks. We find that a new non-linearity that saturates less can often be beneficial. Finally, we study how activations and gradients vary across layers and during training, with the idea that training may be more difficult when the singular values of the Jacobian associated with each layer are far from 1. Based on these considerations, we propose a new initialization scheme that brings substantially faster convergence. 1 Deep Neural Networks Deep learning methods aim at learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features. They include Appearing in Proceedings of the 13 International Conference on Artificial Intelligence and Statistics (AISTATS) 2010, Chia Laguna Resort, Sardinia, Italy. Volume 9 of JMLR: WC Weston et al., 2008). Much attention has recently been devoted to them (see (Bengio, 2009) for a review), because of their theoretical appeal, inspiration from biology and human cognition, and because of empirical success in vision (Ranzato et al., 2007; Larochelle et al., 2007; Vincent et al., 2008) and natural language processing (NLP) (Collobert & Weston, 2008; Mnih & Hinton, 2009). Theoretical results reviewed and discussed by Bengio (2009), suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one may need deep architectures. Most of the recent experimental results with deep architecture are obtained with models that can be turned into deep supervised neural networks, but with initialization or training schemes different from the classical feedforward neural networks (Rumelhart et al., 1986). Why are these new algorithms working so much better than the standard random initialization and gradient-based optimization of a supervised training criterion? Part of the answer may be found in recent analyses of the effect of unsupervised pretraining (Erhan et al., 2009), showing that it acts as a regularizer that initializes the parameters in a “better” basin of attraction of the optimization procedure, corresponding to an apparent local minimum associated with better generalization. But earlier work (Bengio et al., 2007) had shown that even a purely supervised but greedy layer-wise procedure would give better results. So here instead of focusing on what unsupervised pre-training or semi-supervised criteria bring to deep architectures, we focus on analyzing what may be going wrong with good old (but deep) multilayer neural networks. Our analysis is driven by investigative experiments to monitor activations (watching for saturation of hidden units) and gradients, across layers and across training iterations. We also evaluate the effects on these of choices of activation function (with the idea that it might affect saturation) and initialization procedure (since unsupervised pretraining is a particular form of initialization and it has a drastic impact).
---
paper_title: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

paper_content:

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch. Batch Normalization allows us to use much higher learning rates and be less careful about initialization, and in some cases eliminates the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.82% top-5 test error, exceeding the accuracy of human raters.
---
paper_title: GradientBased Learning Applied to Document Recognition

paper_content:

Multilayer Neural Networks trained with the backpropagation algorithm constitute the best example of a successful Gradient-Based Learning technique. Given an appropriate network architecture, Gradient-Based Learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional Neural Networks, that are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation, recognition, and language modeling. A new learning paradigm, called Graph Transformer Networks (GTN), allows such multi-module systems to be trained globally using Gradient-Based methods so as to minimize an overall performance measure. Two systems for on-line handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of Graph Transformer Networks. A Graph Transformer Network for reading bank check is also described. It uses Convolutional Neural Network character recognizers combined with global training techniques to provides record accuracy on business and personal checks. It is deployed commercially and reads several million checks per day.
---
paper_title: Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

paper_content:

A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by “learning without a teacher”, and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname “neocognitron”. After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel. The network consits of an input layer (photoreceptor array) followed by a cascade connection of a number of modular structures, each of which is composed of two layers of cells connected in a cascade. The first layer of each module consists of “S-cells”, which show characteristics similar to simple cells or lower order hypercomplex cells, and the second layer consists of “C-cells” similar to complex cells or higher order hypercomplex cells. The afferent synapses to each S-cell have plasticity and are modifiable. The network has an ability of unsupervised learning: We do not need any “teacher” during the process of self-organization, and it is only needed to present a set of stimulus patterns repeatedly to the input layer of the network. The network has been simulated on a digital computer. After repetitive presentation of a set of stimulus patterns, each stimulus pattern has become to elicit an output only from one of the C-cell of the last layer, and conversely, this C-cell has become selectively responsive only to that stimulus pattern. That is, none of the C-cells of the last layer responds to more than one stimulus pattern. The response of the C-cells of the last layer is not affected by the pattern's position at all. Neither is it affected by a small change in shape nor in size of the stimulus pattern.
---
paper_title: Sparse coding with an overcomplete basis set: A strategy employed by V1?

paper_content:

The spatial receptive fields of simple cells in mammalian striate cortex have been reasonably well described physiologically and can be characterized as being localized, oriented, and ban@ass, comparable with the basis functions of wavelet transforms. Previously, we have shown that these receptive field properties may be accounted for in terms of a strategy for producing a sparse distribution of output activity in response to natural images. Here, in addition to describing this work in a more expansive fashion, we examine the neurobiological implications of sparse coding. Of particular interest is the case when the code is overcomplete--i.e., when the number of code elements is greater than the effective dimensionality of the input space. Because the basis functions are non-orthogonal and not linearly independent of each other, sparsifying the code will recruit only those basis functions necessary for representing a given input, and so the input-output function will deviate from being purely linear. These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli. © 1997 Elsevier Science Ltd
---
paper_title: Long Short-Term Memory

paper_content:

Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
---
paper_title: DCASE 2017 Challenge setup: Tasks, datasets and baseline system

paper_content:

DCASE 2017 Challenge consists of four tasks: acoustic scene classification , detection of rare sound events, sound event detection in real-life audio, and large-scale weakly supervised sound event detection for smart cars. This paper presents the setup of these tasks: task definition, dataset, experimental setup, and baseline system results on the development dataset. The baseline systems for all tasks rely on the same implementation using multilayer perceptron and log mel-energies, but differ in the structure of the output layer and the decision making process, as well as the evaluation of system output using task specific metrics.
---
paper_title: Deep learning for ocean acoustic source localization using one sensor

paper_content:

A deep learning approach based on big data is proposed to locate broadband acoustic sources with one hydrophone in ocean waveguides. Several 50-layer residual neural networks, trained on a huge number of sound field replicas generated by an acoustic propagation model, are used to handle the environmental uncertainty in source localization. A two-step training strategy is presented to improve the training of the deep models. First, the range is discretized in a coarse (5 km) grid. Subsequently, the source range within the selected interval and source depth are discretized on a finer (0.1 km and 2 m) grid. The deep learning methods were tested for simulated data in uncertain environments and on experimental data. In both tests, the deep learning methods successfully estimated source ranges and depths in uncertain environments.
---
paper_title: The generalized correlation method for estimation of time delay

paper_content:

A maximum likelihood (ML) estimator is developed for determining time delay between signals received at two spatially separated sensors in the presence of uncorrelated noise. This ML estimator can be realized as a pair of receiver prefilters followed by a cross correlator. The time argument at which the correlator achieves a maximum is the delay estimate. The ML estimator is compared with several other proposed processors of similar form. Under certain conditions the ML estimator is shown to be identical to one proposed by Hannan and Thomson [10] and MacDonald and Schultheiss [21]. Qualitatively, the role of the prefilters is to accentuate the signal passed to the correlator at frequencies for which the signal-to-noise (S/N) ratio is highest and, simultaneously, to suppress the noise power. The same type of prefiltering is provided by the generalized Eckart filter, which maximizes the S/N ratio of the correlator output. For low S/N ratio, the ML estimator is shown to be equivalent to Eckart prefiltering.
---
paper_title: Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks

paper_content:

In this paper, we propose a convolutional recurrent neural network for joint sound event localization and detection (SELD) of multiple overlapping sound events in three-dimensional (3-D) space. The proposed network takes a sequence of consecutive spectrogram time frames as input and maps it to two outputs in parallel. As the first output, the sound event detection (SED) is performed as a multi-label classification task on each time frame producing temporal activity for all the sound event classes. As the second output, localization is performed by estimating the 3-D Cartesian coordinates of the direction-of-arrival (DOA) for each sound event class using multi-output regression. The proposed method is able to associate multiple DOAs with respective sound event labels and further track this association with respect to time. The proposed method uses separately the phase and magnitude component of the spectrogram calculated on each audio channel as the feature, thereby avoiding any method- and array-specific feature extraction. The method is evaluated on five Ambisonic and two circular array format datasets with different overlapping sound events in anechoic, reverberant, and real-life scenarios. The proposed method is compared with two SED, three DOA estimation, and one SELD baselines. The results show that the proposed method is generic and applicable to any array structures, robust to unseen DOA values, reverberation, and low SNR scenarios. The proposed method achieved a consistently higher recall of the estimated number of DOAs across datasets in comparison to the best baseline. Additionally, this recall was observed to be significantly better than the best baseline method for a higher number of overlapping sound events.
---
paper_title: Broadband DOA estimation using Convolutional neural networks trained with noise signals

paper_content:

A convolution neural network (CNN) based classification method for broadband DOA estimation is proposed, where the phase component of the short-time Fourier transform coefficients of the received microphone signals are directly fed into the CNN and the features required for DOA estimation are learned during training. Since only the phase component of the input is used, the CNN can be trained with synthesized noise signals, thereby making the preparation of the training data set easier compared to using speech signals. Through experimental evaluation, the ability of the proposed noise trained CNN framework to generalize to speech sources is demonstrated. In addition, the robustness of the system to noise, small perturbations in microphone positions, as well as its ability to adapt to different acoustic conditions is investigated using experiments with simulated and real data.
---
paper_title: DCASE 2017 Challenge setup: Tasks, datasets and baseline system

paper_content:

DCASE 2017 Challenge consists of four tasks: acoustic scene classification , detection of rare sound events, sound event detection in real-life audio, and large-scale weakly supervised sound event detection for smart cars. This paper presents the setup of these tasks: task definition, dataset, experimental setup, and baseline system results on the development dataset. The baseline systems for all tasks rely on the same implementation using multilayer perceptron and log mel-energies, but differ in the structure of the output layer and the decision making process, as well as the evaluation of system output using task specific metrics.
---
paper_title: Convolutional Recurrent Neural Networks for Polyphonic Sound Event Detection

paper_content:

Sound events often occur in unstructured environments where they exhibit wide variations in their frequency content and temporal structure. Convolutional neural networks CNNs are able to extract higher level features that are invariant to local spectral and temporal variations. Recurrent neural networks RNNs are powerful in learning the longer term temporal context in the audio signals. CNNs and RNNs as classifiers have recently shown improved performances over established methods in various sound recognition tasks. We combine these two approaches in a convolutional recurrent neural network CRNN and apply it on a polyphonic sound event detection task. We compare the performance of the proposed CRNN method with CNN, RNN, and other established methods, and observe a considerable improvement for four different datasets consisting of everyday sound events.
---
paper_title: Deep learning for ocean acoustic source localization using one sensor

paper_content:

A deep learning approach based on big data is proposed to locate broadband acoustic sources with one hydrophone in ocean waveguides. Several 50-layer residual neural networks, trained on a huge number of sound field replicas generated by an acoustic propagation model, are used to handle the environmental uncertainty in source localization. A two-step training strategy is presented to improve the training of the deep models. First, the range is discretized in a coarse (5 km) grid. Subsequently, the source range within the selected interval and source depth are discretized on a finer (0.1 km and 2 m) grid. The deep learning methods were tested for simulated data in uncertain environments and on experimental data. In both tests, the deep learning methods successfully estimated source ranges and depths in uncertain environments.
---
paper_title: Speech Dereverberation Using Fully Convolutional Networks

paper_content:

Speech derverberation using a single microphone is addressed in this paper. Motivated by the recent success of the fully convolutional networks (FCN) in many image processing applications, we investigate their applicability to enhance the speech signal represented by short-time Fourier transform (STFT) images. We present two variations: a “U-Net” which is an encoder-decoder network with skip connections and a generative adversarial network (GAN) with U-Net as generator, which yields a more intuitive cost function for training. To evaluate our method we used the data from the REVERB challenge, and compared our results to other methods under the same conditions. We have found that our method outperforms the competing methods in most cases.
---
paper_title: Multi-Speaker DOA Estimation Using Deep Convolutional Networks Trained with Noise Signals

paper_content:

Supervised learning-based methods for source localization, being data driven, can be adapted to different acoustic conditions via training and have been shown to be robust to adverse acoustic environments. In this paper, a convolutional neural network (CNN) based supervised learning method for estimating the direction of arrival (DOA) of multiple speakers is proposed. Multi-speaker DOA estimation is formulated as a multi-class multi-label classification problem, where the assignment of each DOA label to the input feature is treated as a separate binary classification problem. The phase component of the short-time Fourier transform (STFT) coefficients of the received microphone signals are directly fed into the CNN, and the features for DOA estimation are learnt during training. Utilizing the assumption of disjoint speaker activity in the STFT domain, a novel method is proposed to train the CNN with synthesized noise signals. Through experimental evaluation with both simulated and measured acoustic impulse responses, the ability of the proposed DOA estimation approach to adapt to unseen acoustic conditions and its robustness to unseen noise type is demonstrated. Through additional empirical investigation, it is also shown that with an array of M microphone our proposed framework yields the best localization performance with M -1 convolution layers. The ability of the proposed method to accurately localize speakers in a dynamic acoustic scenario with varying number of sources is also shown.
---
paper_title: The LOCATA Challenge Data Corpus for Acoustic Source Localization and Tracking

paper_content:

Algorithms for acoustic source localization and tracking are essential for a wide range of applications such as personal assistants, smart homes, tele-conferencing systems, hearing aids, or autonomous systems. Numerous algorithms have been proposed for this purpose which, however, are not evaluated and compared against each other by using a common database so far. The IEEE-AASP Challenge on sound source localization and tracking (LOCATA) provides a novel, comprehensive data corpus for the objective benchmarking of state-of-the-art algorithms on sound source localization and tracking. The data corpus comprises six tasks ranging from the localization of a single static sound source with a static microphone array to the tracking of multiple moving speakers with a moving microphone array. It contains real-world multichannel audio recordings, obtained by hearing aids, microphones integrated in a robot head, a planar and a spherical microphone array in an enclosed acoustic environment, as well as positional information about the involved arrays and sound sources represented by moving human talkers or static loudspeakers.
---
paper_title: CRNN-Based Multiple DoA Estimation Using Acoustic Intensity Features for Ambisonics Recordings

paper_content:

Localizing audio sources is challenging in real reverberant environments, especially when several sources are active. We propose to use a neural network built from stacked convolutional and recurrent layers in order to estimate the directions of arrival of multiple sources from a first-order Ambisonics recording. It returns the directions of arrival over a discrete grid of a known number of sources. We propose to use features derived from the acoustic intensity vector as inputs. We analyze the behavior of the neural network by means of a visualization technique called layerwise relevance propagation. This analysis highlights which parts of the input signal are relevant in a given situation. We also conduct experiments to evaluate the performance of our system in various environments, from simulated rooms to real recordings, with one or two speech sources. The results show that the proposed features significantly improve performances with respect to raw Ambisonics inputs.
---
paper_title: Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks

paper_content:

In this paper, we propose a convolutional recurrent neural network for joint sound event localization and detection (SELD) of multiple overlapping sound events in three-dimensional (3-D) space. The proposed network takes a sequence of consecutive spectrogram time frames as input and maps it to two outputs in parallel. As the first output, the sound event detection (SED) is performed as a multi-label classification task on each time frame producing temporal activity for all the sound event classes. As the second output, localization is performed by estimating the 3-D Cartesian coordinates of the direction-of-arrival (DOA) for each sound event class using multi-output regression. The proposed method is able to associate multiple DOAs with respective sound event labels and further track this association with respect to time. The proposed method uses separately the phase and magnitude component of the spectrogram calculated on each audio channel as the feature, thereby avoiding any method- and array-specific feature extraction. The method is evaluated on five Ambisonic and two circular array format datasets with different overlapping sound events in anechoic, reverberant, and real-life scenarios. The proposed method is compared with two SED, three DOA estimation, and one SELD baselines. The results show that the proposed method is generic and applicable to any array structures, robust to unseen DOA values, reverberation, and low SNR scenarios. The proposed method achieved a consistently higher recall of the estimated number of DOAs across datasets in comparison to the best baseline. Additionally, this recall was observed to be significantly better than the best baseline method for a higher number of overlapping sound events.
---
paper_title: Maximum likelihood from incomplete data via the EM algorithm

paper_content:

Vibratory power unit for vibrating conveyers and screens comprising an asynchronous polyphase motor, at least one pair of associated unbalanced masses disposed on the shaft of said motor, with the first mass of a pair of said unbalanced masses being rigidly fastened to said shaft and with said second mass of said pair being movably arranged relative to said first mass, means for controlling and regulating the conveying rate during conveyer operation by varying the rotational speed of said motor between predetermined minimum and maximum values, said second mass being movably outwardly by centrifugal force against the pressure of spring means, said spring means being prestressed in such a manner that said second mass is, at rotational motor speeds lower than said minimum speed, held in its initial position, and at motor speeds between said lower and upper values in positions which are radially offset with respect to the axis of said motor to an extent depending on the value of said rotational motor speed.
---
paper_title: Compressed sensing

paper_content:

Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0<ples1. The N most important coefficients in that expansion allow reconstruction with lscr2 error O(N1/2-1p/). It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients. Moreover, a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing. The nonadaptive measurements have the character of "random" linear combinations of basis/frame elements. Our results use the notions of optimal recovery, of n-widths, and information-based complexity. We estimate the Gel'fand n-widths of lscrp balls in high-dimensional Euclidean space in the case 0<ples1, and give a criterion identifying near- optimal subspaces for Gel'fand n-widths. We show that "most" subspaces are near-optimal, and show that convex optimization (Basis Pursuit) is a near-optimal way to extract information derived from these near-optimal subspaces
---
paper_title: Distributed Expectation-Maximization Algorithm for Speaker Localization in Reverberant Environments

paper_content:

Localization of acoustic sources has attracted a considerable amount of research attention in recent years. A major obstacle to achieving high localization accuracy is the presence of reverberation, the influence of which obviously increases with the number of active speakers in the room. Human hearing is capable of localizing acoustic sources even in extreme conditions. In this study, we propose to combine a method based on human hearing mechanisms and a modified incremental distributed expectation-maximization (IDEM) algorithm. Rather than using phase difference measurements that are modeled by a mixture of complex-valued Gaussians, as proposed in the original IDEM framework, we propose to use time difference of arrival measurements in multiple subbands and model them by a mixture of real-valued truncated Gaussians. Moreover, we propose to first filter the measurements in order to reduce the effect of the multipath conditions. The proposed method is evaluated using both simulated data and real-life recordings.
---
paper_title: Tree-based recursive expectation-maximization algorithm for localization of acoustic sources

paper_content:

The problem of distributed localization for ad hoc wireless acoustic sensor networks (WASNs) is addressed in this paper. WASNs are characterized by low computational resources in each node and by limited connectivity between the nodes. Novel bi-directional tree-based distributed estimation--maximization (DEM) algorithms are proposed to circumvent these inherent limitations. We show that the proposed algorithms are capable of localizing static acoustic sources in reverberant enclosures without a priori information on the number of sources. Unlike serial estimation procedures (like ring-based algorithms), the new algorithms enable simultaneous computations in the nodes and exhibit greater robustness to communication failures. Specifically, the recursive distributed EM (RDEM) variant is better suited to online applications due to its recursive nature. Furthermore, the RDEM outperforms the other proposed variants in terms of convergence speed and simplicity. Performance is demonstrated by an extensive experimental study consisting of both simulated and actual environments.
---
paper_title: Localization of Multiple Simultaneously Active Speakers in an Acoustic Sensor Network

paper_content:

This paper addresses the localization of an unknown number of acoustic sources in an enclosure. We extend a well established algorithm for localization of acoustic sources, which is based on the Expectation Maximization (EM) algorithm for clustering of phase differences by a Gaussian mixture model. Supporting a more appropriate probabilistic model for spherical data such as direction of arrival or phase differences, the von Mises distribution is used to derive a localization algorithm for multiple simultaneously active sources. Experiments with simulated room impulse responses confirm the superiority of the proposed algorithm to the existing method in terms of localization performance.
---
paper_title: Online dictionary learning for sparse coding

paper_content:

Sparse coding---that is, modelling data vectors as sparse linear combinations of basis elements---is widely used in machine learning, neuroscience, signal processing, and statistics. This paper focuses on learning the basis set, also called dictionary, to adapt it to specific data, an approach that has recently proven to be very effective for signal reconstruction and classification in the audio and image processing domains. This paper proposes a new online optimization algorithm for dictionary learning, based on stochastic approximations, which scales up gracefully to large datasets with millions of training samples. A proof of convergence is presented, along with experiments with natural images demonstrating that it leads to faster performance and better dictionaries than classical batch algorithms for both small and large datasets.
---
paper_title: Sparse Bayesian learning for basis selection

paper_content:

Sparse Bayesian learning (SBL) and specifically relevance vector machines have received much attention in the machine learning literature as a means of achieving parsimonious representations in the context of regression and classification. The methodology relies on a parameterized prior that encourages models with few nonzero weights. In this paper, we adapt SBL to the signal processing problem of basis selection from overcomplete dictionaries, proving several results about the SBL cost function that elucidate its general behavior and provide solid theoretical justification for this application. Specifically, we have shown that SBL retains a desirable property of the /spl lscr//sub 0/-norm diversity measure (i.e., the global minimum is achieved at the maximally sparse solution) while often possessing a more limited constellation of local minima. We have also demonstrated that the local minima that do exist are achieved at sparse solutions. Later, we provide a novel interpretation of SBL that gives us valuable insight into why it is successful in producing sparse representations. Finally, we include simulation studies comparing sparse Bayesian learning with basis pursuit and the more recent FOCal Underdetermined System Solver (FOCUSS) class of basis selection algorithms. These results indicate that our theoretical insights translate directly into improved performance.
---
paper_title: A theory for multiresolution signal decomposition: the wavelet representation

paper_content:

Multiresolution representations are effective for analyzing the information content of images. The properties of the operator which approximates a signal at a given resolution were studied. It is shown that the difference of information between the approximation of a signal at the resolutions 2/sup j+1/ and 2/sup j/ (where j is an integer) can be extracted by decomposing this signal on a wavelet orthonormal basis of L/sup 2/(R/sup n/), the vector space of measurable, square-integrable n-dimensional functions. In L/sup 2/(R), a wavelet orthonormal basis is a family of functions which is built by dilating and translating a unique function psi (x). This decomposition defines an orthogonal multiresolution representation called a wavelet representation. It is computed with a pyramidal algorithm based on convolutions with quadrature mirror filters. Wavelet representation lies between the spatial and Fourier domains. For images, the wavelet representation differentiates several spatial orientations. The application of this representation to data compression in image coding, texture discrimination and fractal analysis is discussed. >
---
paper_title: Speaker Tracking Using Recursive EM Algorithms

paper_content:

The problem of localizing and tracking a known number of concurrent speakers in noisy and reverberant enclosures is addressed in this paper. We formulate the localization task as a maximum likelihood (ML) parameter estimation problem, and solve it by utilizing the expectation-maximization (EM) procedure. For the tracking scenario, we propose to adapt two recursive EM (REM) variants. The first, based on Titterington's scheme, is a Newton-based recursion. In this work we also extend Titterington's method to deal with constrained maximization, encountered in the problem at hand. The second is based on Cappe and Moulines' scheme. We discuss the similarities and dissimilarities of these two variants and show their applicability to the tracking problem by a simulated experimental study.
---
paper_title: A Scale for the Measurement of the Psychological Magnitude Pitch

paper_content:

A subjective scale for the measurement of pitch was constructed from determinations of the half‐value of pitches at various frequencies. This scale differs from both the musical scale and the frequency scale, neither of which is subjective. Five observers fractionated tones of 10 different frequencies at a loudness level of 60 db. From these fractionations a numerical scale was constructed which is proportional to the perceived magnitude of subjective pitch. In numbering the scale the 1000‐cycle tone was assigned the pitch of 1000 subjective units (mels).The close agreement of the pitch scale with an integration of the differential thresholds (DL's) shows that, unlike the DL's for loudness, all DL's for pitch are of uniform subjective magnitude. The agreement further implies that pitch and differential sensitivity to pitch are both rectilinear functions of extent on the basilar membrane.The correspondence of the pitch scale and the experimentally determined location of the resonant areas of the basilar me...
---
paper_title: Image method for efficiently simulating small‐room acoustics

paper_content:

Image methods are commonly used for the analysis of the acoustic properties of enclosures. In this paper we discuss the theoretical and practical use of image techniques for simulating, on a digital computer, the impulse response between two points in a small rectangular room. The resulting impulse response, when convolved with any desired input signal, such as speech, simulates room reverberation of the input signal. This technique is useful in signal processing or psychoacoustic studies. The entire process is carried out on a digital computer so that a wide range of room parameters can be studied with accurate control over the experimental conditions. A fortran implementation of this model has been included.
---
paper_title: Supervised source localization using diffusion kernels

paper_content:

Recently, we introduced a method to recover the controlling parameters of linear systems using diffusion kernels. In this paper, we apply our approach to the problem of source localization in a reverberant room using measurements from a single microphone. Prior recordings of signals from various known locations in the room are required for training and calibration. The proposed algorithm relies on a computation of a diffusion kernel with a specially-tailored distance measure. Experimental results in a real reverberant environment demonstrate accurate recovery of the source location.
---
paper_title: Relative transfer function modeling for supervised source localization

paper_content:

Speaker localization is one of the most prevalent problems in speech processing. Despite significant efforts in the last decades, high reverberation level still limits the performance of localization algorithms. Furthermore, using conventional localization methods, the information that can be extracted from dual microphone measurements is restricted to the time difference of arrival (TDOA). Under far-field regime, this is equivalent to either azimuth or elevation angles estimation. Full description of speaker's coordinates necessitates several microphones. In this contribution we tackle these two limitations by taking a manifold learning perspective for system identification. We present a training-based algorithm, motivated by the concept of diffusion maps, that aims at recovering the fundamental controlling parameters driving the measurements. This approach turns out to be more robust to reverberation, and capable of recovering the speech source location using merely two microphones signals.
---
paper_title: Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

paper_content:

A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by “learning without a teacher”, and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname “neocognitron”. After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel. The network consits of an input layer (photoreceptor array) followed by a cascade connection of a number of modular structures, each of which is composed of two layers of cells connected in a cascade. The first layer of each module consists of “S-cells”, which show characteristics similar to simple cells or lower order hypercomplex cells, and the second layer consists of “C-cells” similar to complex cells or higher order hypercomplex cells. The afferent synapses to each S-cell have plasticity and are modifiable. The network has an ability of unsupervised learning: We do not need any “teacher” during the process of self-organization, and it is only needed to present a set of stimulus patterns repeatedly to the input layer of the network. The network has been simulated on a digital computer. After repetitive presentation of a set of stimulus patterns, each stimulus pattern has become to elicit an output only from one of the C-cell of the last layer, and conversely, this C-cell has become selectively responsive only to that stimulus pattern. That is, none of the C-cells of the last layer responds to more than one stimulus pattern. The response of the C-cells of the last layer is not affected by the pattern's position at all. Neither is it affected by a small change in shape nor in size of the stimulus pattern.
---
paper_title: Playing billiards in the concert hall: The mathematical foundations of geometrical room acoustics

paper_content:

Abstract Billiard theory is emerging as a fundamental model for understanding the behaviour of complex physical systems. Yet it reduces to the analysis of one ball moving at constant velocity along a straight trajectory that experiences specular reflections on the billiard boundaries. Hence, the analogy between billiard trajectories and sound rays. This analogy makes it possible to derive ergodic properties for sound rays. Among these, the concept of mixing singles itself out as the key to a proper understanding of diffusion. Furthermore combining billiard theory with energy conservation allows one to estimate the average number of rays reaching a receiver at any time, thus casting some light on the role of hidden image sources. Billiard theory also casts some new light on the reverberation process; this is illustrated by means of a simple stochastical computer model.
---
paper_title: The generalized correlation method for estimation of time delay

paper_content:

A maximum likelihood (ML) estimator is developed for determining time delay between signals received at two spatially separated sensors in the presence of uncorrelated noise. This ML estimator can be realized as a pair of receiver prefilters followed by a cross correlator. The time argument at which the correlator achieves a maximum is the delay estimate. The ML estimator is compared with several other proposed processors of similar form. Under certain conditions the ML estimator is shown to be identical to one proposed by Hannan and Thomson [10] and MacDonald and Schultheiss [21]. Qualitatively, the role of the prefilters is to accentuate the signal passed to the correlator at frequencies for which the signal-to-noise (S/N) ratio is highest and, simultaneously, to suppress the noise power. The same type of prefiltering is provided by the generalized Eckart filter, which maximizes the S/N ratio of the correlator output. For low S/N ratio, the ML estimator is shown to be equivalent to Eckart prefiltering.
---
paper_title: A Hybrid Approach for Speaker Tracking Based on TDOA and Data-Driven Models

paper_content:

The problem of speaker tracking in noisy and reverberant enclosures is addressed in this paper. We present a hybrid algorithm, combining traditional tracking schemes with a new learning-based approach. A state-space representation, consisting of a propagation and observation models, is learned from signals measured by several distributed microphone pairs. The proposed representation is based on two data modalities corresponding to high-dimensional acoustic features representing the full reverberant acoustic channels as well as low-dimensional time difference of arrival (TDOA) estimates. The state-space representation is accompanied by a statistical model based on a Gaussian process used to relate the variations of the acoustic channels to the physical variations of the associated source positions, thereby forming a data-driven propagation model for the source movement. In the observation model, the source positions are nonlinearly mapped to the associated TDOA readings. The obtained propagation and observation models establish the basis for employing an extended Kalman filter. The simulation results demonstrate the robustness of the proposed method in noisy and reverberant conditions.
---
paper_title: Semi-Supervised Gaussian Process Classifiers

paper_content:

In this paper, we propose a graph-based construction of semi-supervised Gaussian process classifiers. Our method is based on recently proposed techniques for incorporating the geometric properties of unlabeled data within globally defined kernel functions. The full machinery for standard supervised Gaussian process inference is brought to bear on the problem of learning from labeled and unlabeled data. This approach provides a natural probabilistic extension to unseen test examples. We employ Expectation Propagation procedures for evidence-based model selection. In the presence of few labeled examples, this approach is found to significantly outperform cross-validation techniques. We present empirical results demonstrating the strengths of our approach.
---
paper_title: Speaker Tracking on Multiple-Manifolds with Distributed Microphones

paper_content:

Speaker tracking in a reverberant enclosure with an ad hoc network of multiple distributed microphones is addressed in this paper. A set of prerecorded measurements in the enclosure of interest is used to construct a data-driven statistical model. The function mapping the measurement-based features to the corresponding source position represents complex unknown relations, hence it is modelled as a random Gaussian process. The process is defined by a covariance function which encapsulates the relations among the available measurements and the different views presented by the distributed microphones. This model is intertwined with a Kalman filter to capture both the smoothness of the source movement in the time-domain and the smoothness with respect to patterns identified in the set of available prerecorded measurements. Simulation results demonstrate the ability of the proposed method to localize a moving source in reverberant conditions.
---
paper_title: Semi-Supervised Source Localization on Multiple Manifolds With Distributed Microphones

paper_content:

The problem of single-source localization with ad hoc microphone networks in noisy and reverberant enclosures is addressed in this paper. A training set is formed by prerecorded measurements collected in advance and consists of a limited number of labelled measurements, attached with corresponding positions, and a larger number of unlabelled measurements from unknown locations. Further information about the enclosure characteristics or the microphone positions is not required. We propose a Bayesian inference approach for estimating a function that maps measurement-based features to the corresponding positions. The signals measured by the microphones represent different viewpoints, which are combined in a unified statistical framework. For this purpose, the mapping function is modelled by a Gaussian process with a covariance function that encapsulates both the connections between pairs of microphones and the relations among the samples in the training set. The parameters of the process are estimated by optimizing a maximum likelihood criterion. In addition, a recursive adaptation mechanism is derived, where the new streaming measurements are used to update the model. Performance is demonstrated for both simulated data and real-life recordings in a variety of reverberation and noise levels.
---
paper_title: Sensitivity of matched field processing to sound‐speed profile mismatch for vertical arrays in a deep water Pacific environment

paper_content:

In this paper, the sensitivity of matched field processing to sound‐speed profile mismatch will be examined (based upon archival profiles resulting in various degrees of mismatch). A 10‐Hz source is considered whose field is generated by a normal mode model and only the water‐borne energy is used, thereby eliminating issues relating to the estimation of bottom parameters. This paper will examine how array parameters, i.e., number of phones and array depth, affect range and depth localization for various degrees of mismatch. In particular, it will be seen where an array is most and least sensitive to sound‐speed mismatch as a function of depth from the surface and range from the source, and the degree to which range and depth resolution are possible under ideal, as well as under likely, mismatch conditions.
---
paper_title: Environmental and system effects on source localization in shallow water by the matched‐field processing of a vertical array

paper_content:

This paper presents simulated and experimental results on source localization in shallow water by the matched‐field processing of a vertical array. Range‐depth ambiguity surfaces are obtained by the spatial correlation of the incident field (modeled or real) with a modeled replica of that field. All modeled fields are generated by the Centre’s normal‐mode program. The simulated results indicate that a high‐quality ambiguity surface can be obtained provided that the incident field is formed by a significant number of contributing modes, and that the array spans a large fraction of the water column. Changes in the acoustic environment that reduce the effective number of modes result in a degraded surface. The effect of mismatches between the ‘‘true’’ environmental parameters and those used to compute the replica field is also investigated. Errors in the bottom parameters do not severely effect the results, but small mismatches in sound‐speed profile or water depth can degrade the result significantly. Examp...
---
paper_title: Geoacoustic model inversion using artificial neural networks

paper_content:

An inversion technique using artificial neural networks (ANNs) is described for estimating geoacoustic model parameters of the ocean bottom and information about the sound source from acoustic field data. The method is applied to transmission loss data from the TRIAL SABLE experiment that was carried out in shallow water off Nova Scotia. The inversion is designed to incorporate the a priori information available for the site in order to improve the estimation accuracy. The inversion scheme involves training feedforward ANNs to estimate the geoacoustic and geometric parameters using simulated input/output training pairs generated with a forward acoustic propagation model. The inputs to the ANNs are the spectral components of the transmission loss at each sensor of a vertical hydrophone array for the two lowest frequencies that were transmitted in the experiment, 35 and 55 Hz. The output is the set of environmental model parameters, both geometric and geoacoustic, corresponding to the received field. In order to decrease the training time, a separate network was trained for each parameter. The errors for the parallel estimation are 10% lower than for those obtained using a single network to estimate all the parameters simultaneously, and the training time is decreased by a factor of six. When the experimental data are presented to the ANNs the geometric parameters, such as source range and depth, are estimated with a high accuracy. Geoacoustic parameters, such as the compressional speed in the sediment and the sediment thickness, are found with a moderate accuracy.
---
paper_title: Acoustic estimation of seafloor parameters: a Radial Basis Functions approach

paper_content:

A novel approach to the estimation of seafloor geoacoustic parameters from the measurement of the acoustic field in the water column is introduced. The approach is based on the idea of approximating the inverse function that links the geoacoustic parameters with the measured field through a series expansion of radial basis functions. In particular, Gaussian basis functions are used in order to ensure continuity and smoothness of the approximated inverse. The main advantage of the proposed approach relies on the fact that the series expansion can be computed off‐line from simulated data as soon as the experimental configuration is known. Data inversion can then be performed in true real time as soon as the data are acquired. Simulation results are presented in order to show the advantages and limitations of the method. Finally, some inversion results from horizontal towed array data are reported, and are compared with independent estimates of geoacoustic bottom properties.
---
paper_title: Theano: A Python framework for fast computation of mathematical expressions

paper_content:

Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models. ::: The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it.
---
paper_title: Bayesian focalization: quantifying source localization with environmental uncertainty.

paper_content:

This paper applies a Bayesian formulation to study ocean acoustic source localization as a function of uncertainty in environmental properties (water column and seabed) and of data information content [signal-to-noise ratio (SNR) and number of frequencies]. The approach follows that of the optimum uncertain field processor [A. M. Richardson and L. W. Nolte, J. Acoust. Soc. Am. 89, 2280-2284 (1991)], in that localization uncertainty is quantified by joint marginal probability distributions for source range and depth integrated over uncertain environmental properties. The integration is carried out here using Metropolis Gibbs' sampling for environmental parameters and heat-bath Gibbs' sampling for source location to provide efficient sampling over complicated parameter spaces. The approach is applied to acoustic data from a shallow-water site in the Mediterranean Sea where previous geoacoustic studies have been carried out. It is found that reliable localization requires a sufficient combination of prior (environmental) information and data information. For example, sources can be localized reliably for single-frequency data at low SNR (-3 dB) only with small environmental uncertainties, whereas successful localization with large environmental uncertainties requires higher SNR and/or multifrequency data.
---
paper_title: Stochastic Backpropagation and Approximate Inference in Deep Generative Models

paper_content:

We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as a stochastic encoder of the data. We develop stochastic back-propagation -- rules for back-propagation through stochastic variables -- and use this to develop an algorithm that allows for joint optimisation of the parameters of both the generative and recognition model. We demonstrate on several real-world data sets that the model generates realistic samples, provides accurate imputations of missing data and is a useful tool for high-dimensional data visualisation.
---
paper_title: Multiple source localization using a maximum a posteriori Gibbs sampling approach

paper_content:

Multiple source localization in underwater environments is approached within a matched-field processing framework. A maximum a posteriori estimation method is proposed that estimates source location and spectral characteristics of multiple sources via Gibbs sampling. The method facilitates localization of weak sources which are typically masked by the presence of strong interferers. A performance evaluation study based on Monte Carlo simulations shows that the proposed maximum a posteriori estimation approach is superior to simple coherent matched-field interference cancellation. The proposed method is also tested on the estimation of the number of sources present, providing probability distributions in addition to point estimates for the number of sources.
---
paper_title: The relation between the waveguide invariant and array invariant

paper_content:

The waveguide invariant β is based on the dependence of group speed on phase speed and summarizes the robust interference phenomenon in the range-frequency plane. Over the last decade the elegant approach has been utilized for various applications including passive source ranging. Separately, the array invariant approach [Lee and Makris, J. Acoust. Soc. Am., vol. 119, pp. 336–351 (2006)] has been proposed for a robust source-range estimator from beam-time intensity data using either a horizontal or vertical array. In this letter, it is shown that the array invariant can be derived from the waveguide invariant theory assuming β = 1.
---
paper_title: The array invariant

paper_content:

A method is derived for instantaneous source-range estimation in a horizontally stratified ocean waveguide from passive beam-time intensity data obtained after conventional plane-wave beamforming of acoustic array measurements. The method has advantages over existing source localization methods, such as matched field processing or the waveguide invariant. First, no knowledge of the environment is required except that the received field should not be dominated by purely waterborne propagation. Second, range can be estimated in real time with little computational effort beyond plane-wave beamforming. Third, array gain is fully exploited. The method is applied to data from the Main Acoustic Clutter Experiment of 2003 for source ranges between 1to8km, where it is shown that simple, accurate, and computationally efficient source range estimates can be made.
---
paper_title: Source ranging with minimal environmental information using a virtual receiver and waveguide invariant theory.

paper_content:

A method is presented for estimating the range of an unknown broadband acoustic source in a waveguide, using a vertical array and a signal sample from another broadband source at a known location relative to the array. The method requires no modeling of the acoustic field, and little to no environmental information for flat bathymetries. Waveguide invariant theory [e.g., D'Spain and Kuperman, J. Acoust. Soc. Am 106, 2454-2468 (1999)] is applied to the "virtual receiver" [Siderius et al., J. Acoust. Soc. Am. 102, 3439-3449 (1997)] to create a "virtual aperture" (VA). In effect, the method effectively converts a source at known range r(g) into a continuum of receivers lying between ranges (1 +/- alpha/beta)*r(g), where beta is a scalar parameter called the acoustic invariant, and alpha approximately 0.1. This effective displacement is achieved by correlating the known source field, measured at frequency component omega, with the unknown source field, measured at frequency component omega + omegas. When the VA output is plotted as a function of omega and omegas, the slope of the resulting correlation contours yields the unknown source range. The concept is illustrated via both simulation and analysis of data collected from a pseudo-random noise source with 75-150-Hz bandwidth during SWellEx-3, a shallow water experiment conducted off the San Diego coast. The virtual aperture can be reformulated for range-dependent environments, if adiabatic propagation assumptions are valid, and if the bathymetry surrounding the array is known.
---
paper_title: Underwater acoustic source localization using generalized regression neural network.

paper_content:

In this paper, machine learning is introduced to source localization in underwater ocean waveguides. Source localization is regarded as a supervised learning regression problem and is solved by generalized regression neural network (GRNN). As a feed-forward network, GRNN is built using training data with fixed structure and configuration. The normalized sample covariance matrix (SCM) formed over a number of snapshots, and the corresponding source position are used as the input and output for GRNN. The source position can be estimated directly from the normalized SCM with GRNN; the proposed approach is thus in theory data driven. In addition, there is only one parameter, the spread factor, to be learned for GRNN. The optimal spread factor is determined using cross-validation. The regression method of GRNN is compared with the classification method of feed-forward neural network (FNN), as well as the classical method of matched field processing (MFP) for vertical array data from the SWellEx-96 experiment. The results show that GRNN achieves a satisfactory localization performance that outperforms both FNN and MFP. The proposed approach provides an alternative way for underwater source localization, especially in the absence of a priori environmental information or an appropriate propagation model.
---
paper_title: Supervised Machine Learning for Estimation of Target Aspect Angle From Bistatic Acoustic Scattering

paper_content:

When an aspect-dependent target is insonified by an acoustic source, distinct features are produced in the resulting bistatic scattered field. These features change as the aspect between the source and the target is varied. This paper describes the use of these features for estimation of the target aspect angle using data collected by an autonomous underwater vehicle (AUV). An experiment was conducted in November 2014 in Massachusetts Bay to collect data using a ship-based acoustic source producing 7–9-kHz linear frequency modulation (LFM) chirps insonifying a steel pipe. The true target orientation was unknown, as the target was dropped from the ship with no rotation control. The AUV Unicorn , fitted with a 16-element nose array, was deployed in data collection behaviors around the target, and the ship was moved to create two target aspects. A support vector machine regression model was trained using simulated scattering bistatic field data. This model was then used to estimate the target aspect angle from the data collected during the experiment. The difference between the estimates was consistent with experimental observations of relative source positioning. The simulation-based model appeared successful in estimating the target aspect angle despite uncertainties in target and source location and mismatch between true environment and simulation parameters.
---
paper_title: Sound Source Localization in a Multipath Environment Using Convolutional Neural Networks

paper_content:

The propagation of sound in a shallow water environment is characterized by boundary reflections from the sea surface and sea floor. These reflections result in multiple (indirect) sound propagation paths, which can degrade the performance of passive sound source localization methods. This paper proposes the use of convolutional neural networks (CNNs) for the localization of sources of broadband acoustic radiated noise (such as motor vessels) in shallow water multipath environments. It is shown that CNNs operating on cepstrogram and generalized cross-correlogram inputs are able to more reliably estimate the instantaneous range and bearing of transiting motor vessels when the source localization performance of conventional passive ranging methods is degraded. The ensuing improvement in source localization performance is demonstrated using real data collected during an at-sea experiment.
---
paper_title: Ship localization in Santa Barbara Channel using machine learning classifiers

paper_content:

Machine learning classifiers are shown to outperform conventional matched field processing for a deep water (600 m depth) ocean acoustic-based ship range estimation problem in the Santa Barbara Channel Experiment when limited environmental information is known. Recordings of three different ships of opportunity on a vertical array were used as training and test data for the feed-forward neural network and support vector machine classifiers, demonstrating the feasibility of machine learning methods to locate unseen sources. The classifiers perform well up to 10 km range whereas the conventional matched field processing fails at about 4 km range without accurate environmental information.
---
paper_title: Convolutional Neural Networks for Passive Monitoring of a Shallow Water Environment using a Single Sensor

paper_content:

A cost effective approach to remote monitoring of protected areas such as marine reserves and restricted naval waters is to use passive sonar to detect, classify, localize, and track marine vessel activity (including small boats and autonomous underwater vehicles). Cepstral analysis of underwater acoustic data enables the time delay between the direct path arrival and the first multipath arrival to be measured, which in turn enables estimation of the instantaneous range of the source (a small boat). However, this conventional method is limited to ranges where the Lloyd's mirror effect (interference pattern formed between the direct and first multipath arrivals) is discernible. This paper proposes the use of convolutional neural networks (CNNs) for the joint detection and ranging of broadband acoustic noise sources such as marine vessels in conjunction with a data augmentation approach for improving network performance in varied signal-to-noise ratio (SNR) situations. Performance is compared with a conventional passive sonar ranging method for monitoring marine vessel activity using real data from a single hydrophone mounted above the sea floor. It is shown that CNNs operating on cepstrum data are able to detect the presence and estimate the range of transiting vessels at greater distances than the conventional method.
---
paper_title: Source localization using deep neural networks in a shallow water environment.

paper_content:

Deep neural networks (DNNs) are advantageous for representing complex nonlinear relationships. This paper applies DNNs to source localization in a shallow water environment. Two methods are proposed to estimate the range and depth of a broadband source through different neural network architectures. The first adopts the classical two-stage scheme, in which feature extraction and DNN analysis are independent steps. The eigenvectors associated with the modal signal space are extracted as the input feature. Then, the time delay neural network is exploited to model the long term feature representation and constructs the regression model. The second concerns a convolutional neural network-feed-forward neural network (CNN-FNN) architecture, which trains the network directly by taking the raw multi-channel waveforms as input. The CNNs are expected to perform spatial filtering for multi-channel signals, in an operation analogous to time domain filters. The outputs of CNNs are summed as the input to FNN. Several experiments are conducted on the simulated and experimental data to evaluate the performance of the proposed methods. The results demonstrate that DNNs are effective for source localization in complex and varied water environments, especially when there is little precise environmental information.
---
paper_title: Deep Residual Learning for Image Recognition

paper_content:

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers—8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.
---
paper_title: Deep learning for ocean acoustic source localization using one sensor

paper_content:

A deep learning approach based on big data is proposed to locate broadband acoustic sources with one hydrophone in ocean waveguides. Several 50-layer residual neural networks, trained on a huge number of sound field replicas generated by an acoustic propagation model, are used to handle the environmental uncertainty in source localization. A two-step training strategy is presented to improve the training of the deep models. First, the range is discretized in a coarse (5 km) grid. Subsequently, the source range within the selected interval and source depth are discretized on a finer (0.1 km and 2 m) grid. The deep learning methods were tested for simulated data in uncertain environments and on experimental data. In both tests, the deep learning methods successfully estimated source ranges and depths in uncertain environments.
---
paper_title: Source localization using deep neural networks in a shallow water environment.

paper_content:

Deep neural networks (DNNs) are advantageous for representing complex nonlinear relationships. This paper applies DNNs to source localization in a shallow water environment. Two methods are proposed to estimate the range and depth of a broadband source through different neural network architectures. The first adopts the classical two-stage scheme, in which feature extraction and DNN analysis are independent steps. The eigenvectors associated with the modal signal space are extracted as the input feature. Then, the time delay neural network is exploited to model the long term feature representation and constructs the regression model. The second concerns a convolutional neural network-feed-forward neural network (CNN-FNN) architecture, which trains the network directly by taking the raw multi-channel waveforms as input. The CNNs are expected to perform spatial filtering for multi-channel signals, in an operation analogous to time domain filters. The outputs of CNNs are summed as the input to FNN. Several experiments are conducted on the simulated and experimental data to evaluate the performance of the proposed methods. The results demonstrate that DNNs are effective for source localization in complex and varied water environments, especially when there is little precise environmental information.
---
paper_title: Methods for automatic detection of mysticete sounds

paper_content:

Methods for the automatic recognition of low‐frequency sounds of baleen whales are presented. Matched filtering is implemented with a synthetic filter kernel derived from measurements of whale sounds, and this method is found effective at detecting blue whale (Balaenoptera musculus) sounds in white background noise. Spectrogram correlation is implemented and found effective at detection of blue whale vocalizations in the presence of interfering sounds. Spectrogram correlation employs image correlation using spectrograms and carefully designed image kernels to give good performance in the presence of noise. The two methods are briefly compared. The effectiveness of the spectrogram correlator is also demonstrated with finback whale (B. physalus) sounds. A supplementary method for detecting regular, repetitive sequences of sounds is applied to minke whale (B. acutorostrata) pulse trains, and found to improve detection in conditions of high ambient noise.
---
paper_title: Acoustic sequences in non-human animals: a tutorial review and prospectus

paper_content:

This review was developed at an investigative workshop, “Analyzing Animal Vocal Communication Sequences” that took place on October 21–23 2013 in Knoxville, Tennessee, sponsored by the National Institute for Mathematical and Biological Synthesis (NIMBioS). NIMBioS is an Institute sponsored by the National Science Foundation, the U.S. Department of Homeland Security, and the U.S. Department of Agriculture through NSF Awards #EF-0832858 and #DBI-1300426, with additional support from The University of Tennessee, Knoxville. In addition to the authors, Vincent Janik participated in the workshop. D.T.B.’s research is currently supported by NSF DEB-1119660. M.A.B.’s research is currently supported by NSF IOS-0842759 and NIH R01DC009582. M.A.R.’s research is supported by ONR N0001411IP20086 and NOPP (ONR/BOEM) N00014-11-1-0697. S.L.DeR.’s research is supported by the U.S. Office of Naval Research. R.F.-i-C.’s research was supported by the grant BASMATI (TIN2011-27479-C04-03) from the Spanish Ministry of Science and Innovation. E.C.G.’s research is currently supported by a National Research Council postdoctoral fellowship. E.E.V.’s research is supported by CONACYT, Mexico, award number I010/214/2012.
---
paper_title: Estimating cetacean population density using fixed passive acoustic sensors: An example with Blainville’s beaked whales

paper_content:

Methods are developed for estimating the size/density of cetacean populations using data from a set of fixed passive acoustic sensors. The methods convert the number of detected acoustic cues into animal density by accounting for (i) the probability of detecting cues, (ii) the rate at which animals produce cues, and (iii) the proportion of false positive detections. Additional information is often required for estimation of these quantities, for example, from an acoustic tag applied to a sample of animals. Methods are illustrated with a case study: estimation of Blainville’s beaked whale density over a 6day period in spring 2005, using an 82 hydrophone wide-baseline array located in the Tongue of the Ocean, Bahamas. To estimate the required quantities, additional data are used from digital acoustic tags, attached to five whales over 21 deep dives, where cues recorded on some of the dives are associated with those received on the fixed hydrophones. Estimated density was 25.3 or 22.5 animals/1000km2, depend...
---
paper_title: Species-specific differences in pure tonal whistle vocalizations of five western North Atlantic dolphin species

paper_content:

Pure tonal ‘whistle’ vocalizations from five species of dolphins found in the western North Atlantic had consistent, species-specific characteristics. The degree of differences between species, as based on the results of multivariate discriminant analysis (Fig. 2), correlated with the taxonomic and zoogeographic relations of the five dolphin species. Congeneric species had more similar vocalizations than species of different genera. Differences between sympatric species were greater than differences between allopatric species. Of the six whistle parameters measured, maximum frequency had the lowest coefficient of variation for all five species, and duration and number of inflection points had the highest coefficients of variation for all five species.
---
paper_title: Automated categorization of bioacoustic signals: Avoiding perceptual pitfalls

paper_content:

Dividing the acoustic repertoires of animals into biologically relevant categories presents a widespread problem in the study of animal sound communication, essential to any comparison of repertoires between contexts, individuals, populations, or species. Automated procedures allow rapid, repeatable, and objective categorization, but often perform poorly at detecting biologically meaningful sound classes. Arguably this is because many automated methods fail to address the nonlinearities of animal sound perception. We present a new method of categorization that incorporates dynamic time-warping and an adaptive resonance theory (ART) neural network. This method was tested on 104 randomly chosen whistle contours from four captive bottlenose dolphins (Tursiops truncatus), as well as 50 frequency contours extracted from calls of transient killer whales (Orcinus orca). The dolphin data included known biologically meaningful categories in the form of 42 stereotyped whistles produced when each individual was isol...
---
paper_title: Classification of mysticete sounds using machine learning techniques

paper_content:

Automatic classification of mysticete sounds has long been a challenging task in the bioacoustics field. The unknown statistical properties of the signals as well as the use of different recording apparatus and low signal-to-noise ratio conditions often lead to non-optimal systems. The goal of this paper is to design methods for the automatic classification of mysticete sounds using a restricted Boltzmann machine and a sparse auto-encoder that are widely used in the field of artificial intelligence. Experiments on five species of mysticetes are presented. The different methods are employed on the subset of species whose frequency range overlaps, as well as in all five species' calls. Moreover, results are offered with and without the use of a noise class. Overall, the systems are able to achieve an average classification accuracy of over 69% (with noise) and 80% (without noise) given the different architectures.
---
paper_title: Dynamic programming algorithm optimization for spoken word recognition

paper_content:

This paper reports on an optimum dynamic progxamming (DP) based time-normalization algorithm for spoken word recognition. First, a general principle of time-normalization is given using time-warping function. Then, two time-normalized distance definitions, called symmetric and asymmetric forms, are derived from the principle. These two forms are compared with each other through theoretical discussions and experimental studies. The symmetric form algorithm superiority is established. A new technique, called slope constraint, is successfully introduced, in which the warping function slope is restricted so as to improve discrimination between words in different categories. The effective slope constraint characteristic is qualitatively analyzed, and the optimum slope constraint condition is determined through experiments. The optimized algorithm is then extensively subjected to experimental comparison with various DP-algorithms, previously applied to spoken word recognition by different research groups. The experiment shows that the present algorithm gives no more than about two-thirds errors, even compared to the best conventional algorithm.
---
paper_title: Marine mammal call discrimination using artificial neural networks.

paper_content:

Recent work has applied a linear spectrogram correlator filter (SCF) to detect bowhead whale (Balaena mysticetus) song notes, outperforming both a time-series-matched filter and a hidden Markov model. The method relies on an empirical weighting matrix. An artificial neural net (ANN) may be better yet, since it offers two advantages; (i) the equivalent weighting matrix is determined by training and can converge to a more optimal solution and (ii) an ANN is a nonlinear estimator and can embody more sophisticated responses. A three-layer feed-forward ANN is ideally suited to this application and has been implemented on 1475 sounds, of which 54% were used for training and 46% kept as "unseen" test data. The trained ANN error rate was 1.5%, a twofold improvement over previous methods. It is shown that ANN hidden neurons can be interrogated to reveal the operating paradigm developed during training. The function of each of these neurons can be determined in terms of spectrographic features of the training calls. Furthermore, the operating paradigm can be controlled and training time reduced by assigning specific recognition tasks to hidden neurons prior to training, rather than initiating training with randomized weights. The ANN is compared to the SCF and the role of the "hidden" neurons and equivalent weighting matrices are discussed.
---
paper_title: A Scale for the Measurement of the Psychological Magnitude Pitch

paper_content:

A subjective scale for the measurement of pitch was constructed from determinations of the half‐value of pitches at various frequencies. This scale differs from both the musical scale and the frequency scale, neither of which is subjective. Five observers fractionated tones of 10 different frequencies at a loudness level of 60 db. From these fractionations a numerical scale was constructed which is proportional to the perceived magnitude of subjective pitch. In numbering the scale the 1000‐cycle tone was assigned the pitch of 1000 subjective units (mels).The close agreement of the pitch scale with an integration of the differential thresholds (DL's) shows that, unlike the DL's for loudness, all DL's for pitch are of uniform subjective magnitude. The agreement further implies that pitch and differential sensitivity to pitch are both rectilinear functions of extent on the basilar membrane.The correspondence of the pitch scale and the experimentally determined location of the resonant areas of the basilar me...
---
paper_title: A quantitative measure of similarity for tursiops truncatus signature whistles.

paper_content:

Bottlenose dolphins (tursiops truncatus) produce individually distinctive narrow‐band ‘‘signature whistles.’’ These whistles may be differentiated by the structure of their frequency contours. An algorithm is presented for extracting frequency contours from whistles and comparing two such contours. This algorithm performs nonuniform time dilation to align the contours and provides a quantitative distance measure between the contours. Two recognition experiments using the algorithm on three dolphin whistles from each of five individuals classified 15 out of 15 single‐loop whistles correctly, and 14 out of 15 central loops for multiple‐loop whistles correctly.
---
paper_title: Worldwide decline in tonal frequencies of blue whale songs

paper_content:

Blue whale Balaenoptera musculus songs can be divided into at least 10 types world- wide, each type retaining the same units and similar phrasing over decades, unlike humpback whale song which changes substantially from year to year. Historical acoustic recordings dating back as far as the 1960s were examined, measuring the tonal frequencies of 1000s of blue whale songs. Within a given year, individuals match the song frequency (related to 'pitch' in musical nomenclature) to within less than 3%. The best documented song type, that observed offshore of California, USA, now is sung at a frequency 31% lower than it was in the 1960s. Data available for 7 of the world's 10 known song types show they are all shifting downward in frequency, though at different rates. Any behavioral, ecological, oceanographic or anthropogenic change hypothesis seeking to explain the observed shifts should account for the worldwide occurrence of a nearly linear downward shift in the tonal frequencies of blue whale song. Hypotheses examined consider sexual selection, increasing ocean noise, increasing whale body size post whaling, global warming, interference from other ani- mal sounds and post whaling increases in abundance. None of the commonly suggested hypotheses were found to provide a full explanation; however, increasing population size post whaling provides an intriguing and testable hypothesis that recovery is altering the sexually selected tradeoff for singing males between song amplitude (the ability to be heard at a greater distance) and song frequency (the ability to produce songs of lower pitch).
---
paper_title: Methods for automatic detection of mysticete sounds

paper_content:

Methods for the automatic recognition of low‐frequency sounds of baleen whales are presented. Matched filtering is implemented with a synthetic filter kernel derived from measurements of whale sounds, and this method is found effective at detecting blue whale (Balaenoptera musculus) sounds in white background noise. Spectrogram correlation is implemented and found effective at detection of blue whale vocalizations in the presence of interfering sounds. Spectrogram correlation employs image correlation using spectrograms and carefully designed image kernels to give good performance in the presence of noise. The two methods are briefly compared. The effectiveness of the spectrogram correlator is also demonstrated with finback whale (B. physalus) sounds. A supplementary method for detecting regular, repetitive sequences of sounds is applied to minke whale (B. acutorostrata) pulse trains, and found to improve detection in conditions of high ambient noise.
---
paper_title: Automated recognition of bird song elements from continuous recordings using dynamic time warping and hidden Markov models: a comparative study.

paper_content:

The performance of two techniques is compared for automated recognition of bird song units from continuous recordings. The advantages and limitations of dynamic time warping (DTW) and hidden Markov models (HMMs) are evaluated on a large database of male songs of zebra finches (Taeniopygia guttata) and indigo buntings (Passerina cyanea), which have different types of vocalizations and have been recorded under different laboratory conditions. Depending on the quality of recordings and complexity of song, the DTW-based technique gives excellent to satisfactory performance. Under challenging conditions such as noisy recordings or presence of confusing short-duration calls, good performance of the DTW-based technique requires careful selection of templates that may demand expert knowledge. Because HMMs are trained, equivalent or even better performance of HMMs can be achieved based only on segmentation and labeling of constituent vocalizations, albeit with many more training examples than DTW templates. One weakness in HMM performance is the misclassification of short-duration vocalizations or song units with more variable structure (e.g., some calls, and syllables of plastic songs). To address these and other limitations, new approaches for analyzing bird vocalizations are discussed.
---
paper_title: Parametric Representations of Bird Sounds for Automatic Species Recognition

paper_content:

This paper is related to the development of signal processing techniques for automatic recognition of bird species. Three different parametric representations are compared. The first representation is based on sinusoidal modeling which has been earlier found useful for highly tonal bird sounds. Mel-cepstrum parameters are used since they have been found very useful in the parallel problem of speech recognition. Finally, a vector of various descriptive features is tested because such models are popular in audio classification applications, and bird song is almost like music. We briefly introduce the methods and evaluate their performance in the classification and recognition of both individual syllables and song fragments of 14 common North-European Passerine bird species
---
paper_title: Automatic large-scale classification of bird sounds is strongly improved by unsupervised feature learning

paper_content:

Automatic species classification of birds from their sound is a computational tool of increasing importance in ecology, conservation monitoring and vocal communication studies. To make classification useful in practice, it is crucial to improve its accuracy while ensuring that it can run at big data scales. Many approaches use acoustic measures based on spectrogram-type data, such as the Mel-frequency cepstral coefficient (MFCC) features which represent a manually-designed summary of spectral information. However, recent work in machine learning has demonstrated that features learnt automatically from data can often outperform manually-designed feature transforms. Feature learning can be performed at large scale and “unsupervised”, meaning it requires no manual data labelling, yet it can improve performance on “supervised” tasks such as classification. In this work we introduce a technique for feature learning from large volumes of bird sound recordings, inspired by techniques that have proven useful in other domains. We experimentally compare twelve different feature representations derived from the Mel spectrum (of which six use this technique), using four large and diverse databases of bird vocalisations, classified using a random forest classifier. We demonstrate that in our classification tasks, MFCCs can often lead to worse performance than the raw Mel spectral data from which they are derived. Conversely, we demonstrate that unsupervised feature learning provides a substantial boost over MFCCs and Mel spectra without adding computational complexity after the model has been trained. The boost is particularly notable for single-label classification tasks at large scale. The spectro-temporal activations learned through our procedure resemble spectro-temporal receptive fields calculated from avian primary auditory forebrain. However, for one of our datasets, which contains substantial audio data but few annotations, increased performance is not discernible. We study the interaction between dataset characteristics and choice of feature representation through further empirical analysis.
---
paper_title: Gaussian mixture model classification of odontocetes in the Southern California Bight and the Gulf of California

paper_content:

A method for the automatic classification of free-ranging delphinid vocalizations is presented. The vocalizations of short-beaked and long-beaked common (Delphinus delphis and Delphinus capensis), Pacific white-sided (Lagenorhynchus obliquidens), and bottlenose (Tursiops truncatus) dolphins were recorded in a pelagic environment of the Southern California Bight and the Gulf of California over a period of 4 years. Cepstral feature vectors are extracted from call data which contain simultaneous overlapping whistles, burst-pulses, and clicks from a single species. These features are grouped into multisecond segments. A portion of the data is used to train Gaussian mixture models of varying orders for each species. The remaining call data are used to test the performance of the models. Species are predicted based upon probabilistic measures of model similarity with test segment groups having durations between 1 and 25s. For this data set, 256 mixture Gaussian mixture models and segments of at least 10s of cal...
---
paper_title: Acoustic identification of twelve species of echolocating bat by discriminant function analysis and artificial neural networks.

paper_content:

We recorded echolocation calls from 14 sympatric species of bat in Britain. Once digitised, one temporal and four spectral features were measured from each call. The frequency-time course of each call was approximated by fitting eight mathematical functions, and the goodness of fit, represented by the mean-squared error, was calculated. Measurements were taken using an automated process that extracted a single call from background noise and measured all variables without intervention. Two species of Rhinolophus were easily identified from call duration and spectral measurements. For the remaining 12 species, discriminant function analysis and multilayer back-propagation perceptrons were used to classify calls to species level. Analyses were carried out with and without the inclusion of curve-fitting data to evaluate its usefulness in distinguishing among species. Discriminant function analysis achieved an overall correct classification rate of 79% with curve-fitting data included, while an artificial neural network achieved 87%. The removal of curve-fitting data improved the performance of the discriminant function analysis by 2 %, while the performance of a perceptron decreased by 2 %. However, an increase in correct identification rates when curve-fitting information was included was not found for all species. The use of a hierarchical classification system, whereby calls were first classified to genus level and then to species level, had little effect on correct classification rates by discriminant function analysis but did improve rates achieved by perceptrons. This is the first published study to use artificial neural networks to classify the echolocation calls of bats to species level. Our findings are discussed in terms of recent advances in recording and analysis technologies, and are related to factors causing convergence and divergence of echolocation call design in bats.
---
paper_title: Characterizing Acoustic Features of Marine Animal Sounds

paper_content:

Funding was provided by NAVSEA under Contract No. N00140-90-D-1979 and a series of ::: contracts and grants by ONR including Grant N00014-91-J-1445 with supplemental support by ::: NOARL and ORINCON/DARPA.
---
paper_title: From frequency to quefrency: a history of the cepstrum

paper_content:

The idea of the log spectrum or cepstral averaging has been useful in many applications such as audio processing, speech processing, speech recognition, and echo detection for the estimation and compensation of convolutional distortions. To suggest what prompted the invention of the term cepstrum, this article narrates the historical and mathematical background that led to its discovery. The computations of earlier simple echo representations have shown that the spectrum representation domain results does not belong in the frequency or time domain. Bogert et al. (1963) chose to refer to it as quefrency domain and later termed the spectrum of the log of a time waveform as the cepstrum. The article also recounts the analysis of Al Oppenheim in relation to the cepstrum. It was in his theory for nonlinear signal processing, referred to as homomorphic systems, that the realization of the characteristic system of homomorphic convolution was reminiscent of the cepstrum. To retain both the relationship to the work of Bogart et al. and the distinction, the term power cepstrum was eventually applied to the nonlinear mapping in homomorphic deconvolution . While most of the terms in the glossary have faded into the background, the term cepstrum has survived and has become part of the digital signal processing lexicon.
---
paper_title: Classification of echolocation clicks from odontocetes in the Southern California Bight

paper_content:

This study presents a system for classifying echolocation clicks of six species of odontocetes in the Southern California Bight: Visually confirmed bottlenose dolphins, short- and long-beaked common dolphins, Pacific white-sided dolphins, Risso’s dolphins, and presumed Cuvier’s beaked whales. Echolocation clicks are represented by cepstral feature vectors that are classified by Gaussian mixture models. A randomized cross-validation experiment is designed to provide conditions similar to those found in a field-deployed system. To prevent matched conditions from inappropriately lowering the error rate, echolocation clicks associated with a single sighting are never split across the training and test data. Sightings are randomly permuted before assignment to folds in the experiment. This allows different combinations of the training and test data to be used while keeping data from each sighting entirely in the training or test set. The system achieves a mean error rate of 22% across 100 randomized three-fold...
---
paper_title: Species-specific differences in pure tonal whistle vocalizations of five western North Atlantic dolphin species

paper_content:

Pure tonal ‘whistle’ vocalizations from five species of dolphins found in the western North Atlantic had consistent, species-specific characteristics. The degree of differences between species, as based on the results of multivariate discriminant analysis (Fig. 2), correlated with the taxonomic and zoogeographic relations of the five dolphin species. Congeneric species had more similar vocalizations than species of different genera. Differences between sympatric species were greater than differences between allopatric species. Of the six whistle parameters measured, maximum frequency had the lowest coefficient of variation for all five species, and duration and number of inflection points had the highest coefficients of variation for all five species.
---
paper_title: Bat Detective - Deep Learning Tools for Bat Acoustic Signal Detection

paper_content:

Passive acoustic sensing has emerged as a powerful tool for quantifying anthropogenic impacts on biodiversity, especially for echolocating bat species. To better assess bat population trends there is a critical need for accurate, reliable, and open source tools that allow the detection and classification of bat calls in large collections of audio recordings. The majority of existing tools are commercial or have focused on the species classification task, neglecting the important problem of first localizing echolocation calls in audio which is particularly problematic in noisy recordings. We developed a convolutional neural network (CNN) based open-source pipeline for detecting ultrasonic, full-spectrum, search-phase calls produced by echolocating bats (BatDetect). Our deep learning algorithms (CNN FULL and CNN FAST) were trained on full-spectrum ultrasonic audio collected along road-transects across Romania and Bulgaria by citizen scientists as part of the iBats programme and labelled by users of www.batdetective.org. We compared the performance of our system to other algorithms and commercial systems on expert verified test datasets recorded from different sensors and countries. As an example application, we ran our detection pipeline on iBats monitoring data collected over five years from Jersey (UK), and compared results to a widely-used commercial system. Here, we show that both CNNFULL and CNNFAST deep learning algorithms have a higher detection performance (average precision, and recall) of search-phase echolocation calls with our test sets, when compared to other existing algorithms and commercial systems tested. Precision scores for commercial systems were reasonably good across all test datasets (>0.7), but this was at the expense of recall rates. In particular, our deep learning approaches were better at detecting calls in road-transect data, which contained more noisy recordings. Our comparison of CNNFULL and CNNFAST algorithms was favourable, although CNNFAST had a slightly poorer performance, displaying a trade-off between speed and accuracy. Our example monitoring application demonstrated that our open-source, fully automatic, BatDetect CNNFAST pipeline does as well or better compared to a commercial system with manual verification previously used to analyse monitoring data. We show that it is possible to both accurately and automatically detect bat search-phase echolocation calls, particularly from noisy audio recordings. Our detection pipeline enables the automatic detection and monitoring of bat populations, and further facilitates their use as indicator species on a large scale, particularly when combined with automatic species identification. We release our system and datasets to encourage future progress and transparency.
---
paper_title: Classification of mysticete sounds using machine learning techniques

paper_content:

Automatic classification of mysticete sounds has long been a challenging task in the bioacoustics field. The unknown statistical properties of the signals as well as the use of different recording apparatus and low signal-to-noise ratio conditions often lead to non-optimal systems. The goal of this paper is to design methods for the automatic classification of mysticete sounds using a restricted Boltzmann machine and a sparse auto-encoder that are widely used in the field of artificial intelligence. Experiments on five species of mysticetes are presented. The different methods are employed on the subset of species whose frequency range overlaps, as well as in all five species' calls. Moreover, results are offered with and without the use of a noise class. Overall, the systems are able to achieve an average classification accuracy of over 69% (with noise) and 80% (without noise) given the different architectures.
---
paper_title: Random Forests

paper_content:

Random forests are a combination of tree predictors such that each tree depends on the values of a random vector sampled independently and with the same distribution for all trees in the forest. The generalization error for forests converges a.s. to a limit as the number of trees in the forest becomes large. The generalization error of a forest of tree classifiers depends on the strength of the individual trees in the forest and the correlation between them. Using a random selection of features to split each node yields error rates that compare favorably to Adaboost (Y. Freund & R. Schapire, Machine Learning: Proceedings of the Thirteenth International conference, aaa, 148–156), but are more robust with respect to noise. Internal estimates monitor error, strength, and correlation and these are used to show the response to increasing the number of features used in the splitting. Internal estimates are also used to measure variable importance. These ideas are also applicable to regression.
---
paper_title: Boosting the margin: A new explanation for the effectiveness of voting methods

paper_content:

One of the surprising recurring phenomena observed in experiments with boosting is that the test error of the generated classifier usually does not increase as its size becomes very large, and often is observed to decrease even after the training error reaches zero. In this paper, we show that this phenomenon is related to the distribution of margins of the training examples with respect to the generated voting classification rule, where the margin of an example is simply the difference between the number of correct votes and the maximum number of votes received by any incorrect label. We show that techniques used in the analysis of Vapnik's support vector classifiers and of neural networks with small weights can be applied to voting methods to relate the margin distribution to the test error. We also show theoretically and experimentally that boosting is especially effective at increasing the margins of the training examples. Finally, we compare our explanation to those based on the bias-variance
---
paper_title: Deep learning

paper_content:

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
---
paper_title: Chinese Whispers - An Efficient Graph Clustering Algorithm And Its Application To Natural Language Processing Problems

paper_content:

We introduce Chinese Whispers, a randomized graph-clustering algorithm, which is time-linear in the number of edges. After a detailed definition of the algorithm and a discussion of its strengths and weaknesses, the performance of Chinese Whispers is measured on Natural Language Processing (NLP) problems as diverse as language separation, acquisition of syntactic word classes and word sense disambiguation. At this, the fact is employed that the small-world property holds for many graphs in NLP.
---
paper_title: Automated categorization of bioacoustic signals: Avoiding perceptual pitfalls

paper_content:

Dividing the acoustic repertoires of animals into biologically relevant categories presents a widespread problem in the study of animal sound communication, essential to any comparison of repertoires between contexts, individuals, populations, or species. Automated procedures allow rapid, repeatable, and objective categorization, but often perform poorly at detecting biologically meaningful sound classes. Arguably this is because many automated methods fail to address the nonlinearities of animal sound perception. We present a new method of categorization that incorporates dynamic time-warping and an adaptive resonance theory (ART) neural network. This method was tested on 104 randomly chosen whistle contours from four captive bottlenose dolphins (Tursiops truncatus), as well as 50 frequency contours extracted from calls of transient killer whales (Orcinus orca). The dolphin data included known biologically meaningful categories in the form of 42 stereotyped whistles produced when each individual was isol...
---
paper_title: Recurring patterns in the songs of humpback whales (Megaptera novaeangliae)

paper_content:

Humpback whales, unlike most mammalian species, learn new songs as adults. Populations of singers progressively and collectively change the sounds and patterns within their songs throughout their lives and across generations. In this study, humpback whale songs recorded in Hawaii from 1985 to 1995 were analyzed using self-organizing maps (SOMs) to classify the sounds within songs, and to identify sound patterns that were present across multiple years. These analyses supported the hypothesis that recurring, persistent patterns exist within whale songs, and that these patterns are defined at least in part by acoustic relationships between adjacent sounds within songs. Sound classification based on acoustic differences between adjacent sounds yielded patterns within songs that were more consistent from year to year than classifications based on the properties of single sounds. Maintenance of fixed ratios of acoustic modulation across sounds, despite large variations in individual sounds, suggests intrinsic constraints on how sounds change within songs. Such acoustically invariant cues may enable whales to recognize and assess variations in songs despite propagation-related distortion of individual sounds and yearly changes in songs.
---
paper_title: Inverse Problem Theory and Methods for Model Parameter Estimation

paper_content:

1. The general discrete inverse problem 2. Monte Carol methods 3. The least-squares criterion 4. Least-absolute values criterion and minimax criterion 5. Functional inverse problems 6. Appendices 7. Problems References Index.
---
paper_title: 3d seismic imaging

paper_content:

In the past decade, 3D reflection seismology has replaced 2D seismology almost entirely in the seismic industry. Recording 3D surveys has become the norm instead of the exception. The application of 2D seismology is limited mostly to reconnaissance surveys or to locations where recording 3D data is still prohibitively expensive, such as in rough mountains and wild forests. However, academic research and teaching have struggled to keep up with the 3D revolution. As a consequence of this tardiness, no books are available that introduce the theory of seismic imaging from the 3D perspective. This book is aimed at filling the gap. ::: ::: Seismic processing of 3D data is inherently different from 2D processing. The differences begin with data acquisition: 3D data geometries are considerably more irregular than 2D geometries. Furthermore, 3D acquisition geometries are never complete because sources and receivers are never laid out in dense areal arrays covering the surface above the target. These fundamental differences, along with the increased dimensionality of the problem, strongly influence the methods applied to process, visualize, and interpret the final images. Most 3D imaging methods and algorithms cannot be derived from their 2D equivalent by merely adding a couple of dimensions to the 2D equations. This book introduces seismic imaging from the 3D perspective, starting from a 3D earth model. However, because the reader is likely to be familiar with 2D processing methods, I discuss the connections between 3D algorithms and the corresponding 2D algorithms whenever useful. ::: ::: The book covers all the important aspects of 3D imaging. It links the migration methods with data acquisition and velocity estimation, because they are inextricably intertwined in practice. Data geometries strongly influence the choice of 3D imaging methods. At the beginning of the book, I present the most common acquisition geometries, and I continue to discuss the relationships between imaging methods and acquisition geometries throughout the text. The imaging algorithms are introduced assuming regular and adequate sampling. However, Chapters 8 and 9 explicitly discuss the problems and solutions related to irregular and inadequate spatial sampling of the data. ::: ::: Velocity estimation is an integral component of the imaging process. On one hand, we need to provide a good velocity function to the migration process to create a good image. On the other hand, velocity is estimated in complex areas by iterative migration and velocity updating. Migration methods are presented first in the book because they provide the basic understanding necessary to discuss the velocity updating process. ::: ::: Seismic-imaging algorithms can be divided into two broad categories, integral methods (e.g., Kirchhoff methods) and wavefield-continuation methods. Integral methods can be described by simple geometric objects such as rays and summation surfaces. Thus, they are understood more easily by intuition than wavefield-continuation methods are. My introduction of the basic principles of 3D imaging exploits the didactic advantages of integral methods. However, wavefield-continuation methods can yield more accurate images of complex subsurface structures. This book introduces wavefield-continuation imaging methods by leveraging the intuitive understanding gained during the study of integral methods. Wavefield-continuation methods are the subject of my ongoing research and that of my graduate students. Therefore, the wavefield-continuation methods described are more advanced, although less well established, than the corresponding integral methods. ::: ::: Seismic-imaging technology is data driven, and the book contains many examples of applications. The examples illustrate the rationale of the methods and expose their strengths and weaknesses. The data examples are drawn both from real data sets and from a realistic synthetic data set, the SEG-EAGE salt data set, which is distributed freely and used widely in the geophysical community. For the reader's convenience, a subset of this data set (known as C3 narrow-azimuth) is contained in the DVD included with this book. Appendix 2 briefly describes this data set. ::: ::: The software needed to produce many examples also will be distributed freely over the Internet. A reader with the necessary computer equipment (a powerful Unix workstation) and the patience to wait for weeks-long runs could reproduce the images obtained from the SEG-EAGE salt data set. Appendix A describes the foundations of SEPlib3d, the main software package needed to generate most of the results shown in this text. ::: ::: The book starts from the introduction of the basic concepts and methods in 3D seismic imaging. To follow the first part of the book, the reader is expected to have only an elementary understanding of 2D seismic methods. The book thus can be used for teaching a first-level graduate class as well as a short course for professionals. The second part of the book covers more complex topics and recent research advances. This material can be used in an advanced graduate class in seismic imaging. To facilitate the teaching of the material in this book, the attached DVD includes a document in PDF format that has been formatted specifically to be projected electronically during a lecture. All the figures in this electronic document can be animated by clicking on a button in the figure caption. Several of these figures are movies that provide a more cogent illustration of the concepts described in the text. All figures are included on the attached DVD as GIF files.
---
paper_title: An Imaging Perspective of Low-Rank Seismic Data Interpolation and Denoising

paper_content:

Algorithms for low-rank seismic data denoising or interpolation have generally focused on improving the aggregate signal to noise ratio (SNR) in the filtered or reconstructed data. Motivated by the observation that improved SNR level does not always imply proportionally enhanced imaging quality, especially if the SNR improvement is attained mainly by wave components with limited contribution to imaging, we propose a modified least squares reverse time migration algorithm which integrates low-rank data approximation into the imaging process. More specifically, based on a matrix formulation of the adjoint-state framework, the new imaging algorithm absorbs low-rank approximation into the migration and imaging processes where the contribution from different sources and receivers are weighted properly according to the low rank structure, instead of being summed up uniformly. Effectively this translates the benefit of data denoising and interpolation directly into improved gradient directions and therefore enhanced imaging quality, and provides a mechanism allowing tradeoff between noise reduction/data completion and data distortion directly based on imaging quality. The performance of the new imaging algorithm is demonstrated via synthetic examples.
---
paper_title: Semi-automatic detection of faults in 3D seismic data

paper_content:

The semi-automated detection of objects has been quite successful in detecting various types of seismic object, such as chimneys. The same technique can be applied successfully to detect faults in 3D seismic data. We show that several different attributes – among others, similarity, frequency and curvature, all of which potentially enhance the visibility of faults – can be combined successfully by an artificial neural network. This results in a fault ‘probability’ cube in which faults are more continuous and noise is suppressed compared with single-attribute cubes. It is believed that the fault-cube can be improved further by applying image-processing techniques to enhance the fault prediction.
---
paper_title: A comparison of classification techniques for seismic facies recognition

paper_content:

AbstractDuring the past decade, the size of 3D seismic data volumes and the number of seismic attributes have increased to the extent that it is difficult, if not impossible, for interpreters to examine every seismic line and time slice. To address this problem, several seismic facies classification algorithms including k-means, self-organizing maps, generative topographic mapping, support vector machines, Gaussian mixture models, and artificial neural networks have been successfully used to extract features of geologic interest from multiple volumes. Although well documented in the literature, the terminology and complexity of these algorithms may bewilder the average seismic interpreter, and few papers have applied these competing methods to the same data volume. We have reviewed six commonly used algorithms and applied them to a single 3D seismic data volume acquired over the Canterbury Basin, offshore New Zealand, where one of the main objectives was to differentiate the architectural elements of a tu...
---
paper_title: Seismic Multi-attribute Classification for Salt Boundary Detection - A Comparison

paper_content:

Accurate delineation of salt bodies is one of the major tasks of hydrocarbon exploration and production from 3D seismic surveying. With the increasing demand of high-resolution seismic interpretation, the size of 3D seismic volumes as well as the number of available seismic attributes has been rapidly rising, which adds the difficulties for interpreters to examine and interpret every vertical line and time slice in a seismic volume. Various machine learning techniques have been introduced from the field of image/video processing to help address this limitation; however, little effort has been devoted to fair comparisons between these techniques. This study implements six commonly-used classification techniques and compares their capabilities for salt-boundary detection, including logistic regression, decision tree, random forest, support vector machine, artificial neural network, and k-means clustering, through applications to the F3 seismic dataset of multiple salt bodies over the Netherlands North Sea. The good match between the detected salt boundaries and the original seismic images indicates that based on well-selected attributes, all six classification techniques are capable of providing reliable salt detection from 3D seismic data to assist structural framework modelling in the presence of salt domes.
---
paper_title: What in the world do we hear? an ecological approach to auditory event perception

paper_content:

Everyday listening is the experience of hearing events in the world rather than sounds per se. In this article, I take an ecological approach to everyday listening to overcome constraints on its study implied by more traditional approaches. In particular, I am concerned with developing a new framework for describing sound in terms of audible source attributes. An examination of the continuum of structured energy from event to audition suggests that sound conveys information about events at locations in an environment. Qualitative descriptions of the physics of sound~producing events, complemented by protocol studies, suggest a tripartite division of sound-producing events into those involving vibrating solids, gasses, or liquids. Within each of these categories, basic-level events are defined by the simple interactions that can cause these materials to sound, whereas more complex events can be described in terms of temporal patterning, compound, or hybrid sources. The results of these investigations are u...
---
paper_title: Aircraft take-off noises classification based on human auditory’s matched features extraction

paper_content:

Abstract Air transportation is one of the most important services in the world, contributing greatly to the advancement of modern society. However, it has a local and a global impact on the environment making aircraft take-off noise an important environmental public health concern near airports, and this is a significant subject for monitoring and research. In this work an experimentally validated computational model for aircraft classification is presented. In addition, potentially harmful effects to human health and comfort associated with noise exposure are discussed. The feature extraction of aircraft take-off signals is conducted through a 1/24 octave analysis and Mel frequency cepstral coefficients (MFCC). The aircraft classification is made by using two parallel feed forward neural networks. The aircraft are clustered into classes depending on the installed engine type. This model has 13 aircraft classes and a classification level above 83% with measurements in real time environment.
---
paper_title: Speech feature denoising and dereverberation via deep autoencoders for noisy reverberant speech recognition

paper_content:

Denoising autoencoders (DAs) have shown success in generating robust features for images, but there has been limited work in applying DAs for speech. In this paper we present a deep denoising autoencoder (DDA) framework that can produce robust speech features for noisy reverberant speech recognition. The DDA is first pre-trained as restricted Boltzmann machines (RBMs) in an unsupervised fashion. Then it is unrolled to autoencoders, and fine-tuned by corresponding clean speech features to learn a nonlinear mapping from noisy to clean features. Acoustic models are re-trained using the reconstructed features from the DDA, and speech recognition is performed. The proposed approach is evaluated on the CHiME-WSJ0 corpus, and shows a 16-25% absolute improvement on the recognition accuracy under various SNRs.
---
paper_title: Loudness constancy with varying sound source distance

paper_content:

At a listener's ears, sound source power and sound source distance are confounded in measures of acoustic intensity, a physical property long thought to be the primary determinate of loudness. Although the relationship between sound source loudness and power is well known when source distance is fixed, relatively little is known about source loudness under conditions of varying distance. Here we show a robust loudness constancy, similar in many ways to visual size constancy, that results under distance-varying conditions that produce inaccurate estimates of source distance. Our results suggest that the auditory system does not require accurate distance estimates to judge source loudness, even when distance is variable. We offer an alternative explanation of loudness constancy based solely on a reverberant sound energy cue.
---
paper_title: The reverb challenge: A common evaluation framework for dereverberation and recognition of reverberant speech

paper_content:

Recently, substantial progress has been made in the field of reverberant speech signal processing, including both single- and multichannel dereverberation techniques, and automatic speech recognition (ASR) techniques robust to reverberation. To evaluate state-of-the-art algorithms and obtain new insights regarding potential future research directions, we propose a common evaluation framework including datasets, tasks, and evaluation metrics for both speech enhancement and ASR techniques. The proposed framework will be used as a common basis for the REVERB (REverberant Voice Enhancement and Recognition Benchmark) challenge. This paper describes the rationale behind the challenge, and provides a detailed description of the evaluation framework and benchmark results.
---
paper_title: Statistics of natural reverberation enable perceptual separation of sound and space

paper_content:

Abstract ::: In everyday listening, sound reaches our ears directly from a source as well as indirectly via reflections known as reverberation. Reverberation profoundly distorts the sound from a source, yet humans can both identify sound sources and distinguish environments from the resulting sound, via mechanisms that remain unclear. The core computational challenge is that the acoustic signatures of the source and environment are combined in a single signal received by the ear. Here we ask whether our recognition of sound sources and spaces reflects an ability to separate their effects and whether any such separation is enabled by statistical regularities of real-world reverberation. To first determine whether such statistical regularities exist, we measured impulse responses (IRs) of 271 spaces sampled from the distribution encountered by humans during daily life. The sampled spaces were diverse, but their IRs were tightly constrained, exhibiting exponential decay at frequency-dependent rates: Mid frequencies reverberated longest whereas higher and lower frequencies decayed more rapidly, presumably due to absorptive properties of materials and air. To test whether humans leverage these regularities, we manipulated IR decay characteristics in simulated reverberant audio. Listeners could discriminate sound sources and environments from these signals, but their abilities degraded when reverberation characteristics deviated from those of real-world environments. Subjectively, atypical IRs were mistaken for sound sources. The results suggest the brain separates sound into contributions from the source and the environment, constrained by a prior on natural reverberation. This separation process may contribute to robust recognition while providing information about spaces around us.
---
paper_title: Improving speech recognition in reverberation using a room-aware deep neural network and multi-task learning

paper_content:

In this paper, we propose two approaches to improve deep neural network (DNN) acoustic models for speech recognition in reverberant environments. Both methods utilize auxiliary information in training the DNN but differ in the type of information and the manner in which it is used. The first method uses parallel training data for multi-task learning, in which the network is trained to perform both a primary senone classification task and a secondary feature enhancement task using a shared representation. The second method uses a parameterization of the reverberant environment extracted from the observed signal to train a room-aware DNN. Experiments were performed on the single microphone task of the REVERB Challenge corpus. The proposed approach obtained a word error rate of 7.8% on the SimData test set, which is lower than all reported systems using the same training data and evaluation conditions, and 27.5% on the mismatched RealData test set, which is lower than all but two systems.
---
paper_title: The REVERB Challenge: A Benchmark Task for Reverberation-Robust ASR Techniques

paper_content:

The REVERB challenge is a benchmark task designed to evaluate reverberation-robust automatic speech recognition techniques under various conditions. A particular novelty of the REVERB challenge database is that it comprises both real reverberant speech recordings and simulated reverberant speech, both of which include tasks to evaluate techniques for 1-, 2-, and 8-microphone situations. In this chapter, we describe the problem of reverberation and characteristics of the REVERB challenge data, and finally briefly introduce some results and findings useful for reverberant speech processing in the current deep-neural-network era.
---
paper_title: Letters: Binaural semi-blind dereverberation of noisy convoluted speech signals

paper_content:

In order to overcome a limited performance of a conventional monaural model, this letter proposes a binaural blind dereverberation model. Its learning rule is derived using a blind least-squares measure by exploiting higher-order characteristics of output components. In order to prevent an unwanted whitening of speech signal, we adopt a semi-blind approach by employing a pre-determined whitening filter. The proposed model is evaluated using several simulated conditions and the results show better speech quality than those of the monaural model. The applicability of the model to the real environment is also shown by applying to real-recorded data. Especially, the proposed model attains much improved word error rates from 13.9+/-5.7(%) to 4.1+/-3.5(%) across 13 speakers for testing in the real speech recognition experiments.
---
paper_title: Unified approach for underdetermined BSS, VAD, dereverberation and DOA estimation with multichannel factorial HMM

paper_content:

This paper proposes a novel method for simultaneously solving the problems of underdetermined blind source separation (BSS), source activity detection, dereverberation and direction-of-arrival (DOA) estimation by introducing an extension of the "multichannel factorial hidden Markov model (MFH-MM)." The MFHMM is an extension of the multichannel non-negative matrix factorization (NMF) modeL in which the basis spectra are allowed to vary over time according to the transitions of the hidden states. This model has allowed us to perform source separation, source activity detection and dereverberation in a unified manner. In our previous model, the spatial covariance of each source has been treated as a model parameter. This has led the entire generative model to have an unnecessarily high degree of freedom, and thus the parameter inference has been prone to getting trapped into undesired local optima. To reasonably restrict the solution space of the spatial covariance matrix of each source, we propose to describe it as a weighted sum of the fixed spatial covariance matrix corresponding to the discrete set of DOAs. Through the parameter inference, the proposed model allows us to simultaneously solve the problems of underdetermined BSS, source activity detection, dereverberation and DOA estimation. Experimental results revealed that the proposed method was superior to a previous method in terms of the signal-to-distortion ratios of separated signals.
---
paper_title: Audio Event Detection using Weakly Labeled Data

paper_content:

Acoustic event detection is essential for content analysis and description of multimedia recordings. The majority of current literature on the topic learns the detectors through fully-supervised techniques employing strongly labeled data. However, the labels available for majority of multimedia data are generally weak and do not provide sufficient detail for such methods to be employed. In this paper we propose a framework for learning acoustic event detectors using only weakly labeled data. We first show that audio event detection using weak labels can be formulated as an Multiple Instance Learning problem. We then suggest two frameworks for solving multiple-instance learning, one based on support vector machines, and the other on neural networks. The proposed methods can help in removing the time consuming and expensive process of manually annotating data to facilitate fully supervised learning. Moreover, it can not only detect events in a recording but can also provide temporal locations of events in the recording. This helps in obtaining a complete description of the recording and is notable since temporal information was never known in the first place in weakly labeled data.
---
paper_title: SoundNet: Learning Sound Representations from Unlabeled Video

paper_content:

We learn rich natural sound representations by capitalizing on large amounts of unlabeled sound data collected in the wild. We leverage the natural synchronization between vision and sound to learn an acoustic representation using two-million unlabeled videos. Unlabeled video has the advantage that it can be economically acquired at massive scales, yet contains useful signals about natural sound. We propose a student-teacher training procedure which transfers discriminative visual knowledge from well established visual recognition models into the sound modality using unlabeled video as a bridge. Our sound representation yields significant performance improvements over the state-of-the-art results on standard benchmarks for acoustic scene/object classification. Visualizations suggest some high-level semantics automatically emerge in the sound network, even though it is trained without ground truth labels.
---
paper_title: Look, Listen and Learn

paper_content:

We consider the question: what can be learnt by looking at and listening to a large number of unlabelled videos? There is a valuable, but so far untapped, source of information contained in the video itself – the correspondence between the visual and the audio streams, and we introduce a novel “Audio-Visual Correspondence” learning task that makes use of this. Training visual and audio networks from scratch, without any additional supervision other than the raw unconstrained videos themselves, is shown to successfully solve this task, and, more interestingly, result in good visual and audio representations. These features set the new state-of-the-art on two sound classification benchmarks, and perform on par with the state-of-the-art selfsupervised approaches on ImageNet classification. We also demonstrate that the network is able to localize objects in both modalities, as well as perform fine-grained recognition tasks.
---
paper_title: Learning to Localize Sound Source in Visual Scenes

paper_content:

Visual events are usually accompanied by sounds in our daily lives. We pose the question: Can the machine learn the correspondence between visual scene and the sound, and localize the sound source only by observing sound and visual scene pairs like human? In this paper, we propose a novel unsupervised algorithm to address the problem of localizing the sound source in visual scenes. A two-stream network structure which handles each modality, with attention mechanism is developed for sound source localization. Moreover, although our network is formulated within the unsupervised learning framework, it can be extended to a unified architecture with a simple modification for the supervised and semi-supervised learning settings as well. Meanwhile, a new sound source dataset is developed for performance evaluation. Our empirical evaluation shows that the unsupervised method eventually go through false conclusion in some cases. We show that even with a few supervision, false conclusion is able to be corrected and the source of sound in a visual scene can be localized effectively.
---
paper_title: The Sound of Pixels

paper_content:

We introduce PixelPlayer, a system that, by leveraging large amounts of unlabeled videos, learns to locate image regions which produce sounds and separate the input sounds into a set of components that represents the sound from each pixel. Our approach capitalizes on the natural synchronization of the visual and audio modalities to learn models that jointly parse sounds and images, without requiring additional manual supervision. Experimental results on a newly collected MUSIC dataset show that our proposed Mix-and-Separate framework outperforms several baselines on source separation. Qualitative results suggest our model learns to ground sounds in vision, enabling applications such as independently adjusting the volume of sound sources.
---
paper_title: Objects that Sound

paper_content:

In this paper our objectives are, first, networks that can embed audio and visual inputs into a common space that is suitable for cross-modal retrieval; and second, a network that can localize the object that sounds in an image, given the audio signal. We achieve both these objectives by training from unlabelled video using only audio-visual correspondence (AVC) as the objective function. This is a form of cross-modal self-supervision from video. ::: To this end, we design new network architectures that can be trained using the AVC task for these functionalities: for cross-modal retrieval, and for localizing the source of a sound in an image. We make the following contributions: (i) show that audio and visual embedding can be learnt that enable both within-mode (e.g. audio-to-audio) and between-mode retrieval; (ii) explore various architectures for the AVC task, including those for the visual stream that ingest a single image, or multiple images, or a single image and multi-frame optical flow; (iii) show that the semantic object that sounds within an image can be localized (using only the sound, no motion or flow information); and (iv) give a cautionary tale in how to avoid undesirable shortcuts in the data preparation.
---
paper_title: Audio-Visual Scene Analysis with Self-Supervised Multisensory Features

paper_content:

The thud of a bouncing ball, the onset of speech as lips open—when visual and audio events occur together, it suggests that there might be a common, underlying event that produced both signals. In this paper, we argue that the visual and audio components of a video signal should be modeled jointly using a fused multisensory representation. We propose to learn such a representation in a self-supervised way, by training a neural network to predict whether video frames and audio are temporally aligned. We use this learned representation for three applications: (a) sound source localization, i.e. visualizing the source of sound in a video; (b) audio-visual action recognition; and (c) on/off-screen audio source separation, e.g. removing the off-screen translator’s voice from a foreign official’s speech. Code, models, and video results are available on our webpage: http://andrewowens.com/multisensory.
---
paper_title: An overview of applications and advancements in automatic sound recognition

paper_content:

Automatic sound recognition (ASR) has attracted increased and wide ranging interests in recent years. In this paper, we carry out a review of some important contributions in ASR techniques, mainly over the last one and a half decades. Similar to speech recognition systems, the robustness of an ASR system largely depends on the choice of feature(s) and classifier(s). We take a wider perspective in providing an overview of the features and classifiers used in ASR systems starting from early works in content-based audio classification to more recent developments in applications such as sound event recognition, audio surveillance, and environmental sound recognition. We also review techniques that have been utilized in noise robust sound recognition systems and feature optimization methods. Finally, some of the less commonly known applications of ASR are discussed.
---
paper_title: WaveNet: A Generative Model for Raw Audio

paper_content:

This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.
---
paper_title: Monaural Sound Source Separation by Nonnegative Matrix Factorization With Temporal Continuity and Sparseness Criteria

paper_content:

An unsupervised learning algorithm for the separation of sound sources in one-channel music signals is presented. The algorithm is based on factorizing the magnitude spectrogram of an input signal into a sum of components, each of which has a fixed magnitude spectrum and a time-varying gain. Each sound source, in turn, is modeled as a sum of one or more components. The parameters of the components are estimated by minimizing the reconstruction error between the input spectrogram and the model, while restricting the component spectrograms to be nonnegative and favoring components whose gains are slowly varying and sparse. Temporal continuity is favored by using a cost term which is the sum of squared differences between the gains in adjacent frames, and sparseness is favored by penalizing nonzero gains. The proposed iterative estimation algorithm is initialized with random values, and the gains and the spectra are then alternatively updated using multiplicative update rules until the values converge. Simulation experiments were carried out using generated mixtures of pitched musical instrument samples and drum sounds. The performance of the proposed method was compared with independent subspace analysis and basic nonnegative matrix factorization, which are based on the same linear model. According to these simulations, the proposed method enables a better separation quality than the previous algorithms. Especially, the temporal continuity criterion improved the detection of pitched musical sounds. The sparseness criterion did not produce significant improvements
---
paper_title: Neural processing of natural sounds

paper_content:

The auditory system is finely tuned to the unique statistical structure of natural sounds. In this Review, Theunissen and Elie discuss how specific neurons found at the higher levels of the auditory system respond selectively to perceptually salient sounds such as vocalizations used in communication.
---
paper_title: A scalable feature learning and tag prediction framework for natural environment sounds

paper_content:

Building feature extraction approaches that can effectively characterize natural environment sounds is challenging due to the dynamic nature. In this paper, we develop a framework for feature extraction and obtaining semantic inferences from such data. In particular, we propose a new pooling strategy for deep architectures, that can preserve the temporal dynamics in the resulting representation. By constructing an ensemble of semantic embeddings, we employ an l 1 -reconstruction based prediction algorithm for estimating the relevant tags. We evaluate our approach on challenging environmental sound recognition datasets, and show that the proposed features outperform traditional spectral features.
---
paper_title: Auditory perception of material is fragile while action is strikingly robust.

paper_content:

While many psychoacoustic studies have found that listeners can recover some causal properties of sound-generating objects (such as the material), comparatively little is known about the causal properties of the sound-generating actions and how they are perceived. This article reports on a study comparing the performance of listeners required to identify either the actions or the materials used to generate sound stimuli. Stimuli were recordings of a set of cylinders of two sizes and four materials (wood, plastic, glass, metal) undergoing four different actions (scraping, rolling, hitting, bouncing). Experiment 1 tested how well each sound conveyed that it was generated with a different action or material. Experiment 2 measured both accuracy and reaction times for the identification of actions and materials. Listeners were faster and more accurate at identifying the action than the material. Even for the subset of sounds for which actions and materials were equivalently well identified, listeners were faster at identifying the action than the material. These results suggest that the auditory system is well-suited to extract information about sound-generating actions.
---
paper_title: Material identification of real impact sounds: Effects of size variation in steel, glass, wood, and plexiglass plates

paper_content:

Identification of the material of struck objects of variable size was investigated. Previous studies on this issue assumed recognition to be based on acoustical measures of damping. This assumption was tested, comparing the power of a damping measure in explaining identification data with that of several other acoustical descriptors. Listeners’ performance was perfect with respect to gross material categories (steel-glass and wood-plexiglass) comprising materials of vastly different mechanical properties. Impaired performance was observed for materials within the same gross category, identification being based on the size of the objects alone. The damping descriptor accounted for the identification of the gross categories. However other descriptors such as signal duration explained the results equally well. Materials within the same gross category were identified mainly on the basis of signal frequency. Overall poor support for the relevance of damping to material perception was found. An analysis of the acoustical support for perfect material identification was carried out. Sufficient acoustical information for perfect performance was found. Thus, procedural biases for the origin of the effects of size could be discarded, pointing toward their cognitive, rather than methodological nature. Identification performance was explained in terms of the regularities of the everyday acoustical environment.
---
paper_title: Fast modal sounds with scalable frequency-domain synthesis

paper_content:

Audio rendering of impact sounds, such as those caused by falling objects or explosion debris, adds realism to interactive 3D audiovisual applications, and can be convincingly achieved using modal sound synthesis. Unfortunately, mode-based computations can become prohibitively expensive when many objects, each with many modes, are impacted simultaneously. We introduce a fast sound synthesis approach, based on short-time Fourier Tranforms, that exploits the inherent sparsity of modal sounds in the frequency domain. For our test scenes, this "fast mode summation" can give speedups of 5--8 times compared to a time-domain solution, with slight degradation in quality. We discuss different reconstruction windows, affecting the quality of impact sound "attacks". Our Fourier-domain processing method allows us to introduce a scalable, real-time, audio processing pipeline for both recorded and modal sounds, with auditory masking and sound source clustering. To avoid abrupt computation peaks, such as during the simultaneous impacts of an explosion, we use crossmodal perception results on audiovisual synchrony to effect temporal scheduling. We also conducted a pilot perceptual user evaluation of our method. Our implementation results show that we can treat complex audiovisual scenes in real time with high quality.
---
paper_title: Generative Modeling of Audible Shapes for Object Perception

paper_content:

Humans infer rich knowledge of objects from both auditory and visual cues. Building a machine of such competency, however, is very challenging, due to the great difficulty in capturing large-scale, clean data of objects with both their appearance and the sound they make. In this paper, we present a novel, open-source pipeline that generates audiovisual data, purely from 3D object shapes and their physical properties. Through comparison with audio recordings and human behavioral studies, we validate the accuracy of the sounds it generates. Using this generative model, we are able to construct a synthetic audio-visual dataset, namely Sound-20K, for object perception tasks. We demonstrate that auditory and visual information play complementary roles in object perception, and further, that the representation learned on synthetic audio-visual data can transfer to real-world scenarios.
---
paper_title: Inverse-Foley animation: synchronizing rigid-body motions to sound

paper_content:

In this paper, we introduce Inverse-Foley Animation, a technique for optimizing rigid-body animations so that contact events are synchronized with input sound events. A precomputed database of randomly sampled rigid-body contact events is used to build a contact-event graph, which can be searched to determine a plausible sequence of contact events synchronized with the input sound's events. To more easily find motions with matching contact times, we allow transitions between simulated contact events using a motion blending formulation based on modified contact impulses. We fine tune synchronization by slightly retiming ballistic motions. Given a sound, our system can synthesize synchronized motions using graphs built with hundreds of thousands of precomputed motions, and millions of contact events. Our system is easy to use, and has been used to plan motions for hundreds of sounds, and dozens of rigid-body models.
---
paper_title: Visually Indicated Sounds

paper_content:

Objects make distinctive sounds when they are hit or scratched. These sounds reveal aspects of an object's material properties, as well as the actions that produced them. In this paper, we propose the task of predicting what sound an object makes when struck as a way of studying physical interactions within a visual scene. We present an algorithm that synthesizes sound from silent videos of people hitting and scratching objects with a drumstick. This algorithm uses a recurrent neural network to predict sound features from videos and then produces a waveform from these features with an example-based synthesis procedure. We show that the sounds predicted by our model are realistic enough to fool participants in a "real or fake" psychophysical experiment, and that they convey significant information about material properties and physical interactions.
---

Title: Machine Learning in Acoustics: A Review

Section 1: INTRODUCTION
Description 1: Introduce the significance and application of ML in various fields of acoustics, highlight current challenges and motivations for using ML, and provide an overview of the paper's structure.

Section 2: MACHINE LEARNING PRINCIPLES
Description 2: Discuss fundamental principles of machine learning, including essential concepts like data-driven models, supervised and unsupervised learning, generalization, cross-validation, curse of dimensionality, and Bayesian machine learning.

Section 3: SUPERVISED LEARNING
Description 3: Explain supervised learning methods in detail, including linear regression and classification, neural networks, and support vector machines. Provide examples and applications in acoustics.

Section 4: UNSUPERVISED LEARNING
Description 4: Cover unsupervised learning techniques such as principal components analysis (PCA), K-means clustering, dictionary learning, and autoencoder networks, emphasizing their application in acoustics.

Section 5: DEEP LEARNING
Description 5: Describe deep learning methodologies, including architectures like convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Discuss advanced training techniques, transfer learning, and specialized architectures.

Section 6: SOURCE LOCALIZATION IN SPEECH PROCESSING
Description 6: Discuss algorithms and methods used for speech source localization, including EM-based unsupervised techniques and manifold learning approaches, highlighting their robustness in challenging environments.

Section 7: SOURCE LOCALIZATION IN OCEAN ACOUSTICS
Description 7: Explore the applications of ML in ocean acoustics, with a focus on matched-field processing, early ML methods, and recent advancements using deep neural networks for robust source localization.

Section 8: BIOACOUSTICS
Description 8: Present applications of ML in bioacoustics, focusing on species detection, vocalization classification, and the use of supervised and unsupervised learning techniques in ecological and conservation studies.

Section 9: SEISMIC EXPLORATION
Description 9: Detail the use of ML, particularly deep learning, in seismic exploration for hydrocarbon discovery, including image classification for geological structures and facies classification using pre-stack data.

Section 10: REVERBERATION AND ENVIRONMENTAL SOUNDS IN EVERYDAY SCENES
Description 10: Address the challenges and solutions in identifying and classifying sounds in everyday environments. Discuss techniques for robustness against reverberation and the identification of diverse environmental sounds.

Section 11: CONCLUSION
Description 11: Summarize the advances in ML for acoustics presented in the review and emphasize the potential for future research driven by the availability of large open datasets.