A survey on sound source localization in robotics: From binaural to array processing methods☆☆☆
---
paper_title: Unified auditory functions based on Bayesian topic model

paper_content:

Existing auditory functions for robots such as sound source localization and separation have been implemented in a cascaded framework whose overall performance may be degraded by any failure in its subsystems. These approaches often require a careful and environment-dependent tuning for each subsystems to achieve better performance. This paper presents a unified framework for sound source localization and separation where the whole system is integrated as a Bayesian topic model. This method improves both localization and separation with a common configuration under various environments by iterative inference using Gibbs sampling. Experimental results from three environments of different reverberation times confirm that our method outperforms state-of-the-art sound source separation methods, especially in the reverberant environments, and shows localization performance comparable to that of the existing robot audition system.
---
paper_title: The Cocktail Party Problem

paper_content:

This review presents an overview of a challenging problem in auditory perception, the cocktail party phenomenon, the delineation of which goes back to a classic paper by Cherry in 1953. In this review, we address the following issues: (1) human auditory scene analysis, which is a general process carried out by the auditory system of a human listener; (2) insight into auditory perception, which is derived from Marr's vision theory; (3) computational auditory scene analysis, which focuses on specific approaches aimed at solving the machine cocktail party problem; (4) active audition, the proposal for which is motivated by analogy with active vision, and (5) discussion of brain theory and independent component analysis, on the one hand, and correlative neural firing, on the other.
---
paper_title: Binaural Systems in Robotics

paper_content:

Audition is often described by physiologists as the most important sense in humans, due to its essential role in communication and socialization. But quite surprisingly, the interest of this modality for robotics arose only in the 2000s, brought to evidence by cognitive robotics and Human–robot interaction. Since then, numerous contributions have been proposed to the field of robot audition, ranging from sound localization to scene analysis. Binaural approaches were investigated first, then became forsaken due to mixed results. Nevertheless, the last years have witnessed a renewal of interest in binaural active audition, that is, in the opportunities and challenges opened by the coupling of binaural sensing and robot motion. This chapter proposes a comprehensive state of the art of binaural approaches to robot audition. Though the literature on binaural audition and, more generally, on acoustics and signal processing, is a fundamental source of knowledge, the tasks, constraints, and environments of robotics raise original issues. These are reviewed, prior to the most prominent contributions, platforms and projects. Two lines of research in binaural active audition, conducted by the current authors, are then outlined, one of which is tightly connected to psychology of perception.
---
paper_title: Sound Localization by Human Listeners

paper_content:

INTRODUCTION 135 TWO-DIMENSIONAL SOUND LOCALIZATION . .. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . 1 37 HORIZONTAL LOCALIZATION . . . . . . . . . . . . . . . . . . . . . . .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . 140 VERTICAL LOCALIZATION ... . ..... . . . .. . . . . MONAURAL LOCALIZATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DISTANCE PERCEPTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . MOTION DETECTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DYNAMIC CUES FOR LOCALIZATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . SIMULATING EXTERNAL SOURCES OVER HEADPHONES . . . . . . . . . . . . . . . . . . . . . . . . . . . SUMMARy .
---
paper_title: The CIPIC HRTF Database

paper_content:

This paper describes a public-domain database of high-spatial-resolution head-related transfer functions measured at the UC Davis CIPIC Interface Laboratory and the methods used to collect the data.. Release 1.0 (see http://interface.cipic.ucdavis.edu) includes head-related impulse responses for 45 subjects at 25 different azimuths and 50 different elevations (1250 directions) at approximately 5/spl deg/ angular increments. In addition, the database contains anthropometric measurements for each subject. Statistics of anthropometric parameters and correlations between anthropometry and some temporal and spectral features of the HRTFs are reported.
---
paper_title: Introduction to Head-Related Transfer Functions (HRTFs): Representations of HRTFs in Time, Frequency, and Space

paper_content:

In this tutorial, head-related transfer functions (HRTFs) are introduced and treated with respect to their role in the synthesis of spatial sound over headphones. HRTFs are formally defined, and are shown to be important in reducing the ambiguity with which the classical duplex theory decodes a free-field sound's spatial location. Typical HRTF measurement strategies are described, and simple applications of HRTFs to headphone-based spatialized sound synthesis are given. By comparing and contrasting representations of HRTFs in the time, frequency, and spatial domains, different analytic and signal processing techniques used to investigate the structure of HRTFs are highlighted.
---
paper_title: A Free Database of Head Related Impulse Response Measurements in the Horizontal Plane with Multiple Distances

paper_content:

A freely available collection of Head-Related Impulse Response (HRIR) measurements is introduced. The impulse responses were acquired in an anechoic chamber using a KEMAR manikin at four different loudspeaker distances – 3 m, 2 m, 1 m and 0.5 m – reaching from the far field to the near field. The loudspeaker was positioned at ear height and the manikin was rotated with a high-precision stepper motor in one degree increments. Besides the raw HRIRs also datasets are available which have been compensated for the use with specific headphone models. Introduction In order to localize a sound source, the human auditory system takes advantage of the time lag and level difference between the two ear signals as well as cues added by reflection and diffraction by torso, head and external ears [1]. A head-related impulse response (HRIR), acquired under free-field conditions, is able to describe these features. Several databases of HRIRs are freely available [2, 3, 4], but their angular resolution in the horizontal plane is limited and only [4] contains measurements for more than one source distance (0.8 m and 3 m). For distances larger than 1 m the distance between the manikin and the source is assumed to have no influence on the binaural cues, only the amplitude decreases for larger distances [5]. If the source is positioned closer to the listener, changes in the binaural cues have to be considered. This study will provide HRIR databases measured in the horizontal plane with a resolution of 1◦ for sources positioned at a distance of 0.5 m, 1 m, 2 m and 3 m. φ r = 0.5 m, 1 m, 2 m, 3 m Figure: Coordinate system used in this paper. The azimuth angle φ describes the placement of the loudspeaker counter-clockwise around the head. Here φ = −60◦. HRIR Databases The HRIR databases described here are freely available under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 license and can be downloaded at http://dev.qu.tu-berlin.de/projects/measurements/wiki The datasets are available in different formats suitable for a variety of applications: .mat files for use in GNU octave and Matlab, .daff files in the Open Directional Audio File Format (OpenDAFF), an emerging open standard for spatial audio data, and .wav files for use in the SoundScape Renderer (SSR).
---
paper_title: Fast calculation system specialized for head-related transfer function based on boundary element method.

paper_content:

Recently, development of a numerical calculation of the head-related transfer function (HRTF) has been conducted using a computer model of a human head and the boundary element method. The reciprocity theorem is incorporated into the computational process in order to shorten the computational time, which is otherwise very long. On the other hand, another fast HRTF calculation method for any source position, which is realized by calculating factors independent of the source position in advance, has been suggested by the authors. Using this algorithm, the HRTF for any source position can be obtained in a few seconds with a common PC. The resulting HRTFs are more precise and are calculated faster than those by using the reciprocity theorem. However, speeding the process up even further is required in order to respond to a head movement and rotation or to moving sources during binaural sound reproduction. In this paper, a faster calculation method by incorporating a time domain operation into the authors' previous algorithm is proposed. Additionally, the new formulation, which eliminates the extra computational time in the preprocess, is proposed. This method is shown to be faster than the previous ones, but there are some discrepancies at higher frequencies.
---
paper_title: Active Audition for Humanoid

paper_content:

In this paper, we present an active audition system for humanoid robot “SIG the humanoid” . The audition system of the highly intelligent humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor movements. Given the multiple sound sources in the auditory scene, SIG actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. However, such an active head movement inevitably creates motor noise. The system must adaptively cancel motor noise using motor control signals. The experimental result demonstrates that the active audition by integration of audition, vision, and motor control enables sound source tracking in variety of conditions.
---
paper_title: Range dependence of the response of a spherical head model

paper_content:

The head-related transfer function (HRTF) varies with range as well as with azimuth and elevation. To better understand its close-range behavior, a theoretical and experimental investigation of the HRTF for an ideal rigid sphere was performed. An algorithm was developed for computing the variation in sound pressure at the surface of the sphere as a function of direction and range to the sound source. The impulse response was also measured experimentally. The results may be summarized as follows. First, the experimental measurements were in close agreement with the theoretical solution. Second, the variation of low-frequency interaural level difference with range is significant for ranges smaller than about five times the sphere radius. Third, the impulse response reveals the source of the ripples observed in the magnitude response, and provides direct evidence that the interaural time difference is not a strong function of range. Fourth, the time delay is well approximated by well-known ray-tracing formul...
---
paper_title: Applying scattering theory to robot audition system: robust sound source localization and extraction

paper_content:

Robot audition by its own ears (microphones) is essential for natural human-robot communication and interface. Since a microphone is embedded in the head of a robot, the head-related transfer function (HRTF) plays an important role in sound source localization and extraction. Usually, from binaural input, the interaural phase difference (IPD) and interaural intensity difference (IID) are calculated, and then the direction is determined by using IPD and IID with HRTF. The problem of HRTF-based sound source localization is that a HRTF should be measured for each robot in an anechoic chamber, because it depends on the shape of robot's head; HRTF should be interpolated to manipulate a moving talker, because it is available only for discrete azimuth and elevation. To cope with these problems of HRTF, we proposed the auditory epipolar geometry as a continuous function of IPD and IID to dispense with HRTF and have developed a real-time multiple-talker tracking system. This auditory epipolar geometry, however, does not give a good approximation to IID of all range and IPD of peripheral areas. In this paper, the scattering theory in physics is employed to take into consideration the diffraction of sounds around robot's head for better approximation of IID and IPD. The resulting system shows that it is efficient for localization and extraction of sound at higher frequency and from side directions.
---
paper_title: Epipolar geometry based sound localization and extraction for humanoid audition

paper_content:

Sound localization for a robot or an embedded system is usually solved by using inter-aural phase difference (IPD) and inter-aural intensity difference (IID). These values are calculated by using head-related transfer function (HRTF). However, the HRTF depends on the shape of the head and also on changes of the environments. Therefore, sound localization without HRTF is needed for real-world applications. In this paper, we present a new sound localization method based on auditory epipolar geometry with motion control. The auditory epipolar geometry is an extension of an epipolar geometry in stereo vision to audition, and auditory and visual epipolar geometries can share the sound source direction. The key idea is to exploit additional inputs obtained by the motor control in order to compensate damages in the IPD and IID caused by reverberation of the room and the body of the robot. The proposed system can localize and extract simultaneously two sound sources in a real-world room.
---
paper_title: The contribution of the near and far ear toward localization of sound in the sagittal plane.

paper_content:

Eight listeners were required to locate a train of 4.5-kHz high-pass noise bursts emanating from loudspeakers positioned +/- 30, +/- 20, +/- 10, and 0 deg re: interaural axis. The vertical array of loudspeakers was placed at 45, 90, and 135 deg left of midline. The various experimental conditions incorporated binaural and monaural listening with the latter utilizing the ear nearest or ear farthest from the sound source. While performance excelled when listening with only the near ear, the contribution of the far ear was statistically significant when compared to localization performance when both ears were occluded. Based on head related transfer functions for stimuli whose bandwidth was 1.0 kHz, four spectral cues were selected as candidates for influencing location judgments. Two of them associated relative changes in energy across center frequencies (CFs) with vertical source positions. The other two associated an absolute minimum (maximum) energy for specific CFs with a vertical source position. All but one cue when measured for the near ear could account for localization proficiency. On the other hand, when listening with the far ear, maximum energy at a specific CF outperformed the remaining cues in accounting for localization proficiency.
---
paper_title: Auditory distance perception in humans : A summary of past and present research

paper_content:

Although auditory distance perception is a critical component of spatial hearing, it has received substantially less scienti.c attention than the directional aspects of auditory localization. Here we summarize current knowledge on auditory distance perception, with special emphasis on recent research results. The summary will be structured around three central questions. 1. How accurately can humans estimate the distances of stationary sound sources? We show that this psychophysical relationship is well approximated by a compressive power function, which suggests that listeners systematically underestimate distances to faraway sound sources. 2. What determines perceived sound source distance? We examine the various acoustical and non-acoustical factors thought to contribute to source distance percepts, and summarize the psychophysical literature relevant to each factor. 3. What are the neural correlates to perceived sound source distance? Recent evidence points to the role of areas within right temporal cortex in auditory distance perception, as well as in other spatial tasks in different sensory modalities.
---
paper_title: A Scale for the Measurement of the Psychological Magnitude Pitch

paper_content:

A subjective scale for the measurement of pitch was constructed from determinations of the half‐value of pitches at various frequencies. This scale differs from both the musical scale and the frequency scale, neither of which is subjective. Five observers fractionated tones of 10 different frequencies at a loudness level of 60 db. From these fractionations a numerical scale was constructed which is proportional to the perceived magnitude of subjective pitch. In numbering the scale the 1000‐cycle tone was assigned the pitch of 1000 subjective units (mels).The close agreement of the pitch scale with an integration of the differential thresholds (DL's) shows that, unlike the DL's for loudness, all DL's for pitch are of uniform subjective magnitude. The agreement further implies that pitch and differential sensitivity to pitch are both rectilinear functions of extent on the basilar membrane.The correspondence of the pitch scale and the experimentally determined location of the resonant areas of the basilar me...
---
paper_title: Towards a systematic study of binaural cues

paper_content:

Sound source localization is a need for robotic systems interacting with acoustically-active environments. In this domain, numerous binaural localization studies have been conducted within the last few decades. This paper provides an overview of a number of binaural localization cue extraction techniques. These are carefully addressed and applied on a simulated binaural database. Cues are evaluated in azimuth estimation and their discriminatory effectiveness is studied as a function of the reverberation time with statistical data analysis techniques. Results show that big differences exist between the discriminatory abilities of multiple types of cue extraction methods. Thus a careful cue selection must be performed before establishing a sound localization system.
---
paper_title: The Auditory Modeling Toolbox

paper_content:

The Auditory Modeling Toolbox, AMToolbox, is a Matlab/Octave toolbox for developing and applying auditory perceptual models with a particular focus on binaural models. The philosophy behind the AMToolbox is the consistent implementation of auditory models, good documentation, and user-friendly access in order to allow students and researchers to work with and to advance existing models. In addition to providing the model implementations, published human data and model demonstrations are provided. Further, model implementations can be evaluated by running so-called experiments aimed at reproducing results from the corresponding publications. AMToolbox includes many of the models described in this volume. It is freely available from http://amtoolbox.sourceforge.net
---
paper_title: Time-domain modeling of peripheral auditory processing: a modular architecture and a software platform.

paper_content:

A software package with a modular architecture has been developed to support perceptual modeling of the fine-grain spectro-temporal information observed in the auditory nerve. The package contains both functional and physiological modules to simulate auditory spectral analysis, neural encoding, and temporal integration, including new forms of periodicity-sensitive temporal integration that generate stabilized auditory images. Combinations of the modules enable the user to approximate a wide variety of existing, time-domain, auditory models. Sequences of auditory images can be replayed to produce cartoons of auditory perceptions that illustrate the dynamic response of the auditory system to everyday sounds.
---
paper_title: Towards a systematic study of binaural cues

paper_content:

Sound source localization is a need for robotic systems interacting with acoustically-active environments. In this domain, numerous binaural localization studies have been conducted within the last few decades. This paper provides an overview of a number of binaural localization cue extraction techniques. These are carefully addressed and applied on a simulated binaural database. Cues are evaluated in azimuth estimation and their discriminatory effectiveness is studied as a function of the reverberation time with statistical data analysis techniques. Results show that big differences exist between the discriminatory abilities of multiple types of cue extraction methods. Thus a careful cue selection must be performed before establishing a sound localization system.
---
paper_title: Applying scattering theory to robot audition system: robust sound source localization and extraction

paper_content:

Robot audition by its own ears (microphones) is essential for natural human-robot communication and interface. Since a microphone is embedded in the head of a robot, the head-related transfer function (HRTF) plays an important role in sound source localization and extraction. Usually, from binaural input, the interaural phase difference (IPD) and interaural intensity difference (IID) are calculated, and then the direction is determined by using IPD and IID with HRTF. The problem of HRTF-based sound source localization is that a HRTF should be measured for each robot in an anechoic chamber, because it depends on the shape of robot's head; HRTF should be interpolated to manipulate a moving talker, because it is available only for discrete azimuth and elevation. To cope with these problems of HRTF, we proposed the auditory epipolar geometry as a continuous function of IPD and IID to dispense with HRTF and have developed a real-time multiple-talker tracking system. This auditory epipolar geometry, however, does not give a good approximation to IID of all range and IPD of peripheral areas. In this paper, the scattering theory in physics is employed to take into consideration the diffraction of sounds around robot's head for better approximation of IID and IPD. The resulting system shows that it is efficient for localization and extraction of sound at higher frequency and from side directions.
---
paper_title: Epipolar geometry based sound localization and extraction for humanoid audition

paper_content:

Sound localization for a robot or an embedded system is usually solved by using inter-aural phase difference (IPD) and inter-aural intensity difference (IID). These values are calculated by using head-related transfer function (HRTF). However, the HRTF depends on the shape of the head and also on changes of the environments. Therefore, sound localization without HRTF is needed for real-world applications. In this paper, we present a new sound localization method based on auditory epipolar geometry with motion control. The auditory epipolar geometry is an extension of an epipolar geometry in stereo vision to audition, and auditory and visual epipolar geometries can share the sound source direction. The key idea is to exploit additional inputs obtained by the motor control in order to compensate damages in the IPD and IID caused by reverberation of the room and the body of the robot. The proposed system can localize and extract simultaneously two sound sources in a real-world room.
---
paper_title: A biomimetic apparatus for sound-source localization

paper_content:

Sound source localization systems are typically made of free field microphone arrays. We have proposed the design of a localizing system based on the principles observed in nature, where directional acoustic sensing evolved to rely on diffraction about the head with only two ears. Localization is performed using the resultant frequency dependent acoustic phase and intensity differences between the two ears. These interaural functions can be computed analytically by modeling the head as a sphere. Now we report the first successful implementation of these ideas in an artificial head with two antipodally placed microphones. In order to determine source direction, we define a suitable metric associated to interaural functions. The true source direction is given by the global minimum of the metric between a measurement and a table of theoretical functions. The system was tested with a broadband source and was found to perform well even in a non-ideal reverberant laboratory environment. For comparison, we also calculated the source direction using the standard cross-correlation algorithm which is based on interaural time delay. Our approach performs more accurately and allows to use briefer signal durations. A key motivation for this work is to devise effective means to guide robotic navigation in environments with acoustic sources. The apparatus described in this paper is installed on a mobile robot in the laboratory.
---
paper_title: Biomimetic Sound-Source Localization

paper_content:

Sound-source localization systems typically comprise free-field microphone arrays. In nature, directional acoustic sensing evolved to rely on diffraction about the head with only two ears. For localization, the brain uses the resultant frequency-dependent acoustic phase and intensity differences between the two ears. We conceive a biomimetic artificial head with microphones placed on its surface. The interaural functions can be computed analytically by modeling the head as a sphere. We define a suitable metric between interaural functions, whose global minimum provides the true source direction. The natural configuration in which the two sensors are placed antipodally on the sphere has intrinsic rotational symmetry: it allows localization only up to a circle around the interaural axis. We describe two methods for breaking the detrimental symmetry in order to achieve full spherical localization capability. First, we consider rotation of the apparatus relative to the source and the information it adds to the localization metric. We derive analytically the gradient of the pressure field under rotation and compute the induced acoustic flow on the interaural localization functions. Second, we explore placing the sensors in configurations differing from antipodal. We show the efficacy of these methods through simulations.
---
paper_title: Using binaural and spectral cues for azimuth and elevation localization

paper_content:

It is a common assumption that with just two microphones only the azimuth angle of a sound source can be estimated and that a third, orthogonal microphone (or set of microphones) is necessary to estimate the elevation of the source. Recently, using specially designed ears and analyzing spectral cues several researchers managed to estimate sound source elevation with a binaural system. In this work, we show that with two bionic ears both azimuth and elevation angle can be determined using both binaural (e.g. IID and ITD) and spectral cues. This ability can also be used to disambiguate signals coming from the front or back. We present a detailed analysis of both azimuth and elevation localization performance for binaural and spectral cues in comparison. We demonstrate that with a small extension of a standard binaural system a basic elevation estimation capacity can be gained.
---
paper_title: The CIPIC HRTF Database

paper_content:

This paper describes a public-domain database of high-spatial-resolution head-related transfer functions measured at the UC Davis CIPIC Interface Laboratory and the methods used to collect the data.. Release 1.0 (see http://interface.cipic.ucdavis.edu) includes head-related impulse responses for 45 subjects at 25 different azimuths and 50 different elevations (1250 directions) at approximately 5/spl deg/ angular increments. In addition, the database contains anthropometric measurements for each subject. Statistics of anthropometric parameters and correlations between anthropometry and some temporal and spectral features of the HRTFs are reported.
---
paper_title: A learning-based approach to robust binaural sound localization

paper_content:

Sound source localization is an important feature designed and implemented on robots and intelligent systems. Like other artificial audition tasks, it is constrained to multiple problems, notably sound reflections and noises. This paper presents a sound source azimuth estimation approach in reverberant environments. It exploits binaural signals in a humanoid robotic context. Interaural Time and Level Differences (ITD and ILD) are extracted on multiple frequency bands and combined with a neural network-based learning scheme. A cue filtering process is used to reduce the reverberations effects. The system has been evaluated with simulation and real data, in multiple aspects covering realistic robot operating conditions, and was proven satisfying and effective as will be shown and discussed in the paper.
---
paper_title: Real-Time Sound Source Localization and Separation for Robot Audition

paper_content:

Robot audition in the real world should cope with environment noises and reverberation and motor noises caused by the robot’s own movements. This paper presents the active direction-pass filter (ADPF) to separate sounds originating from the specified direction with a pair of microphones. The ADPF is implemented by hierarchical integration of visual and auditory processing with hypothetical reasoning on interaural phase difference (IPD) and interaural intensity difference (IID) for each subband. In creating hypotheses, the reference data of IPD and IID is calculated by the auditory epipolar geometry on demand. Since the performance of the ADPF depends on the direction, the ADPF controls the direction by motor movement. The human tracking and sound source separation based on the ADPF is implemented on an upper-torso humanoid and runs in real-time with 4 PCs connected over Gigabit ethernet. The signal-to-noise ratio (SNR) of each sound separated by the ADPF from a mixture of two speeches with the same loudness is improved to about 10 dB from 0 dB.
---
paper_title: Active Audition for Humanoid

paper_content:

In this paper, we present an active audition system for humanoid robot “SIG the humanoid” . The audition system of the highly intelligent humanoid requires localization of sound sources and identification of meanings of the sound in the auditory scene. The active audition reported in this paper focuses on improved sound source tracking by integrating audition, vision, and motor movements. Given the multiple sound sources in the auditory scene, SIG actively moves its head to improve localization by aligning microphones orthogonal to the sound source and by capturing the possible sound sources by vision. However, such an active head movement inevitably creates motor noise. The system must adaptively cancel motor noise using motor control signals. The experimental result demonstrates that the active audition by integration of audition, vision, and motor control enables sound source tracking in variety of conditions.
---
paper_title: Sound source localisation through active audition

paper_content:

This paper presents a novel method for enabling a robot to determine the direction to a sound source through interacting with its environment. The method uses a new neural network, the parameter-less self-organizing map algorithm, and reinforcement learning to achieve rapid and accurate response.
---
paper_title: Real-time speaker localization and speech separation by audio-visual integration

paper_content:

Robot audition in real-world should cope with motor and other noises caused by the robot's own movements in addition to environmental noises and reverberation. This paper reports how auditory processing is improved by audio-visual integration with active movements. The key idea resides in hierarchical integration of auditory and visual streams to disambiguate auditory or visual processing. The system runs in real-time by using distributed processing on 4 PCs connected by a Gigabit Ethernet. The system implemented in a upper-torso humanoid tracks multiple talkers and extracts speech from a mixture of sounds. The performance of epipolar geometry based sound source localization and sound source separation by active and adaptive direction-pass filtering is also reported.
---
paper_title: Towards a systematic study of binaural cues

paper_content:

Sound source localization is a need for robotic systems interacting with acoustically-active environments. In this domain, numerous binaural localization studies have been conducted within the last few decades. This paper provides an overview of a number of binaural localization cue extraction techniques. These are carefully addressed and applied on a simulated binaural database. Cues are evaluated in azimuth estimation and their discriminatory effectiveness is studied as a function of the reverberation time with statistical data analysis techniques. Results show that big differences exist between the discriminatory abilities of multiple types of cue extraction methods. Thus a careful cue selection must be performed before establishing a sound localization system.
---
paper_title: Audio servo for robotic systems with pinnae

paper_content:

Sound localization is one of important abilities for robots which utilize auditory information. Binaural information such as interaural time difference (ITD) or interaural intensity difference(IID) is effective for horizontal sound localization. In order to extend this ability to vertical sound localization, spectral cues were utilized in this paper instead of increasing number of microphones. Spectral cues were the information about the vertical position of the sound source on frequency domain. Since spectral cues strongly depended on the shape of pinnae, artificial pinnae for auditory robots were designed based on diffraction-reflection model. A method to detect spectral cues was also developed. By utilizing them, a controller which made the robot orient to the sound source was proposed. Results of an experiment showed the validity of the proposed method.
---
paper_title: Mobile robot broadband sound localisation using a biologically inspired spiking neural network

paper_content:

A biologically inspired azimuthal broadband sound localisation system is introduced to simulates the functional organisation of the human auditory midbrain up to the inferior colliculus (IC). Supported by recent neurophysiological studies on the role of the IC and superior olivary complex (SOC) in sound processing, our system models two ascending pathways of the auditory midbrain: the ITD (Interaural Time Difference) pathway and ILD (Interaural Level Difference) pathway. In our approach to modelling the ITD pathway, we take account of Yinpsilas finding that only a single delay line exists in the ITD processing from cochlea to SOC for the ipsilateral ear while multiple delay lines exists for the contralateral ear. The ILD pathway is modelled without varied delay lines because of neurophysiological evidence that indicates the delays along that pathway are minimal and constant. First, two-dimensional (2D) tonotopical ITD and ILD spike maps over frequency and ITD/ILD are calculated by a spiking neural network which follows the biological delay structure. Then these maps are weighted considering the advance of ITD in low frequency and ILD in middle and high frequency. Finally, ITD and ILD maps are merged together to find out the best estimation of the sound source. Experimental results involving noise and voice show that our model performs sound localisation that approaches biological performance. Our approach brings not only new insight into the brain mechanism of the auditory system, but also demonstrates a practical application of sound localisation for mobile robots.
---
paper_title: Design and evaluation of two-channel-based sound source localization over entire azimuth range for moving talkers

paper_content:

We propose a way to evaluate various sound localization systems for moving sounds under the same conditions. To construct a database for moving sounds, we developed a moving sound creation tool using the API library developed by the ARINIS Company. We developed a two-channel-based sound source localization system integrated with a cross-power spectrum phase (CSP) analysis and EM algorithm. The CSP of sound signals obtained with only two microphones is used to localize the sound source without having to use prior information such as impulse response data. The EM algorithm helps the system cope with several moving sound sources and reduce localization error. We evaluated our sound localization method using artificial moving sounds and confirmed that it can well localize moving sounds slower than 1.125 rad/sec. Finally, we solve the problem of distinguishing whether sounds are coming from the front or back by rotating a robotpsilas head equipped with only two microphones. Our system was applied to a humanoid robot called SIG2, and we confirmed its ability to localize sounds over the entire azimuth range.
---
paper_title: Learning sound location from a single microphone

paper_content:

We consider the problem of estimating the incident angle of a sound, using only a single microphone. The ability to perform monaural (single-ear) localization is important to many animals; indeed, monaural cues are also the primary method by which humans decide if a sound comes from the front or back, as well as estimate its elevation. Such monaural localization is made possible by the structure of the pinna (outer ear), which modifies sound in a way that is dependent on its incident angle. In this paper, we propose a machine learning approach to monaural localization, using only a single microphone and an “artificial pinna” (that distorts sound in a direction-dependent way). Our approach models the typical distribution of natural and artificial sounds, as well as the direction-dependent changes to sounds induced by the pinna. Our experimental results also show that the algorithm is able to fairly accurately localize a wide range of sounds, such as human speech, dog barking, waterfall, thunder, and so on. In contrast to microphone arrays, this approach also offers the potential of significantly more compact, as well as lower cost and power, devices for sounds localization.
---
paper_title: Active soft pinnae for robots

paper_content:

Sound localization is one of fundamental abilities of auditory robots that is necessary for human-machine communication. Especially for autonomous robots, it is also necessary to make the system as simple as possible, and binaural configuration is thought to be the minimum setup for auditory systems as animals achieve practical hearing with two ears. However, the limitation of the number of ears leads insufficient performance such as accuracy in sound localization, or sound source separation. In order to overcome those difficulties, this paper proposes a novel artificial soft pinna that can move, or deform its shape actively. The developed pinna has deformable skin of silicon rubber with wires controlled by motors, and it is covered by a fur cover. Considering applications of the proposed device for auditory robots, fundamental characteristics such as accuracy of motion and auditory characteristics were investigated through experiments, which is the main contribution of the paper.
---
paper_title: The Cog Project : Building a Humanoid Robot

paper_content:

To explore issues of developmental structure, physical embodiment, integration of multiple sensory and motor systems, and social interaction, we have constructed an upper-torso humanoid robot called Cog. The robot has twenty-one degrees of freedom and a variety of sensory systems, including visual, auditory, vestibular, kinesthetic, and tactile senses. This chapter gives a background on the methodology that we have used in our investigations, highlights the research issues that have been raised during this project, and provides a summary of both the current state of the project and our long-term goals. We report on a variety of implemented visual-motor routines (smooth-pursuit tracking, saccades, binocular vergence, and vestibular-ocular and opto-kinetic reflexes), orientation behaviors, motor control techniques, and social behaviors (pointing to a visual target, recognizing joint attention through face and eye finding, imitation of head nods, and regulating interaction through expressive feedback). We further outline a number of areas for future research that will be necessary to build a complete embodied system.
---
paper_title: Sound Localization for Humanoid Robots - Building Audio-Motor Maps based on the HRTF

paper_content:

Being able to locate the origin of a sound is important for our capability to interact with the environment. Humans can locate a sound source in both the horizontal and vertical plane with only two ears, using the head related transfer function HRTF, or more specifically features like interaural time difference ITD, interaural level difference ILD, and notches in the frequency spectra. In robotics notches have been left out since they are considered complex and difficult to use. As they are the main cue for humans' ability to estimate the elevation of the sound source this have to be compensated by adding more microphones or very large and asymmetric ears. In this paper, we present a novel method to extract the notches that makes it possible to accurately estimate the location of a sound source in both the horizontal and vertical plane using only two microphones and human-like ears. We suggest the use of simple spiral-shaped ears that has similar properties to the human ears and make it easy to calculate the position of the notches. Finally we show how the robot can learn its HRTF and build audiomotor maps using supervised learning and how it automatically can update its map using vision and compensate for changes in the HRTF due to changes to the ears or the environment.
---
paper_title: Auditory fovea based speech separation and its application to dialog system

paper_content:

This paper presents an active direction-pass filter (ADPF) that separates sounds originating from the specified direction by using a pair of microphones. Its application to front-end processing for speech recognition is also reported. Since the performance of sound source separation by the ADPF depends on the accuracy of sound source localization (direction), various localization modules including the interaural phase difference, interaural intensity difference for each sub-band, and other visual and auditory processing are integrated hierarchically. The resulting performance of auditory localization varies according to the relative position of the sound source. The resolution of the center of the robot is much higher than that of peripherals, indicating similar property of visual fovea. To make the best use of this property, the ADPF controls the direction of a head by motor movement. In order to recognize sound streams separated by the ADPF, a hidden Markov model based automatic speech recognition is built with multiple acoustic models trained by the output of the ADPF under different conditions. A preliminary dialog system is thus implemented on an upper-torso humanoid. The experimental results prove that it works well even when two speakers speak simultaneously.
---
paper_title: Using binaural and spectral cues for azimuth and elevation localization

paper_content:

It is a common assumption that with just two microphones only the azimuth angle of a sound source can be estimated and that a third, orthogonal microphone (or set of microphones) is necessary to estimate the elevation of the source. Recently, using specially designed ears and analyzing spectral cues several researchers managed to estimate sound source elevation with a binaural system. In this work, we show that with two bionic ears both azimuth and elevation angle can be determined using both binaural (e.g. IID and ITD) and spectral cues. This ability can also be used to disambiguate signals coming from the front or back. We present a detailed analysis of both azimuth and elevation localization performance for binaural and spectral cues in comparison. We demonstrate that with a small extension of a standard binaural system a basic elevation estimation capacity can be gained.
---
paper_title: Audio servo for robotic systems with pinnae

paper_content:

Sound localization is one of important abilities for robots which utilize auditory information. Binaural information such as interaural time difference (ITD) or interaural intensity difference(IID) is effective for horizontal sound localization. In order to extend this ability to vertical sound localization, spectral cues were utilized in this paper instead of increasing number of microphones. Spectral cues were the information about the vertical position of the sound source on frequency domain. Since spectral cues strongly depended on the shape of pinnae, artificial pinnae for auditory robots were designed based on diffraction-reflection model. A method to detect spectral cues was also developed. By utilizing them, a controller which made the robot orient to the sound source was proposed. Results of an experiment showed the validity of the proposed method.
---
paper_title: Learning sound location from a single microphone

paper_content:

We consider the problem of estimating the incident angle of a sound, using only a single microphone. The ability to perform monaural (single-ear) localization is important to many animals; indeed, monaural cues are also the primary method by which humans decide if a sound comes from the front or back, as well as estimate its elevation. Such monaural localization is made possible by the structure of the pinna (outer ear), which modifies sound in a way that is dependent on its incident angle. In this paper, we propose a machine learning approach to monaural localization, using only a single microphone and an “artificial pinna” (that distorts sound in a direction-dependent way). Our approach models the typical distribution of natural and artificial sounds, as well as the direction-dependent changes to sounds induced by the pinna. Our experimental results also show that the algorithm is able to fairly accurately localize a wide range of sounds, such as human speech, dog barking, waterfall, thunder, and so on. In contrast to microphone arrays, this approach also offers the potential of significantly more compact, as well as lower cost and power, devices for sounds localization.
---
paper_title: Spectral cues used in the localization of sound sources on the median plane.

paper_content:

The spectral cues used for median‐plane localization are described by three experiments. First, the frequency spectrum necessary for localization is measured by noting the accuracy of subjects localizing low‐ and high‐pass‐filtered white noise. Second, several high‐pass, low‐pass, bandpass, and bandstop filters are associated with the subjective impression of direction by observing what directions are most frequently perceived by subjects localizing white noise colored by each filter. Third, the frequency responses of several artificial ears are measured for different angles of median‐plane sound incidence. Results show that sound spectra from 4 to 16 kHz are necessary for localization. Frontal cues are a 1‐octave notch, with a lower‐frequency cutoff between 4 and 10 kHz and increased energy above 13 kHz. The “above” cue is a 1/4‐octave peak between 7 and 9 kHz, with a high‐frequency cutoff at 10 kHz. The “behind” cue is a small peak from 10 to 12 kHz. Increases in frontal elevation are signaled by an inc...
---
paper_title: Active soft pinnae for robots

paper_content:

Sound localization is one of fundamental abilities of auditory robots that is necessary for human-machine communication. Especially for autonomous robots, it is also necessary to make the system as simple as possible, and binaural configuration is thought to be the minimum setup for auditory systems as animals achieve practical hearing with two ears. However, the limitation of the number of ears leads insufficient performance such as accuracy in sound localization, or sound source separation. In order to overcome those difficulties, this paper proposes a novel artificial soft pinna that can move, or deform its shape actively. The developed pinna has deformable skin of silicon rubber with wires controlled by motors, and it is covered by a fur cover. Considering applications of the proposed device for auditory robots, fundamental characteristics such as accuracy of motion and auditory characteristics were investigated through experiments, which is the main contribution of the paper.
---
paper_title: Sound Localization for Humanoid Robots - Building Audio-Motor Maps based on the HRTF

paper_content:

Being able to locate the origin of a sound is important for our capability to interact with the environment. Humans can locate a sound source in both the horizontal and vertical plane with only two ears, using the head related transfer function HRTF, or more specifically features like interaural time difference ITD, interaural level difference ILD, and notches in the frequency spectra. In robotics notches have been left out since they are considered complex and difficult to use. As they are the main cue for humans' ability to estimate the elevation of the sound source this have to be compensated by adding more microphones or very large and asymmetric ears. In this paper, we present a novel method to extract the notches that makes it possible to accurately estimate the location of a sound source in both the horizontal and vertical plane using only two microphones and human-like ears. We suggest the use of simple spiral-shaped ears that has similar properties to the human ears and make it easy to calculate the position of the notches. Finally we show how the robot can learn its HRTF and build audiomotor maps using supervised learning and how it automatically can update its map using vision and compensate for changes in the HRTF due to changes to the ears or the environment.
---
paper_title: A physical model of sound diffraction and reflections in the human concha.

paper_content:

An approximated physical model of the frequency transfer function of the human concha is developed in this paper. This formulation includes diffraction, reflection, and interference phenomena in the concha cavity. The performance of the proposed diffraction/ reflection model is compared with that of the single-delay-and-add approximation by checking their predictions against the experimental transfer function of a metal spiral-shaped diffracting/reflecting system. Results show that the diffraction/reflection model performs considerably better at predicting both the absolute center frequency of spectral minima and the relative frequency spacing between them. The diffraction/reflection model is then applied to a realistic concha shape and its predictions are compared with experimental head-related transfer functions for azimuth- and elevation-varying sound sources. In this case, the model predicts the elevation-dependent spectral features related to the transverse dimensions of the concha. Additionally, the diffraction/reflection model predicts that, because of sound diffraction, similar spectral features must be generated in the concha for sources at all azimuths within the frontal part of the ipsilateral hemisphere. Experimental and theoretical evidence supporting this prediction is presented.
---
paper_title: Binaural Sound Source Distance Learning in Rooms

paper_content:

A method for learning the distance of a sound source in a room is presented. The proposed method is based on short-time magnitude-squared coherence between the two channels of a binaural signal. Based on white noise as the training data, a coherence profile is obtained at each desired position in the room. These profiles can then be used to identify the most likely distance of a speech signal in the same room. The proposed approach is compared to a previous method for learning the position of a sound source. The results indicate that the both methods are able to identify the distance of a speech sound source correctly in a grid with 0.5-m spacing in most cases, when the orientation of the listener is 0deg , 30deg , 60deg , 90deg , or 180deg on the horizontal plane.
---
paper_title: Binaural Estimation of Sound Source Distance via the Direct-to-Reverberant Energy Ratio for Static and Moving Sources

paper_content:

One of the principal cues believed to be used by listeners to estimate the distance to a sound source is the ratio of energies along the direct and indirect paths to the receiver. In essence, this “direct-to-reverberant” energy ratio reveals the absolute distance component of the direct energy by normalizing by what is assumed to be distance-independent reverberant energy. Earlier approaches to direct-to-reverberant energy ratio calculation made use of the estimated room impulse response, but these techniques are computationally expensive and inaccurate in practice. This paper proposes and evaluates an alternative approach which uses binaural signals to segregate energy arriving from the estimated direction of the direct source from that arriving from other directions, employing a novel binaural equalization-cancellation technique. The system is integrated with a probabilistic inference framework, particle filtering, to handle the nonstationarity of energy-based measurements. The algorithm is capable of using reverberation to estimate source distance in large rooms with errors of less than 1 m for static sources and 1.5-3.5 m for sources with varying degrees of motion complexity. Model performance can be accounted for largely in terms of a competition between auditory horizon and source energy fluctuation effects.
---
paper_title: Active binaural localization of intermittent moving sources in the presence of false measurements

paper_content:

This paper takes place within the field of active sound source localization in a binaural context. A stochastic filtering strategy is presented for the localization of a still or moving source from a moving binaural sensor. The proposed method accounts for the source intermittence as well as for false measurements induced by the non-stationarity of the emitted signal. Its effectiveness is showed on experimental results.
---
paper_title: Active speaker localization with circular likelihoods and bootstrap filtering

paper_content:

This paper deals with speaker localization in two dimensions from a mobile binaural head. A bootstrap particle filtering scheme is used to perform active localization, i.e. to infer source location by fusing the binaural perception with the sensor motor commands. It relies on an original pseudo-likelihood of the source azimuth which captures both the interaural level and phase differences. Since the pseudo-likelihood is discrete, it is fitted with a mixture of circular distributions in order to enhance its resolution. For the fitting task two mixtures are compared and evaluated, namely the mixture of von Mises and wrapped Cauchy distributions. Furthermore, a solution is presented for calculating the von Mises curvefitting with low uncertainty, since the direct implementation can quickly surpass double precision floating number representation. The performance of the filter is compared using both the raw and fitted pseudo-likelihoods on experiments recorded in an acoustically prepared room with ground-truth obtained from a motion capture system. The results show that the proposed algorithm successfully localizes the speaker with an advantage in the direction of the fitted von Mises mixture likelihood.
---
paper_title: A study on distance estimation in binaural sound localization

paper_content:

The position of a sound source is an important information for robotic systems to be extracted from a sound. Of the three spherical coordinates (azimuth, elevation, distance) only the azimuth direction is extracted in most robot audition systems. So far rarely investigated is the issue of estimating the distance between robot and sound source. In this article we describe a study on distance estimation using a binaural robot system in an indoor environment for sounds ranging in distance from 0.5 to 6m. We investigated several proposed audio cues like interaural differences (IID and ITD), sound amplitude, and spectral characteristics. All cues are computed within the framework of audio proto objects. In an extensive experimental setup with more than 10000 sounds we found that both mean signal amplitude and binaural cues can, under certain circumstances, provide a very reliable distance estimation. There was no observable effect of frequency dependent attenuation so that the spectral amplitude cue was only slightly above chance level. We also investigated the loss of precision of azimuth estimation with distance. In contrast to what could be expected, the performance does not severely deteriorate when the system is calibrated for different distances.
---
paper_title: A versatile System-on-a-Programmable-Chip for array processing and binaural robot audition

paper_content:

This paper champions the use of Systems-on-a-Programmable-Chip (SoPCs) on FPGAs for robot audition. Technical and methodological aspects are discussed. On this basis, evolutions of the “Embedded Audition for Robotics” (EAR) sensor are presented. A versatile design has been obtained for array processing, enabling to implement information-theoretic multiple source detection and broadband beamspace MUSIC source localization at up to 45Hz. Recent evolutions towards binaural and large-scale arrays are finally discussed.
---
paper_title: Sensorimotor learning of sound localization from an auditory evoked behavior

paper_content:

A new method for self-supervised sensorimotor learning of sound source localization is presented, that allows a simulated listener to learn online an auditorimotor map from the sensorimotor experience provided by an auditory evoked behavior. The map represents the auditory space and is used to estimate the azimuthal direction of sound sources. The learning mainly consists in non-linear dimensionality reduction of sensorimotor data. Our results show that an auditorimotor map can be learned, both from real and simulated data, and that the online learning leads to accurate estimations of azimuthal sources direction.
---
paper_title: Binaural model for artificial spatial sound localization based on interaural time delays and movements of the interaural axis.

paper_content:

This paper presents a mathematical model for sound localization in space using two-microphone devices that possess at least two degrees of freedom. It proves a series of theorems and lemmas that are based on time difference of arrival measurements and movements of the interaural axis, forming a powerful instrument for practical robot applications. For instance, it shows that a single determined rotation of the interaural axis is sufficient to exactly yield the azimuth or the elevation of an immobile sound source in the far field, independently of microphone spacing and the speed of sound and hence of the surrounding medium. It proves that at any moment the knowledge of one value determines the magnitude of the other, with the restriction that the sign of the second value is undefined, which means that, depending on the rotation, either the back-front or the up-down ambiguity is kept unsolved. This paper also shows that parallax motion unlocks essential information about the distance and the Cartesian coordinates of the sound source. Shifting the microphone system sideways fixes the distance and the coordinate on the interaural axis. Combining rotation and translation movements completely solves the localization problem. In order to illustrate the efficacy of the model, this paper presents experiments with a low cost robot developer kit during which the azimuth, the elevation, and the distance of continuous sound sources are determined at a precision of 10 degrees and 0.5 m, respectively. Achieving this performance with low power material demonstrates how easily the model can be implemented into any robotic system.
---
paper_title: Design and Implementation of Robot Audition System 'HARK' — Open Source Software for Listening to Three Simultaneous Speakers

paper_content:

This paper presents the design and implementation of the HARK robot audition software system consisting of sound source localization modules, sound source separation modules and automatic speech recognition modules of separated speech signals that works on any robot with any microphone configuration. Since a robot with ears may be deployed to various auditory environments, the robot audition system should provide an easy way to adapt to them. HARK provides a set of modules to cope with various auditory environments by using an open-sourced middleware, FlowDesigner, and reduces the overheads of data transfer between modules. HARK has been open-sourced since April 2008. The resulting implementation of HARK with MUSIC-based sound source localization, GSS-based sound source separation and Missing Feature Theory-based automatic speech recognition on Honda ASIMO, SIG2 and Robovie R2 attains recognizing three simultaneous utterances with the delay of 1.9 s at the word correct rate of 80–90% for three speakers.
---
paper_title: Motion strategies for binaural localisation of speech sources in azimuth and distance by artificial listeners

paper_content:

Localisation in azimuth and distance of sound sources such as speech is an important ability for both human and artificial listeners. While progress has been made, particularly for azimuth estimation, most work has been directed at the special case of static listeners and static sound sources. Although dynamic sound sources create their own localisation challenges such as motion blur, moving listeners have the potential to exploit additional cues not available in the static situation. An example is motion parallax, based on a sequence of azimuth estimates, which can be used to triangulate sound source location. The current study examines what types of listener (or sensor) motion are beneficial for localisation. Is any kind of motion useful, or do certain motion trajectories deliver robust estimates rapidly? Eight listener motion strategies and a no motion baseline were tested, including simple approaches such as random walks and motion limited to head rotations only, as well as more sophisticated strategies designed to maximise the amount of new information available at each time step or to minimise the overall estimate uncertainty. Sequential integration of estimates was achieved using a particle filtering framework. Evaluations, performed in a simulated acoustic environment with single sources under both anechoic and reverberant conditions, demonstrated that two strategies were particularly effective for localisation. The first was simply to move towards the most likely source location, which is beneficial in increasing signal-to-noise ratio, particularly in reverberant conditions. The other high performing approach was based on moving in the direction which led to the largest reduction in the uncertainty of the location estimate. Both strategies achieved estimation errors nearly an order of magnitude less than those obtainable with a static approach, demonstrating the power of motion-based cues to sound source localisation.
---
paper_title: Motion planning based on simultaneous perturbation stochastic approximation for mobile auditory robots

paper_content:

In this paper, a motion planning method for mobile auditory robots is proposed based on an optimization technique. Since it is one of the most important abilities for auditory robots to recognize vocal messages correctly, the proposed method is designed to maximize the confidence measure of a speech recognition since the measure is thought to be strongly related to the accuracy of the speech recognition. However, the cost function to optimize is hard to model explicitly, and it is difficult to obtain the gradient that is normally utilized to derive the motion. In order to overcome this difficulty, simultaneous perturbation stochastic approximation(SPSA) that does not require an explicit model of the cost function is applied to generate robot motion. The effectiveness of the approach was verified through real experiments: the robot could get better speech recognition rate after it approached the sound source by measuring the confidence measure.
---
paper_title: Active hearing, active speaking

paper_content:

A static view of the world permeates most research in speech and hearing. In this idealised situation, sources don’t move and neither do listeners; the acoustic environment doesn’t change; and speakers speak without any effect of auditory input from their own voice or other speakers. Corpora for speech research and most behavioural tasks have grown to reflect the static viewpoint. Yet it is clear that speech and hearing takes place in a world where none of the static assumptions hold, or at least not for long. The dynamic view complicates tradi- tional signal processing approaches, and renders conventional evaluation processes unrepeatable since the observer’s dynamics influence the signals received at the ears. However, the dynamic viewpoint also provides many opportunities for active processes to exploit. Some of these, such as the use of head movements to resolve front-back confusions, are well-known, while others exist solely as hypotheses. This paper reviews known and potential benefits of active processes in both hearing and speech production, and goes on to describe two recent studies which demonstrate the value of such processes. The first shows how dynamic cues can be used to estimate distance in an acoustic environment. The second demonstrates that the changes in speech production which take place when other speakers are active result in increased glimpsing opportunities at the ear of the interlocutor.
---
paper_title: Epipolar geometry based sound localization and extraction for humanoid audition

paper_content:

Sound localization for a robot or an embedded system is usually solved by using inter-aural phase difference (IPD) and inter-aural intensity difference (IID). These values are calculated by using head-related transfer function (HRTF). However, the HRTF depends on the shape of the head and also on changes of the environments. Therefore, sound localization without HRTF is needed for real-world applications. In this paper, we present a new sound localization method based on auditory epipolar geometry with motion control. The auditory epipolar geometry is an extension of an epipolar geometry in stereo vision to audition, and auditory and visual epipolar geometries can share the sound source direction. The key idea is to exploit additional inputs obtained by the motor control in order to compensate damages in the IPD and IID caused by reverberation of the room and the body of the robot. The proposed system can localize and extract simultaneously two sound sources in a real-world room.
---
paper_title: A Scale for the Measurement of the Psychological Magnitude Pitch

paper_content:

A subjective scale for the measurement of pitch was constructed from determinations of the half‐value of pitches at various frequencies. This scale differs from both the musical scale and the frequency scale, neither of which is subjective. Five observers fractionated tones of 10 different frequencies at a loudness level of 60 db. From these fractionations a numerical scale was constructed which is proportional to the perceived magnitude of subjective pitch. In numbering the scale the 1000‐cycle tone was assigned the pitch of 1000 subjective units (mels).The close agreement of the pitch scale with an integration of the differential thresholds (DL's) shows that, unlike the DL's for loudness, all DL's for pitch are of uniform subjective magnitude. The agreement further implies that pitch and differential sensitivity to pitch are both rectilinear functions of extent on the basilar membrane.The correspondence of the pitch scale and the experimentally determined location of the resonant areas of the basilar me...
---
paper_title: The generalized correlation method for estimation of time delay

paper_content:

A maximum likelihood (ML) estimator is developed for determining time delay between signals received at two spatially separated sensors in the presence of uncorrelated noise. This ML estimator can be realized as a pair of receiver prefilters followed by a cross correlator. The time argument at which the correlator achieves a maximum is the delay estimate. The ML estimator is compared with several other proposed processors of similar form. Under certain conditions the ML estimator is shown to be identical to one proposed by Hannan and Thomson [10] and MacDonald and Schultheiss [21]. Qualitatively, the role of the prefilters is to accentuate the signal passed to the correlator at frequencies for which the signal-to-noise (S/N) ratio is highest and, simultaneously, to suppress the noise power. The same type of prefiltering is provided by the generalized Eckart filter, which maximizes the S/N ratio of the correlator output. For low S/N ratio, the ML estimator is shown to be equivalent to Eckart prefiltering.
---
paper_title: Acoustic event localization using a crosspower-spectrum phase based technique

paper_content:

Linear microphone arrays can be employed for acoustic event localization in a noisy environment using time delay estimation. Three techniques are investigated that allow delay estimation, namely normalized cross correlation, LMS adaptive filters, crosspower-spectrum phase: they are combined with a bidimensional representation, the coherence measure, in order to emphasize information that can be exploited for estimating position of both non-moving and moving acoustic sources. To compare the given techniques, different acoustic sources were considered, that generated events in different positions in space. Expressing performance in terms of accuracy of the wavefront direction angle, experiments showed that the crosspower-spectrum phase based technique outperforms the other two. This technique provided very promising preliminary results also in terms of source position estimation. >
---
paper_title: The smoothed coherence transform

paper_content:

The smoothed coherence transform (SCOT) is defined and examples of its uses and shortcomings are given. Computation of this function shows promise for measuring time delays between weak broad-band correlated noises received at two sensors.
---
paper_title: Estimating group delay

paper_content:

SUMMARY It is suggested that in estimating spectra and cross spectra for a pair of stationary series the estimation of the phase may be replaced by an estimate of the group delay, which is the derivative of phase with respect to frequency. An estimate is proposed and its asymptotic properties are investigated. The results are illustrated by simulation and by application to real data. The estimation of a group delay that is constant over a nondegenerate band is also discussed.
---
paper_title: Applying scattering theory to robot audition system: robust sound source localization and extraction

paper_content:

Robot audition by its own ears (microphones) is essential for natural human-robot communication and interface. Since a microphone is embedded in the head of a robot, the head-related transfer function (HRTF) plays an important role in sound source localization and extraction. Usually, from binaural input, the interaural phase difference (IPD) and interaural intensity difference (IID) are calculated, and then the direction is determined by using IPD and IID with HRTF. The problem of HRTF-based sound source localization is that a HRTF should be measured for each robot in an anechoic chamber, because it depends on the shape of robot's head; HRTF should be interpolated to manipulate a moving talker, because it is available only for discrete azimuth and elevation. To cope with these problems of HRTF, we proposed the auditory epipolar geometry as a continuous function of IPD and IID to dispense with HRTF and have developed a real-time multiple-talker tracking system. This auditory epipolar geometry, however, does not give a good approximation to IID of all range and IPD of peripheral areas. In this paper, the scattering theory in physics is employed to take into consideration the diffraction of sounds around robot's head for better approximation of IID and IPD. The resulting system shows that it is efficient for localization and extraction of sound at higher frequency and from side directions.
---
paper_title: Broadband variations of the MUSIC high-resolution method for Sound Source Localization in Robotics

paper_content:

The MUSIC algorithm (multiple signal classification) is a well-known high-resolution method to sound source localization. However, as it is essentially narrowband, several extensions can be envisaged when dealing with broadband sources like human voice. This paper presents such extensions and proposes a comparative study w.r.t. specific robotics constraints. An online beamspace MUSIC method, together with a recently developed beamforming scheme, are shown to constitute a mathematically sound and potentially efficient solution.
---
paper_title: A versatile System-on-a-Programmable-Chip for array processing and binaural robot audition

paper_content:

This paper champions the use of Systems-on-a-Programmable-Chip (SoPCs) on FPGAs for robot audition. Technical and methodological aspects are discussed. On this basis, evolutions of the “Embedded Audition for Robotics” (EAR) sensor are presented. A versatile design has been obtained for array processing, enabling to implement information-theoretic multiple source detection and broadband beamspace MUSIC source localization at up to 45Hz. Recent evolutions towards binaural and large-scale arrays are finally discussed.
---
paper_title: Robust Sound Source Localization Using a Microphone Array on a Mobile Robot

paper_content:

The hearing sense on a mobile robot is important because it is omnidirectional and it does not require direct line-of-sight with the sound source. Such capabilities can nicely complement vision to help localize a person or an interesting event in the environment. To do so the robot auditory system must be able to work in noisy, unknown and diverse environmental conditions. In this paper, we present a robust sound source localization method in three-dimensional space using an array of 8 microphones. The method is based on time delay of arrival estimation. Results show that a mobile robot can localize in real time different types of sound sources over a range of 3 meters and with a precision of 3/spl deg/.
---
paper_title: Robust 3D Localization and Tracking of Sound Sources Using Beamforming and Particle Filtering

paper_content:

In this paper we present a new robust sound source localization and tracking method using an array of eight microphones (US patent pending). The method uses a steered beamformer based on the reliability-weighted phase transform (RWPHAT) along with a particle filter-based tracking algorithm. The proposed system is able to estimate both the direction and the distance of the sources. In a videoconferencing context, the direction was estimated with an accuracy better than one degree while the distance was accurate within 10% RMS. Tracking of up to three simultaneous moving speakers is demonstrated in a noisy environment.
---
paper_title: Robust estimation of sound direction for robot interface

paper_content:

This paper proposes a robust method to estimate sound direction for mobile robot interface. In contrast to other methods, the proposed method estimates sound direction at intervals of short time to deal with deterioration by movement of sound source or robot. The two methods, discrete Kalman filter and time delay of arrival (TDOA) using generalized cross-correlation (GCC), are combined by quantifying reliability of TDOA. The combination prevents the deterioration by low power of signals, noise, and reverberation because discrete Kalman filter efficiently filters out such effects. The result of experiments, in which mobile robot estimates azimuth angle with three microphones, shows that the proposed method provides robustness on estimating sound direction even if the sound source or robot is moving during estimation procedure.
---
paper_title: The contribution of the near and far ear toward localization of sound in the sagittal plane.

paper_content:

Eight listeners were required to locate a train of 4.5-kHz high-pass noise bursts emanating from loudspeakers positioned +/- 30, +/- 20, +/- 10, and 0 deg re: interaural axis. The vertical array of loudspeakers was placed at 45, 90, and 135 deg left of midline. The various experimental conditions incorporated binaural and monaural listening with the latter utilizing the ear nearest or ear farthest from the sound source. While performance excelled when listening with only the near ear, the contribution of the far ear was statistically significant when compared to localization performance when both ears were occluded. Based on head related transfer functions for stimuli whose bandwidth was 1.0 kHz, four spectral cues were selected as candidates for influencing location judgments. Two of them associated relative changes in energy across center frequencies (CFs) with vertical source positions. The other two associated an absolute minimum (maximum) energy for specific CFs with a vertical source position. All but one cue when measured for the near ear could account for localization proficiency. On the other hand, when listening with the far ear, maximum energy at a specific CF outperformed the remaining cues in accounting for localization proficiency.
---
paper_title: Sound source localization and signal separation for office robot "JiJo-2"

paper_content:

A method of sound source localization and source separation for the sound sources in the near-field range is described in this paper. For the sound localization, the subspace method was employed to obtain a high resolution which is necessary for the range estimation with a small-sized array. The modified MVBF with the estimated localization information was used for signal separation. In an experiment conducted in a middle-sized meeting room, 60-70% of word recognition rate was achieved.
---
paper_title: 3D Position Sensing Using the Differences in the Time-of-flights from a Wave Source to Various Receivers

paper_content:

This paper presents a novel formulation for the estimation of the coordinates of a wave source based on the differences in the time-of-flights from a single transmitter to various receivers fixed in 3D space. The formulation is closed form linear, and estimates the speed of sound at every ranging operation. This leads to a robust system that is insensitive to changes in the environmental conditions. Typical applications that will benefit from this technology are 3D position sensing systems that may be used in robotics, navigation of autonomously guided vehicles, tracking of objects for virtual reality cells, and vibration analysis.
---
paper_title: Intelligent Sound Source Localization and its application to multimodal human tracking

paper_content:

We have assessed robust tracking of humans based on intelligent Sound Source Localization (SSL) for a robot in a real environment. SSL is fundamental for robot audition, but has three issues in a real environment: robustness against noise with high power, lack of a general framework for selective listening to sound sources, and tracking of inactive and/or noisy sound sources. To address the first issue, we extended Multiple SIgnal Classification by incorporating Generalized EigenValue Decomposition (GEVD-MUSIC) so that it can deal with high power noise and can select target sound sources. To address the second issue, we proposed Sound Source Identification (SSI) based on hierarchical gaussian mixture models and integrated it with GEVD-MUSIC to realize a selective listening function. To address the third issue, we integrated audio-visual human tracking using particle filtering. Integration of these three techniques into an intelligent human tracking system showed: 1) GEVD-MUSIC improved the noise-robustness of SSL by a signal-to-noise ratio of 5–6 dB; 2) SSI performed more than 70% in F-measure even in a noisy environment; and 3) audio-visual integration improved the average tracking error by approximately 50%.
---
paper_title: Posture estimation of hose-shaped robot using microphone array localization

paper_content:

This paper presents a posture estimation of hose-shaped robot using microphone array localization. The hose-shaped robots, one of major rescue robots, have problems with navigation because their posture is too flexible for a remote operator to control to go as far as desired. For navigational and mission usability, the posture estimation of the hose-shaped robot is essential. We developed a posture estimation method with a microphone array and small loudspeakers equipped on the hose-shaped robot. Our method consists of two steps: (1) playing a known sound from the loudspeaker one-by-one, and (2) estimating the microphone positions on the hose-shaped robot instead of estimating the posture directly. We designed a time difference of arrival (TDOA) estimation method to be robust against directional noise and implemented a prototype system using a posture model of the hose-shaped robot and an Extended Kalman Filter (EKF). The validity of our approach is evaluated by the experiments with both signals recorded in an anechoic chamber and simulated data.
---
paper_title: Real-time speaker localization and speech separation by audio-visual integration

paper_content:

Robot audition in real-world should cope with motor and other noises caused by the robot's own movements in addition to environmental noises and reverberation. This paper reports how auditory processing is improved by audio-visual integration with active movements. The key idea resides in hierarchical integration of auditory and visual streams to disambiguate auditory or visual processing. The system runs in real-time by using distributed processing on 4 PCs connected by a Gigabit Ethernet. The system implemented in a upper-torso humanoid tracks multiple talkers and extracts speech from a mixture of sounds. The performance of epipolar geometry based sound source localization and sound source separation by active and adaptive direction-pass filtering is also reported.
---
paper_title: Towards a systematic study of binaural cues

paper_content:

Sound source localization is a need for robotic systems interacting with acoustically-active environments. In this domain, numerous binaural localization studies have been conducted within the last few decades. This paper provides an overview of a number of binaural localization cue extraction techniques. These are carefully addressed and applied on a simulated binaural database. Cues are evaluated in azimuth estimation and their discriminatory effectiveness is studied as a function of the reverberation time with statistical data analysis techniques. Results show that big differences exist between the discriminatory abilities of multiple types of cue extraction methods. Thus a careful cue selection must be performed before establishing a sound localization system.
---
paper_title: Range and bearing estimation of wideband sources using an orthogonal beamspace processing structure

paper_content:

We propose a new method for range and bearing estimation of wideband sources based on a novel beamspace structure. The proposed structure consists of an orthogonal set of beamspace processors that have a frequency-invariant beampattern property, and can be steered easily to any nearfield range while maintaining the frequency-invariance of the spatial response. These properties mean that the proposed structure can perform coherent signal subspace processing for nearfield wideband sources, making it ideal for wideband range and bearing estimation. Performance of the estimator is demonstrated through Monte Carlo simulations.
---
paper_title: The effects of microphone array processing on pitch extraction in real noisy environments

paper_content:

Pitch extraction is important for communication robots, since pitch may carry information about intention, attitude or emotion expression from the user's speech. However, current pitch extraction methods are not robust enough in real noisy environments. In the present work, we propose pitch extraction methods by combining microphone array and auditory scene analysis technologies, and evaluate pitch extraction of multiple speakers in real noisy environments. Evaluation results show that the proposed ML-PSACF (maximum likelihood adaptive beamformer with peak-pruned summary autocorrelation function) contributes to reduce the effects of interference and noise, leading to improvements of 23%±5% on pitch estimation rates, in comparison to the baseline of not using array processing.
---
paper_title: Listening for people: Exploiting the spectral structure of speech to robustly perceive the presence of people

paper_content:

As the desire to see robots ubiquitous in society grows, so does the need for providing the robots with the means of building awareness of any humans with which it may be sharing the environment. This paper presents a real-world suitable system which enables robots to robustly perceive the presence of people acoustically. The proposed binaural system first identifies voiced signal by means of a novel approach to Voice Activity Detection that exploits the spectral signature and characteristics of speech without reliance on a priori knowledge. Bearing estimates for each speaker are then made using a multi-track particle filter with a belief update function comprised of a Cross-correlation bearing estimate and an estimate of the speaker's fundamental frequency. Results, from an evaluation of each of the major system components and a system evaluation in which the robot successfully built human-centric situational awareness of the three humans with which it shared an office lunch-room containing typical background noises, are presented and discussed.
---
paper_title: Search and track power charge docking station based on sound source for autonomous mobile robot applications

paper_content:

The objectives of this paper is to develop a system for an autonomous mobile robot for searching and tracking sound source based target. The target can be a power charge docking station. Finding objects or events by following a sound source direction is an intuitive response for human and animal when they cannot see the target. However, there are relative fewer discussions about sound source tracking in robotics field. We design a navigation algorithm integrated with time delay of arrival (TDOA) techniques based on cross-correlation. To confirm the practicality of our approach, we design a robot docking scenario and develop an autonomous mobile robot with two microphone pairs, a camera and a laser range finder to conduct the experiments. According to the experimental results, our approach reduces the complex localization problems to save computing power, and have an acceptable result in an indoor environment.
---
paper_title: Acoustic robot navigation using distributed microphone arrays

paper_content:

Abstract This paper presents a method for the navigation of a mobile robot using sound localization in the context of a robotic lab tour guide. Sound localization, which is achieved using an array of 24 microphones distributed on two walls of the lab, is performed whenever the robot speaks as part of the tour. The SRP-PHAT sound localization algorithm is used to estimate the current location of the robot using approximately 2 s of recorded signal. Navigation is achieved using several stops during which the estimated location of the robot is used to make course adjustments. Experiments using the acoustic robot navigation system illustrate the accuracy of the proposed technique, which resulted in an average localization error of about 7 cm close to the array and 30 cm far away from the array.
---
paper_title: Auditory robotic tracking of sound sources using hybrid cross-correlation and recurrent networks

paper_content:

This paper describes an auditory robotic system capable of computing the angle of incidence of a sound source on the horizontal plane (azimuth). The system, with the use of an Elman type recurrent neural network (RNN), is able to dynamically track this sound source as it changes azimuthally within the environment. The RNN is used to enable fast tracking responses to the overall system over a set time, as opposed to waiting for the next sound position before moving. The system is first tested in a simulated environment and then these results are compared with testing on the robotic system. The results show that the development of a hybrid system incorporating cross-correlation and recurrent neural networks is an effective mechanism for the control of a robot that tracks sound sources azimuthally.
---
paper_title: Using multiple microphone arrays and reflections for 3D localization of sound sources

paper_content:

We proposed a method for estimating sound source locations in a 3D space by integrating sound directions estimated by multiple microphone arrays and taking advantage of reflection information. Two types of sources with different directivity properties (human speech and loudspeaker speech) were evaluated for different positions and orientations. Experimental results showed the effectiveness of using reflection information, depending on the position and orientation of the sound sources relative to the array, walls, and the source type. The use of reflection information increased the source position detection rates by 10% on average and up to 60% for the best case.
---
paper_title: Continuous sound source localization based on microphone array for mobile robots

paper_content:

It is a great challenge to perform a sound source localization system for mobile robots, because noise and reverberation in room pose a severe threat for continuous localization. This paper presents a novel approach named guided spectro-temporal (ST) position localization for mobile robots. Firstly, since generalized cross-correlation (GCC) function based on time delay of arrival (TDOA) can not get accurate peak, a new weighting function of GCC named PHAT-ργ is proposed to weaken the effect of noise while avoiding intense computational complexity. Secondly, a rough location of sound source is obtained by PHAT-ργ method and room reverberation is estimated using such location as priori knowledge. Thirdly, ST position weighting functions are used for each cell in voice segment and all correlation functions from all cells are integrated to obtain a more optimistical location of sound source. Also, this paper presents a fast, continuous localization method for mobile robots to determine the locations of a number of sources in real-time. Experiments are performed with four microphones on a mobile robot. 2736 sets of data are collected for testing and more than 2500 sets of data are used to obtain accurate results of localization. Even if the noise and reverberation are serious. The proportion data is 92% with angle error less than 15 degrees. What's more, it takes less than 0.4 seconds to locate the position of sound source for each data.
---
paper_title: Integration of sound source localization and separation to improve Dialogue Management on a robot

paper_content:

To demonstrate the influence of an artificial audition system on speech recognition and dialogue management for a robot, this paper presents a case study involving soft coupling of ManyEars, a sound source localization, tracking and separation system, with the CSLU Dialogue Management system. Trials were conducted in a laboratory and a cafeteria. Results indicate that preprocessing of the audio signals by ManyEars improves speech recognition and dialogue management of the system, demonstrating the feasibility and the added flexibility provided by ManyEars for a robot to interact vocally with humans in a wide variety of contexts.
---
paper_title: Outdoor auditory scene analysis using a moving microphone array embedded in a quadrocopter

paper_content:

This paper addresses auditory scene analysis, especially, sound source localization using an aerial vehicle with a microphone array in an outdoor environment. Since such a vehicle is able to search sound sources quickly and widely, it is useful to detect outdoor sound sources, for instance, to find distressed people in a disaster situation. In such an environment, noise is quite loud and dynamically-changing, and conventional microphone array techniques studied in the field of indoor robot audition are of less use. We, thus, proposed MUltiple SIgnal Classification based on incremental Generalized EigenValue Decomposition (iGEVD-MUSIC). It can deal with high power noise by introducing a noise correlation matrix and GEVD even when the signal-to-noise ratio is less than 0 dB. In addition, the noise correlation matrix is incrementally estimated to adapt to dynamic changes in noise. We developed a prototype system for auditory scene analysis based on the proposed method using the Parrot AR.Drone with a microphone array and a Kinect device. Experimental results using the prototype system showed that dynamically-changing noise is properly suppressed with the proposed method and multiple human voice sources are able to be localized even when the AR.Drone is moving in an outdoor environment.
---
paper_title: Noise correlation matrix estimation for improving sound source localization by multirotor UAV

paper_content:

A method has been developed for improving sound source localization (SSL) using a microphone array from an unmanned aerial vehicle with multiple rotors, a “multirotor UAV”. One of the main problems in SSL from a multirotor UAV is that the ego noise of the rotors on the UAV interferes with the audio observation and degrades the SSL performance. We employ a generalized eigenvalue decomposition-based multiple signal classification (GEVD-MUSIC) algorithm to reduce the effect of ego noise. While GEVD-MUSIC algorithm requires a noise correlation matrix corresponding to the auto-correlation of the multichannel observation of the rotor noise, the noise correlation is nonstationary due to the aerodynamic control of the UAV. Therefore, we need an adaptive estimation method of the noise correlation matrix for a robust SSL using GEVD-MUSIC algorithm. Our method uses a Gaussian process regression to estimate the noise correlation matrix in each time period from the measurements of self-monitoring sensors attached to the UAV such as the pitch-roll-yaw tilt angles, xyz speeds, and motor control values. Experiments compare our method with existing SSL methods in terms of precision and recall rates of SSL. The results demonstrate that our method outperforms existing methods, especially under high signal-to-noise-ratio conditions.
---
paper_title: Coherent signal-subspace processing for the detection and estimation of angles of arrival of multiple wide-band sources

paper_content:

This paper presents a method of constructing a single signal subspace for high-resolution estimation of the angles of arrival of multiple wide-band plane waves. The technique relies on an approximately coherent combination of the spatial signal spaces of the temporally narrow-band decomposition of the received signal vector from an array of sensors. The algorithm is presented, and followed by statistical simulation examples. The performance of the technique is contrasted with other suggested methods and statistical bounds in terms of the determination of the correct number of sources (detection), bias, and variance of estimates of the angles.
---
paper_title: Information-theoretic detection of broadband sources in a coherent beamspace MUSIC scheme

paper_content:

This paper deals with the detection of the number of broadband sources in an acoustic environment from a small-size, embeddable microphone array. The coupling of information-theoretic statistical identification methods with MUSIC schemes, which has long been acknowledged in the array processing community, is first reviewed. From these considerations, a source number detector based either on the Akaike Information Criterion (AIC) or on the Minimum Description Length (MDL) criterion is derived within an original coherent broadband beamspace MUSIC method. This constitutes a theoretically sound solution, well-suited to robotics as it requires no subjective threshold setting and has a limited computational cost. Experimental results validate the approach. All the necessary theoretical background is provided, so that the paper is self-contained.
---
paper_title: Real-time super-resolution Sound Source Localization for robots

paper_content:

Sound Source Localization (SSL) is an essential function for robot audition and yields the location and number of sound sources, which are utilized for post-processes such as sound source separation. SSL for a robot in a real environment mainly requires noise-robustness, high resolution and real-time processing. A technique using microphone array processing, that is, Multiple Signal Classification based on Standard Eigen-Value Decomposition (SEVD-MUSIC) is commonly used for localization. We improved its robustness against noise with high power by incorporating Generalized EigenValue Decomposition (GEVD). However, GEVD-based MUSIC (GEVD-MUSIC) has mainly two issues: 1) the resolution of pre-measured Transfer Functions (TFs) determines the resolution of SSL, 2) its computational cost is expensive for real-time processing. For the first issue, we propose a TF interpolation method integrating time-domain-based and frequency-domain-based interpolation. The interpolation achieves super-resolution SSL, whose resolution is higher than that of the pre-measured TFs. For the second issue, we propose two methods, MUSIC based on Generalized Singular Value Decomposition (GSVD-MUSIC), and Hierarchical SSL (H-SSL). GSVD-MUSIC drastically reduces the computational cost while maintaining noise-robustness in localization. H-SSL also reduces the computational cost by introducing a hierarchical search algorithm instead of using greedy search in localization. These techniques are integrated into an SSL system using a robot embedded microphone array. The experimental result showed: the proposed interpolation achieved approximately 1 degree resolution although we have only TFs at 30 degree intervals, GSVD-MUSIC attained 46.4% and 40.6% of the computational cost compared to SEVD-MUSIC and GEVD-MUSIC, respectively, H-SSL reduces 59.2% computational cost in localization of a single sound source.
---
paper_title: A two-layer probabilistic model based on time-delay compensation for binaural sound localization

paper_content:

Interaural Intensity Difference (IID) and Interaural Time Difference (ITD) are two important cues for robot acoustic localization both in Artificial Intelligence (AI) and Human-Robot Interaction (HRI) areas. However, it is a challenge job to localize a sound source accurately and swiftly only by two acoustic sensors. In this paper, a time-delay compensation based two-layer probabilistic model is presented for binaural sound source localization. In the first layer, a weighting function of Generalized Cross Correlation (GCC) named PHAT-ργ is used in low-frequency to obtain the prior time-delay. And in this layer a crude estimate of azimuth can also be acquired. At the same time, the probability of all possible time-delay lags can be achieved from the training data. In the Second layer, a new improved algorithm of IID based on time-delay compensation(named IIDδτ) is introduced to refine the probability of the azimuth and the elevation. Lastly, localization result is obtained by Bayes-Rule method. Comparing with three state-of-art algorithms, experimental results show that the proposed method has higher accuracy and costs less time for sound source localization.
---
paper_title: Auditory distance perception in humans : A summary of past and present research

paper_content:

Although auditory distance perception is a critical component of spatial hearing, it has received substantially less scienti.c attention than the directional aspects of auditory localization. Here we summarize current knowledge on auditory distance perception, with special emphasis on recent research results. The summary will be structured around three central questions. 1. How accurately can humans estimate the distances of stationary sound sources? We show that this psychophysical relationship is well approximated by a compressive power function, which suggests that listeners systematically underestimate distances to faraway sound sources. 2. What determines perceived sound source distance? We examine the various acoustical and non-acoustical factors thought to contribute to source distance percepts, and summarize the psychophysical literature relevant to each factor. 3. What are the neural correlates to perceived sound source distance? Recent evidence points to the role of areas within right temporal cortex in auditory distance perception, as well as in other spatial tasks in different sensory modalities.
---
paper_title: A study on determination of a sound wave propagation direction for tracing a sound source

paper_content:

We think robots that move in rubble can approach a sound source by tracing the sound wave propagation. It is necessary to determine the sound wave propagation direction for doing these tasks. In this report, we propose the theory and systems for the determination of a sound propagation direction. Furthermore, we confirm that it is possible to trace a sound propagation and approach a sound source in a labyrinth the model of a rubble.
---
paper_title: Modal Analysis Based Beamforming for Nearfield or Farfield Speaker Localization in Robotics

paper_content:

This paper describes a broadband beampattern synthesis method for sound source localization in the nearfield or in the farfield of a mobile robot, with a small-size linear array. The method is based on the theory of Modal Analysis and involves an original convex optimization procedure which benefits from the Parseval relation. The optimized beampattern is obtained by numerically minimizing the worst-case error between the modal coefficients of the array response and those of the reference beampattern, up to a finite rank of the series expansion, over a frequency grid. Simulations illustrate the analytical development.
---
paper_title: Passive sonar applications: target tracking and navigation of an autonomous robot

paper_content:

This paper demonstrates the use of small area acoustic array technology as passive sonar for an autonomous mobile robot sound localization and direction control. Real-time target tracking is based solely on received audio signals.
---
paper_title: Real-Time Sound Source Localization and Separation for Robot Audition

paper_content:

Robot audition in the real world should cope with environment noises and reverberation and motor noises caused by the robot’s own movements. This paper presents the active direction-pass filter (ADPF) to separate sounds originating from the specified direction with a pair of microphones. The ADPF is implemented by hierarchical integration of visual and auditory processing with hypothetical reasoning on interaural phase difference (IPD) and interaural intensity difference (IID) for each subband. In creating hypotheses, the reference data of IPD and IID is calculated by the auditory epipolar geometry on demand. Since the performance of the ADPF depends on the direction, the ADPF controls the direction by motor movement. The human tracking and sound source separation based on the ADPF is implemented on an upper-torso humanoid and runs in real-time with 4 PCs connected over Gigabit ethernet. The signal-to-noise ratio (SNR) of each sound separated by the ADPF from a mixture of two speeches with the same loudness is improved to about 10 dB from 0 dB.
---
paper_title: Real-time 2 dimensional sound source localization by 128-channel huge microphone array

paper_content:

This work describes real-time 2 dimensional sound source localization in an ordinary environment, using a 128-channel huge microphone array. Sound source localization by our microphone array is achieved by the sum and delay beam former (SDBF). A dedicated pci 128-channel simultaneous input analog-to-digital (AD) board with a 5 /spl mu/s DMA transfer was developed for a 128-channel microphone array. Sampling frequency of this microphone array is 11 kHz. In order to keep a fixed cycle such as 11 kHz, "ART-Linux" is used. A sound source can be localized in the range of about /spl plusmn/10 cm error. It takes about 1 s to localize a sound source. Sound source localization by our 128-channel huge microphone array is evaluated by sound pressure distribution. Moreover, our microphone array can be used in order to separate multiple sound sources, reflection and reverberation. Our microphone array is used as an application of a robotics room for human behavior analysis from results of sound source localization and separation etc.
---
paper_title: Three ring microphone array for 3D sound localization and separation for mobile robot audition

paper_content:

This paper describes a three ring microphone array estimates the horizontal/vertical direction and distance of sound sources and separates multiple sound sources for mobile robot audition. Arrangement of microphones is simulated and an optimized pattern which has three rings is implemented with 32 microphones. Sound localization and separation are achieved by delay and sum beam forming (DSBF) and frequency band selection (FBS). From on-line experiments results of sound horizontal and vertical localization, we confirmed that one or two sounds sources could be localized with an error of about 5 degrees and 200 to 300 mm in the case of the distance of about lm. The off-line experiments of sound separation were evaluated by power spectrums in each frequency of separated sounds, and we confirmed that an appropriate frequency band could be selected by DSBF and FBS. The system can separate 3 different pressure speech sources without drowning out.
---
paper_title: Nested iGMM recognition and multiple hypothesis tracking of moving sound sources for mobile robot audition

paper_content:

The paper proposes two modules for a mobile robot audition system: 1) recognizing surrounding acoustic event, 2) tracking moving sound sources. We propose nested infinite Gaussian mixture model (iGMM) for recognizing frame based feature vectors. The main advantage is that the number of classes is allowed to increase without bound, if necessary, to represent unknown audio input. The multiple hypothesis tracking module provides time-series of separated audio stream using localized directions and recognition results at each frame. Not only for continuous sounds, the proposed tracker automatically detects appearing and disappearing point of stream from multiple hypothesis. These two modules are connected to microphone array based sound localization and separation, and the combined robot audition system achieved tracking of multiple moving sounds including intermittent sound source.
---
paper_title: Robust Localization and Tracking of Simultaneous Moving Sound Sources Using Beamforming and Particle Filtering

paper_content:

Mobile robots in real-life settings would benefit from being able to localize and track sound sources. Such a capability can help localizing a person or an interesting event in the environment, and also provides enhanced processing for other capabilities such as speech recognition. To give this capability to a robot, the challenge is not only to localize simultaneous sound sources, but to track them over time. In this paper we propose a robust sound source localization and tracking method using an array of eight microphones. The method is based on a frequency-domain implementation of a steered beamformer along with a particle filter-based tracking algorithm. Results show that a mobile robot can localize and track in real-time multiple moving sources of different types over a range of 7 meters. These new capabilities allow a mobile robot to interact using more natural means with people in real life settings.
---
paper_title: Localization of Simultaneous Moving Sound Sources for Mobile Robot Using a Frequency-Domain Steered Beamformer Approach

paper_content:

Mobile robots in real-life settings would benefit from being able to localize sound sources. Such a capability can nicely complement vision to help localize a person or an interesting event in the environment, and also to provide enhanced processing for other capabilities such as speech recognition. In this paper we present a robust sound source localization method in three-dimensional space using an array of 8 microphones. The method is based on a frequency-domain implementation of a steered beamformer along with a probabilistic post-processor. Results show that a mobile robot can localize in real time multiple moving sources of different types over a range of 5 meters with a response time of 200 ms.
---
paper_title: Spherical microphone array for spatial sound localization for a mobile robot

paper_content:

The present paper introduces a microphone array design and describes the results of an evaluation of the developed microphone array. We propose an evaluation index of the directional characteristics of beamforming to optimize microphone array design. Using beamforming simulation, we obtain a microphone arrangement that minimizes sidelobes and improves the basic performance of beamforming. This microphone arrangement has 64 microphones arranged in a 350-mm-diameter sphere designed to be mounted on a mobile robot and omni-directional directivity in azimuth and elevation. The performance of the proposed microphone array is verified in different real environments. The experimental results of sound localization demonstrate the effectiveness of the array in challenging environments and the robustness of the proposed microphone array for different pressure sound sources to cover larger areas.
---

Title: A survey on sound source localization in robotics: From binaural to array processing methods

Section 1: Introduction
Description 1: Introduce the importance of sound source localization in robotics, highlight human auditory capabilities, and outline the motivation for researching robot audition.

Section 2: Binaural Approaches to Sound Source Localization in Robotics
Description 2: Describe methods that mimic human auditory systems using two sensors, including the basics of Head Related Transfer Functions and binaural auditory cues for localization.

Section 3: The Head Related Transfer Function
Description 3: Define Head Related Transfer Function (HRTF), discuss its significance in robotics, and explain methods of obtaining HRTFs through modeling and measurement.

Section 4: HRTF models in Robotics
Description 4: Review different models proposed for robot audition to capture the robot head effects on binaural signals, including simple head models.

Section 5: Binaural and monaural cues for localization
Description 5: Discuss acoustic features used for localization, including interaural (ITD/ILD) and monaural cues, and their extraction based on human auditory models.

Section 6: Sound source localization in humans
Description 6: Summarize how humans localize sound sources horizontally and vertically using ILD, ITD, and spectral notches.

Section 7: Auditory models in Robotics
Description 7: Describe the implementation of auditory models in robotics to approximate human cochlear functions and improve sound signal perception.

Section 8: Binaural cues for horizontal localization in Robotics
Description 8: Explain the use of IPD and ILD for azimuth localization in robotics, and review various models and approaches for extracting these cues.

Section 9: Exploitation in Robotics
Description 9: Summarize historical and current practices in exploiting binaural localization techniques in robotic applications, highlighting specific methods and their limitations.

Section 10: Array processing approaches to localization in Robot Audition
Description 10: Outline the use of microphone arrays in sound source localization, discussing the advantages of multiple sensors and summarizing common array geometries.

Section 11: Notations and definitions
Description 11: Provide essential notations and definitions used in array processing for sound source localization within the context of robotics.

Section 12: The MUSIC method
Description 12: Describe the Multiple Signal Classification (MUSIC) method for high-resolution localization, discussing its adaptations for real-time constraints in robotics.

Section 13: Localization through correlation
Description 13: Review methods for estimating Time Delays of Arrival (TDOAs) using correlation functions to localize sound sources in robotic environments.

Section 14: Beamforming based approaches
Description 14: Discuss the use of beamforming techniques in robotics for sound source localization, including the design of array filters and the evaluation of array patterns.

Section 15: Conclusion
Description 15: Summarize the survey's findings on sound localization methods in robot audition, emphasizing the challenges, advancements, and future directions in the field.