Emotion Recognition using Speech and EEG Signal – A Review
---
paper_title: Emotion Recognition and Evaluation of Mandarin Speech Using Weighted D-KNN Classification

paper_content:

In this paper, we proposed a weighted discrete K-nearest neighbor (weighted D-KNN) classification algorithm for detecting and evaluating emotion from Mandarin speech. In the experiments of the emotion recognition, Mandarin emotional speech database used contains five basic emotions, including anger, happiness, sadness, boredom and neutral, and the extracted acoustic features are Mel-Frequency Cepstral Coefficients (MFCC) and Linear Prediction Cepstral Coefficients (LPCC). The results reveal that the highest recognition rate is 79.55% obtained with weighted D-KNN optimized based on Fibonacci series. Besides, we design an emotion radar chart which can present the intensity of each emotion in our emotion evaluation system. Based on our emotion evaluation system, we implement a computer-assisted speech training system for training the hearing-impaired people to speak more naturally.
---
paper_title: Analysis of emotion recognition using facial expressions, speech and multimodal information

paper_content:

The interaction between human beings and computers will be more natural if computers are able to perceive and respond to human non-verbal communication such as emotions. Although several approaches have been proposed to recognize human emotions based on facial expressions or speech, relatively limited work has been done to fuse these two, and other, modalities to improve the accuracy and robustness of the emotion recognition system. This paper analyzes the strengths and the limitations of systems based only on facial expressions or acoustic information. It also discusses two approaches used to fuse these two modalities: decision level and feature level integration. Using a database recorded from an actress, four emotions were classified: sadness, anger, happiness, and neutral state. By the use of markers on her face, detailed facial motions were captured with motion capture, in conjunction with simultaneous speech recordings. The results reveal that the system based on facial expression gave better performance than the system based on just acoustic information for the emotions considered. Results also show the complementarily of the two modalities and that when these two modalities are fused, the performance and the robustness of the emotion recognition system improve measurably.
---
paper_title: Speech Recognition by Machine, A Review

paper_content:

This paper presents a brief survey on Automatic Speech Recognition and discusses the major themes and advances made in the past 60 years of research, so as to provide a technological perspective and an appreciation of the fundamental progress that has been accomplished in this important area of speech communication. After years of research and development the accuracy of automatic speech recognition remains one of the important research challenges (e.g., variations of the context, speakers, and environment).The design of Speech Recognition system requires careful attentions to the following issues: Definition of various types of speech classes, speech representation, feature extraction techniques, speech classifiers, database and performance evaluation. The problems that are existing in ASR and the various techniques to solve these problems constructed by various research workers have been presented in a chronological order. Hence authors hope that this work shall be a contribution in the area of speech recognition. The objective of this review paper is to summarize and compare some of the well known methods used in various stages of speech recognition system and identify research topic and applications which are at the forefront of this exciting and challenging field.
---
paper_title: Classification of EEG signals using Dempster Shafer theory and a k-nearest neighbor classifier

paper_content:

A brain computer interface (BCI) is a communication system, which translates brain activity into commands for a computer or other devices. Nearly all BCIs contain as a core component a classification algorithm, which is employed to discriminate different brain activities using previously recorded examples of brain activity. In this paper, we study the classification accuracy achievable with a k-nearest neighbor (KNN) method based on Dempster-Shafer theory. To extract features from the electroencephalogram (EEG), autoregressive (AR) models and wavelet decomposition are used. To test the classification method an EEG dataset containing signals recorded during the performance of five different mental tasks is used. We show that the Dempster-Shafer KNN classifier achieves a higher correct classification rate than the classical voting KNN classifier and the distance-weighted KNN classifier.
---
paper_title: Constraining Minimum-Norm Inverse by Phase Synchronization and Signal Power of the Scalp EEG Channels

paper_content:

In this paper, the goal is to further improve the output of the scalp EEG source localization by the Euclidean minimum-norm (MN) inverse during single trials. Trials have been selected based on signal power at specific time intervals in specific locations. Then the source localization has been performed by MN. It has been observed that close to a dominant cortical source of EEG, as determined by the MN, both pairwise phase synchronization of a channel with its nearest neighbors and the cumulative signal power of the channels within that neighborhood become high (normalized values remain above certain thresholds). This has also been verified through simulations on the subject's real head model. The conclusion of our study is that only those sources are to be chosen for which MN inverse, and signal power and phase synchronization profile converge. A novel fast Fourier transform (FFT) based phase synchronization measuring algorithm between a pair of signals has been developed whose time complexity is no more than that of the FFT.
---
paper_title: Independent component analysis applied in biomedical signal processing

paper_content:

The EEG is composed of electrical potentials arising from several sources. Each source (including separate neural clusters, blink artifact, or pulse artifact) projects a unique topography onto the scalp-'scalp maps'. These maps are mixed according to the principle of linear superposition. Independent component analysis (ICA) attempts to reverse the superposition by separating the EEG into mutually independent scalp maps, or components so that the method used is the study is independent component analysis (ICA). The Polysomnogram (PSG), a complex of more signals recorded over night for the patients with respiratory problems contains also redundant information that can be removed using the ICA.
---

Title: Emotion Recognition using Speech and EEG Signal – A Review
Section 1: INTRODUCTION
Description 1: Introduce the interrelated study in EEG, emotion, and speech, highlighting the importance of recognizing emotions for human-machine communication.

Section 2: EEG BRAIN SIGNALS AND RELATED TECHNIQUES
Description 2: Discuss the nature of EEG brain signals, including types of brain waves and the significance of signal processing techniques for analyzing these signals.

Section 3: AUTOMATIC SPEECH RECOGNITION
Description 3: Present the principles of automatic speech recognition, including methods and features used in the process, such as Mel-Frequency Cepstral Coefficients (MFCC) and pitch contour estimation.

Section 4: EMOTION RECOGNITION
Description 4: Examine various methods for emotion recognition, detailing how subjective and objective factors, alongside neural and hormonal systems, contribute to emotional experiences and behaviors.

Section 5: CONCLUSION
Description 5: Summarize the key points discussed in the paper and the potential for combining speech and EEG signals to enhance emotion recognition systems.