1D Convolutional Neural Networks and Applications: A Survey
---
paper_title: Generalized model of biological neural networks: Progressive operational perceptrons

paper_content:

Traditional Artificial Neural Networks (ANNs) such as Multi-Layer Perceptrons (MLPs) and Radial Basis Functions (RBFs) were designed to simulate biological neural networks; however, they are based only loosely on biology and only provide a crude model. This in turn yields well-known limitations and drawbacks on the performance and robustness. In this paper we shall address them by introducing a novel feed-forward ANN model, Generalized Operational Perceptrons (GOPs) that consist of neurons with distinct (non-)linear operators to achieve a generalized model of the biological neurons and ultimately a superior diversity. We modified the conventional back-propagation (BP) to train GOPs and furthermore, proposed Progressive Operational Perceptrons (POPs) to achieve self-organized and depth-adaptive GOPs according to the learning problem. The most crucial property of the POPs is their ability to simultaneously search for the optimal operator set and train each layer individually. The final POP is, therefore, formed layer by layer and this ability enables POPs with minimal network depth to attack the most challenging learning problems that cannot be learned by conventional ANNs even with a deeper and significantly complex configuration.
---
paper_title: Evolutionary artificial neural networks by multi-dimensional particle swarm optimization

paper_content:

In this paper, we propose a novel technique for the automatic design of Artificial Neural Networks (ANNs) by evolving to the optimal network configuration(s) within an architecture space. It is entirely based on a multi-dimensional Particle Swarm Optimization (MD PSO) technique, which re-forms the native structure of swarm particles in such a way that they can make inter-dimensional passes with a dedicated dimensional PSO process. Therefore, in a multidimensional search space where the optimum dimension is unknown, swarm particles can seek both positional and dimensional optima. This eventually removes the necessity of setting a fixed dimension a priori, which is a common drawback for the family of swarm optimizers. With the proper encoding of the network configurations and parameters into particles, MD PSO can then seek the positional optimum in the error space and the dimensional optimum in the architecture space. The optimum dimension converged at the end of a MD PSO process corresponds to a unique ANN configuration where the network parameters (connections, weights and biases) can then be resolved from the positional optimum reached on that dimension. In addition to this, the proposed technique generates a ranked list of network configurations, from the best to the worst. This is indeed a crucial piece of information, indicating what potential configurations can be alternatives to the best one, and which configurations should not be used at all for a particular problem. In this study, the architecture space is defined over feed-forward, fully-connected ANNs so as to use the conventional techniques such as back-propagation and some other evolutionary methods in this field. The proposed technique is applied over the most challenging synthetic problems to test its optimality on evolving networks and over the benchmark problems to test its generalization capability as well as to make comparative evaluations with the several competing techniques. The experimental results show that the MD PSO evolves to optimum or near-optimum networks in general and has a superior generalization capability. Furthermore, the MD PSO naturally favors a low-dimension solution when it exhibits a competitive performance with a high dimension counterpart and such a native tendency eventually yields the evolution process to the compact network configurations in the architecture space rather than the complex ones, as long as the optimality prevails.
---
paper_title: Heterogeneous Multilayer Generalized Operational Perceptron

paper_content:

The traditional Multilayer Perceptron (MLP) using McCulloch-Pitts neuron model is inherently limited to a set of neuronal activities, i.e., linear weighted sum followed by nonlinear thresholding step. Previously, Generalized Operational Perceptron (GOP) was proposed to extend conventional perceptron model by defining a diverse set of neuronal activities to imitate a generalized model of biological neurons. Together with GOP, Progressive Operational Perceptron (POP) algorithm was proposed to optimize a pre-defined template of multiple homogeneous layers in a layerwise manner. In this paper, we propose an efficient algorithm to learn a compact, fully heterogeneous multilayer network that allows each individual neuron, regardless of the layer, to have distinct characteristics. Based on the complexity of the problem, the proposed algorithm operates in a progressive manner on a neuronal level, searching for a compact topology, not only in terms of depth but also width, i.e., the number of neurons in each layer. The proposed algorithm is shown to outperform other related learning methods in extensive experiments on several classification problems.
---
paper_title: Receptive fields and functional architecture of monkey striate cortex

paper_content:

1. The striate cortex was studied in lightly anaesthetized macaque and spider monkeys by recording extracellularly from single units and stimulating the retinas with spots or patterns of light. Most cells can be categorized as simple, complex, or hypercomplex, with response properties very similar to those previously described in the cat. On the average, however, receptive fields are smaller, and there is a greater sensitivity to changes in stimulus orientation. A small proportion of the cells are colour coded.2. Evidence is presented for at least two independent systems of columns extending vertically from surface to white matter. Columns of the first type contain cells with common receptive-field orientations. They are similar to the orientation columns described in the cat, but are probably smaller in cross-sectional area. In the second system cells are aggregated into columns according to eye preference. The ocular dominance columns are larger than the orientation columns, and the two sets of boundaries seem to be independent.3. There is a tendency for cells to be grouped according to symmetry of responses to movement; in some regions the cells respond equally well to the two opposite directions of movement of a line, but other regions contain a mixture of cells favouring one direction and cells favouring the other.4. A horizontal organization corresponding to the cortical layering can also be discerned. The upper layers (II and the upper two-thirds of III) contain complex and hypercomplex cells, but simple cells are virtually absent. The cells are mostly binocularly driven. Simple cells are found deep in layer III, and in IV A and IV B. In layer IV B they form a large proportion of the population, whereas complex cells are rare. In layers IV A and IV B one finds units lacking orientation specificity; it is not clear whether these are cell bodies or axons of geniculate cells. In layer IV most cells are driven by one eye only; this layer consists of a mosaic with cells of some regions responding to one eye only, those of other regions responding to the other eye. Layers V and VI contain mostly complex and hypercomplex cells, binocularly driven.5. The cortex is seen as a system organized vertically and horizontally in entirely different ways. In the vertical system (in which cells lying along a vertical line in the cortex have common features) stimulus dimensions such as retinal position, line orientation, ocular dominance, and perhaps directionality of movement, are mapped in sets of superimposed but independent mosaics. The horizontal system segregates cells in layers by hierarchical orders, the lowest orders (simple cells monocularly driven) located in and near layer IV, the higher orders in the upper and lower layers.
---
paper_title: Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position

paper_content:

Abstract Suggested by the structure of the visual nervous system, a new algorithm is proposed for pattern recognition. This algorithm can be realized with a multilayered network consisting of neuron-like cells. The network, “neocognitron”, is self-organized by unsupervised learning, and acquires the ability to recognize stimulus patterns according to the differences in their shapes: Any patterns which we human beings judge to be alike are also judged to be of the same category by the neocognitron. The neocognitron recognizes stimulus patterns correctly without being affected by shifts in position or even by considerable distortions in shape of the stimulus patterns.
---
paper_title: Kernel Multilayer Perceptron

paper_content:

We enhance the Multi layer Perceptron to map a feature vector not only from the original d-dimensional feature space, but from an intermediate implicit Hilbert feature space in which kernels calculate inner products. The kernel substitutes the usual inner product between weight vectors and the input vector (or the feature vector of the hidden layer). The objective is to boost the generalization capability of this universal function approximator even more. Classification experiments with standard Machine Learning data sets are shown. We are able to improve the classification accuracy performance criterion for certain kernel types and their intrinsic parameters for the majority of the data sets.
---
paper_title: A logical calculus of the ideas immanent in nervous activity

paper_content:

Abstract Because of the “all-or-none” character of nervous activity, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiological assumptions are equivalent, in the sense that for every net behaving under one assumption, there exists another net which behaves under the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed.
---
paper_title: Optimization of Neural Networks Using Variable Structure Systems

paper_content:

This paper proposes a new mixed training algorithm consisting of error backpropagation (EBP) and variable structure systems (VSSs) to optimize parameter updating of neural networks. For the optimization of the number of neurons in the hidden layer, a new term based on the output of the hidden layer is added to the cost function as a penalty term to make optimal use of hidden units related to weights corresponding to each unit in the hidden layer. VSS is used to control the dynamic model of the training process, whereas EBP attempts to minimize the cost function. In addition to the analysis of the imposed dynamics of the EBP technique, the global stability of the mixed training methodology and constraints on the design parameters are considered. The advantages of the proposed technique are guaranteed convergence, improved robustness, and lower sensitivity to initial weights of the neural network.
---
paper_title: Hybrid multilayered perceptron networks

paper_content:

This paper introduces a modified multilayered perception network (MLP) called the Hybrid Multilayered Perceptron (HMLP) network to improve the performance of a MLP network. The convergence rate of the proposed network is further improved by proposing a modified version of the recursive prediction error algorithm as the training algorithm. The capability of the proposed network architecture trained using the modified recursive prediction error algorithm was demonstrated using simulated and real data sets. The results indicated that the proposed network provides a significant improvement over a standard MLP network. These additional linear input connections do not significantly increase the complexity of the MLP network since the connections are linear. In fact, by using the linear input connections, the number of hidden nodes required by the standard MLP network model can be reduced, which will also reduce computational load. The performance of the HMLP network was also compared with Radial Basis Function ...
---
paper_title: Multidimensional Particle Swarm Optimization for Machine Learning and Pattern Recognition

paper_content:

For many engineering problems we require optimization processes with dynamic adaptation as we aim to establish the dimension of the search space where the optimum solution resides and develop robust techniques to avoid the local optima usually associated with multimodal problems. This book explores multidimensional particle swarm optimization, a technique developed by the authors that addresses these requirements in a well-defined algorithmic approach. After an introduction to the key optimization techniques, the authors introduce their unified framework and demonstrate its advantages in challenging application domains, focusing on the state of the art of multidimensional extensions such as global convergence in particle swarm optimization, dynamic data clustering, evolutionary neural networks, biomedical applications and personalized ECG classification, content-based image classification and retrieval, and evolutionary feature synthesis. The content is characterized by strong practical considerations, and the book is supported with fully documented source code for all applications presented, as well as many sample datasets. The book will be of benefit to researchers and practitioners working in the areas of machine intelligence, signal processing, pattern recognition, and data mining, or using principles from these areas in their application domains. It may also be used as a reference text for graduate courses on swarm optimization, data clustering and classification, content-based multimedia search, and biomedical signal processing applications.
---
paper_title: Gradient-Based Learning Applied to Document Recognition

paper_content:

Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
---
paper_title: Arrhythmia Detection Using Convolutional Neural Models

paper_content:

Mostly all works dealing with ECG signal and Convolutional Network approach use 1D CNNs and must train them from scratch, usually applying a signal preprocessing, such as noise reduction, R-peak detection or heartbeat detection. Instead, our approach was focused on demonstrating that effective transfer learning from 2D CNNs can be done using a well-known CNN called AlexNet, that was trained using real images from ImageNet Large Scale Visual Recognition Challenge (ILSVRC) 2012. From any temporal signal, it is possible to generate spectral images (spectrograms) than can be analysed by 2D CNN to do the task of extracting automatic features for the classification stage. In this work, the power spectrogram is generated from a randomly ECG segment, so no conditions of signal extraction are applied. After processing the spectrogram with the CNN, its outputs are used as relevant features to be discriminated by a Multi Layer Perceptron (MLP) which classifies them into arrhythmic or normal rhythm segments. The results obtained are in the 90% accuracy range, as good as the state of the art published with 1D CNNs, confirming that transfer learning is a good strategy to develop decision models in signal and image medical tasks.
---
paper_title: Personalized Monitoring and Advance Warning System for Cardiac Arrhythmias

paper_content:

Each year more than 7 million people die from cardiac arrhythmias. Yet no robust solution exists today to detect such heart anomalies right at the moment they occur. The purpose of this study was to design a personalized health monitoring system that can detect early occurrences of arrhythmias from an individual’s electrocardiogram (ECG) signal. We first modelled the common causes of arrhythmias in the signal domain as a degradation of normal ECG beats to abnormal beats. Using the degradation models, we performed abnormal beat synthesis which created potential abnormal beats from the average normal beat of the individual. Finally, a Convolutional Neural Network (CNN) was trained using real normal and synthesized abnormal beats. As a personalized classifier, the trained CNN can monitor ECG beats in real time for arrhythmia detection. Over 34 patients’ ECG records with a total of 63,341 ECG beats from the MIT-BIH arrhythmia benchmark database, we have shown that the probability of detecting one or more abnormal ECG beats among the first three occurrences is higher than 99.4% with a very low false-alarm rate.
---
paper_title: Automated detection of arrhythmias using different intervals of tachycardia ECG segments with convolutional neural network

paper_content:

Classification of normal and tachycardia arrhythmias ECG segments.Two and five seconds ECG segments are considered.Convolutional neural network is employed.QRS detection is not performed.Accuracy of 92.50% and 94.9% obtained for two and five seconds respectively. Our cardiovascular system weakens and is more prone to arrhythmia as we age. An arrhythmia is an abnormal heartbeat rhythm which can be life-threatening. Atrial fibrillation (Afib), atrial flutter (Afl), and ventricular fibrillation (Vfib) are the recurring life-threatening arrhythmias that affect the elderly population. An electrocardiogram (ECG) is the principal diagnostic tool employed to record and interpret ECG signals. These signals contain information about the different types of arrhythmias. However, due to the complexity and non-linearity of ECG signals, it is difficult to manually analyze these signals. Moreover, the interpretation of ECG signals is subjective and might vary between the experts. Hence, a computer-aided diagnosis (CAD) system is proposed. The CAD system will ensure that the assessment of ECG signals is objective and accurate. In this work, we present a convolutional neural network (CNN) technique to automatically detect the different ECG segments. Our algorithm consists of an eleven-layer deep CNN with the output layer of four neurons, each representing the normal (Nsr), Afib, Afl, and Vfib ECG class. In this work, we have used ECG signals of two seconds and five seconds durations without QRS detection. We achieved an accuracy, sensitivity, and specificity of 92.50%, 98.09%, and 93.13% respectively for two seconds of ECG segments. We obtained an accuracy of 94.90%, the sensitivity of 99.13%, and specificity of 81.44% for five seconds of ECG duration. This proposed algorithm can serve as an adjunct tool to assist clinicians in confirming their diagnosis. Display Omitted
---
paper_title: An Ensemble Deep Convolutional Neural Network Model with Improved D-S Evidence Fusion for Bearing Fault Diagnosis

paper_content:

Intelligent machine health monitoring and fault diagnosis are becoming increasingly important for modern manufacturing industries. Current fault diagnosis approaches mostly depend on expert-designed features for building prediction models. In this paper, we proposed IDSCNN, a novel bearing fault diagnosis algorithm based on ensemble deep convolutional neural networks and an improved Dempster–Shafer theory based evidence fusion. The convolutional neural networks take the root mean square (RMS) maps from the FFT (Fast Fourier Transformation) features of the vibration signals from two sensors as inputs. The improved D-S evidence theory is implemented via distance matrix from evidences and modified Gini Index. Extensive evaluations of the IDSCNN on the Case Western Reserve Dataset showed that our IDSCNN algorithm can achieve better fault diagnosis performance than existing machine learning methods by fusing complementary or conflicting evidences from different models and sensors and adapting to different load conditions.
---
paper_title: Hierarchical adaptive deep convolution neural network and its application to bearing fault diagnosis

paper_content:

Abstract Traditional artificial methods and intelligence-based methods of classifying and diagnosing various mechanical faults with high accuracy by extracting effective features from vibration data, such as support vector machines and back propagation neural networks, have been widely investigated. However, the problems of extracting features automatically without significantly increasing the demand for machinery expertise and maximizing accuracy without overcomplicating machine structure have to date remained unsolved. Therefore, a novel hierarchical learning rate adaptive deep convolution neural network based on an improved algorithm was proposed in this study, and its use to diagnose bearing faults and determine their severity was investigated. To test the effectiveness of the proposed method, an experiment was conducted with bearing-fault data samples obtained from a test rig. The method achieved a satisfactory performance in terms of both fault-pattern recognition and fault-size evaluation. In addition, comparison revealed that the improved algorithm is well suited to the fault-diagnosis model, and that the proposed method is superior to other existing methods.
---
paper_title: A deep convolutional neural network model to classify heartbeats

paper_content:

The electrocardiogram (ECG) is a standard test used to monitor the activity of the heart. Many cardiac abnormalities will be manifested in the ECG including arrhythmia which is a general term that refers to an abnormal heart rhythm. The basis of arrhythmia diagnosis is the identification of normal versus abnormal individual heart beats, and their correct classification into different diagnoses, based on ECG morphology. Heartbeats can be sub-divided into five categories namely non-ectopic, supraventricular ectopic, ventricular ectopic, fusion, and unknown beats. It is challenging and time-consuming to distinguish these heartbeats on ECG as these signals are typically corrupted by noise. We developed a 9-layer deep convolutional neural network (CNN) to automatically identify 5 different categories of heartbeats in ECG signals. Our experiment was conducted in original and noise attenuated sets of ECG signals derived from a publicly available database. This set was artificially augmented to even out the number of instances the 5 classes of heartbeats and filtered to remove high-frequency noise. The CNN was trained using the augmented data and achieved an accuracy of 94.03% and 93.47% in the diagnostic classification of heartbeats in original and noise free ECGs, respectively. When the CNN was trained with highly imbalanced data (original dataset), the accuracy of the CNN reduced to 89.07%% and 89.3% in noisy and noise-free ECGs. When properly trained, the proposed CNN model can serve as a tool for screening of ECG to quickly identify different types and frequency of arrhythmic heartbeats. Display Omitted Convolutional neural network (CNN) is used to classify 5 ECG classes.9-layer deep CNN is implemented.Generated synthetic data to overcome imbalance problem.Accuracy of 94.03% and 93.47% with and without noise removal respectively.
---
paper_title: Robust ECG signal classification for detection of atrial fibrillation using a novel neural network

paper_content:

Electrocardiograms (ECG) provide a non-invasive approach for clinical diagnosis in patients with cardiac problems, particularly atrial fibrillation (AF). Robust, automatic AF detection in clinics remains challenging. Deep learning has emerged as an effective tool for handling complex data analysis with minimal pre- and post-processing. A 16-layer 1D Convolutional Neural Network (CNN) was designed to classify the ECGs including AF. One of the key advances of the proposed CNN was that skip connections were employed to enhance the rate of information transfer throughout the network by connecting layers earlier in the network with layers later in the network. Skip connections led to a significant increase in the feature learning capabilities of the CNN as well as speeding up the training time. For comparisons, we also have implemented recurrent neural networks (RNN) and spectrogram learning. The CNN was trained on 8,528 ECGs and tested on 3,685 ECGs ranging from 9 to 60 seconds in length. The proposed 16-layer CNN outperformed RNNs and spectrogram learning. The training of the CNN took 2 hours on a Titan XPascal GPU (NVidia) with 3840 cores. The testing accuracy for the CNN was 82% and the runtime was ∼0.01 seconds for each signal classification. Particularly, the proposed CNN identified normal rhythm, AF and other rhythms with an accuracy of 90%, 82% and 75% respectively. We have demonstrated a novel CNN with skip connections to perform efficient, automatic ECG signal classification that could potentially aid robust patient diagnosis in real time.
---
paper_title: Convolutional recurrent neural networks for electrocardiogram classification

paper_content:

We propose two deep neural network architectures for classification of arbitrary-length electrocardiogram (ECG) recordings and evaluate them on the atrial fibrillation (AF) classification data set provided by the PhysioNet/CinC Challenge 2017. The first architecture is a deep convolutional neural network (CNN) with averaging-based feature aggregation across time. The second architecture combines convolutional layers for feature extraction with long-short term memory (LSTM) layers for temporal aggregation of features. As a key ingredient of our training procedure we introduce a simple data augmentation scheme for ECG data and demonstrate its effectiveness in the AF classification task at hand. The second architecture was found to outperform the first one, obtaining an $F_1$ score of $82.1$% on the hidden challenge testing set.
---
paper_title: Convolutional Neural Networks for patient-specific ECG classification

paper_content:

We propose a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system using an adaptive implementation of 1D Convolutional Neural Networks (CNNs) that can fuse feature extraction and classification into a unified learner. In this way, a dedicated CNN will be trained for each patient by using relatively small common and patient-specific training data and thus it can also be used to classify long ECG records such as Holter registers in a fast and accurate manner. Alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. The experimental results demonstrate that the proposed system achieves a superior classification performance for the detection of ventricular ectopic beats (VEB) and supraventricular ectopic beats (SVEB).
---
paper_title: Intelligent fault diagnosis of rolling bearing using hierarchical convolutional network based health state classification

paper_content:

A novel deep architecture based bearing diagnosis method is proposed.The method helps salient fault characteristic mining and intelligent diagnosis.The method is validated under various degrees of ambient noise. Rolling bearing tips are often the most susceptible to electro-mechanical system failure due to high-speed and complex working conditions, and recent studies on diagnosing bearing health using vibration data have developed an assortment of feature extraction and fault classification methods. Due to the strong non-linear and non-stationary characteristics, an effective and reliable deep learning method based on a convolutional neural network (CNN) is investigated in this paper making use of cognitive computing theory, which introduces the advantages of image recognition and visual perception to bearing fault diagnosis by simulating the cognition process of the cerebral cortex. The novel feature representation method for bearing data is first discussed using supervised deep learning with the goal of identifying more robust and salient feature representations to reduce information loss. Next, the deep hierarchical structure is trained in a robust manner that is established using a transmitting rule of greedy training layer by layer. Convolution computation, rectified linear units, and sub-sampling are applied for weight replication and reducing the number of parameters that need to be learned to improve the general feed-forward back propagation training. The CNN model could thus reduce learning computation requirements in the temporal dimension, and an invariance level of working condition fluctuation and ambient noise is provided by identifying the elementary features of bearings. A top classifier followed by a back propagation process is used for fault classification. Contrast experiments and analyses have been undertaken to delineate the effectiveness of the CNN model for fault classification of rolling bearings.
---
paper_title: Fault Detection and Severity Identification of Ball Bearings by Online Condition Monitoring

paper_content:

This paper presents a fast, accurate, and simple systematic approach for online condition monitoring and severity identification of ball bearings. This approach utilizes compact one-dimensional (1-D) convolutional neural networks (CNNs) to identify, quantify, and localize bearing damage. The proposed approach is verified experimentally under several single and multiple damage scenarios. The experimental results demonstrated that the proposed approach can achieve a high level of accuracy for damage detection, localization, and quantification. Besides its real-time processing ability and superior robustness against the high-level noise presence, the compact and minimally trained 1-D CNNs in the core of the proposed approach can handle new damage scenarios with utmost accuracy.
---
paper_title: Real-Time Patient-Specific ECG Classification by 1-D Convolutional Neural Networks

paper_content:

Goal: This paper presents a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system. Methods: An adaptive implementation of 1-D convolutional neural networks (CNNs) is inherently used to fuse the two major blocks of the ECG classification into a single learning body: feature extraction and classification. Therefore, for each patient, an individual and simple CNN will be trained by using relatively small common and patient-specific training data, and thus, such patient-specific feature extraction ability can further improve the classification performance. Since this also negates the necessity to extract hand-crafted manual features, once a dedicated CNN is trained for a particular patient, it can solely be used to classify possibly long ECG data stream in a fast and accurate manner or alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. Results: The results over the MIT-BIH arrhythmia benchmark database demonstrate that the proposed solution achieves a superior classification performance than most of the state-of-the-art methods for the detection of ventricular ectopic beats and supraventricular ectopic beats. Conclusion: Besides the speed and computational efficiency achieved, once a dedicated CNN is trained for an individual patient, it can solely be used to classify his/her long ECG records such as Holter registers in a fast and accurate manner. Significance: Due to its simple and parameter invariant nature, the proposed system is highly generic, and, thus, applicable to any ECG dataset.
---
paper_title: Wireless and real-time structural damage detection: A novel decentralized method for wireless sensor networks

paper_content:

Abstract Being an alternative to conventional wired sensors, wireless sensor networks (WSNs) are extensively used in Structural Health Monitoring (SHM) applications. Most of the Structural Damage Detection (SDD) approaches available in the SHM literature are centralized as they require transferring data from all sensors within the network to a single processing unit to evaluate the structural condition. These methods are found predominantly feasible for wired SHM systems; however, transmission and synchronization of huge data sets in WSNs has been found to be arduous. As such, the application of centralized methods with WSNs has been a challenge for engineers. In this paper, the authors are presenting a novel application of 1D Convolutional Neural Networks (1D CNNs) on WSNs for SDD purposes. The SDD is successfully performed completely wireless and real-time under ambient conditions. As a result of this, a decentralized damage detection method suitable for wireless SHM systems is proposed. The proposed method is based on 1D CNNs and it involves training an individual 1D CNN for each wireless sensor in the network in a format where each CNN is assigned to process the locally-available data only, eliminating the need for data transmission and synchronization. The proposed damage detection method operates directly on the raw ambient vibration condition signals without any filtering or preprocessing. Moreover, the proposed approach requires minimal computational time and power since 1D CNNs merge both feature extraction and classification tasks into a single learning block. This ability is prevailingly cost-effective and evidently practical in WSNs considering the hardware systems have been occasionally reported to suffer from limited power supply in these networks. To display the capability and verify the success of the proposed method, large-scale experiments conducted on a laboratory structure equipped with a state-of-the-art WSN are reported.
---
paper_title: Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition

paper_content:

Convolutional Neural Networks (CNN) have showed success in achieving translation invariance for many image processing tasks. The success is largely attributed to the use of local filtering and max-pooling in the CNN architecture. In this paper, we propose to apply CNN to speech recognition within the framework of hybrid NN-HMM model. We propose to use local filtering and max-pooling in frequency domain to normalize speaker variance to achieve higher multi-speaker speech recognition performance. In our method, a pair of local filtering layer and max-pooling layer is added at the lowest end of neural network (NN) to normalize spectral variations of speech signals. In our experiments, the proposed CNN architecture is evaluated in a speaker independent speech recognition task using the standard TIMIT data sets. Experimental results show that the proposed CNN method can achieve over 10% relative error reduction in the core TIMIT test sets when comparing with a regular NN using the same number of hidden layers and weights. Our results also show that the best result of the proposed CNN model is better than previously published results on the same TIMIT test sets that use a pre-trained deep NN model.
---
paper_title: Visualizing and Understanding Convolutional Networks

paper_content:

Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark Krizhevsky et al. [18]. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we explore both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. Used in a diagnostic role, these visualizations allow us to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. We also perform an ablation study to discover the performance contribution from different model layers. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.
---
paper_title: Dropout: a simple way to prevent neural networks from overfitting

paper_content:

Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
---
paper_title: Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks

paper_content:

Abstract Structural health monitoring (SHM) and vibration-based structural damage detection have been a continuous interest for civil, mechanical and aerospace engineers over the decades. Early and meticulous damage detection has always been one of the principal objectives of SHM applications. The performance of a classical damage detection system predominantly depends on the choice of the features and the classifier. While the fixed and hand-crafted features may either be a sub-optimal choice for a particular structure or fail to achieve the same level of performance on another structure, they usually require a large computation power which may hinder their usage for real-time structural damage detection. This paper presents a novel, fast and accurate structural damage detection system using 1D Convolutional Neural Networks (CNNs) that has an inherent adaptive design to fuse both feature extraction and classification blocks into a single and compact learning body. The proposed method performs vibration-based damage detection and localization of the damage in real-time. The advantage of this approach is its ability to extract optimal damage-sensitive features automatically from the raw acceleration signals. Large-scale experiments conducted on a grandstand simulator revealed an outstanding performance and verified the computational efficiency of the proposed real-time damage detection method.
---
paper_title: Reducing the Dimensionality of Data with Neural Networks

paper_content:

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.
---
paper_title: Convolutional Neural Network Based Fault Detection for Rotating Machinery

paper_content:

Abstract Vibration analysis is a well-established technique for condition monitoring of rotating machines as the vibration patterns differ depending on the fault or machine condition. Currently, mainly manually-engineered features, such as the ball pass frequencies of the raceway, RMS, kurtosis an crest, are used for automatic fault detection. Unfortunately, engineering and interpreting such features requires a significant level of human expertise. To enable non-experts in vibration analysis to perform condition monitoring, the overhead of feature engineering for specific faults needs to be reduced as much as possible. Therefore, in this article we propose a feature learning model for condition monitoring based on convolutional neural networks. The goal of this approach is to autonomously learn useful features for bearing fault detection from the data itself. Several types of bearing faults such as outer-raceway faults and lubrication degradation are considered, but also healthy bearings and rotor imbalance are included. For each condition, several bearings are tested to ensure generalization of the fault-detection system. Furthermore, the feature-learning based approach is compared to a feature-engineering based approach using the same data to objectively quantify their performance. The results indicate that the feature-learning system, based on convolutional neural networks, significantly outperforms the classical feature-engineering based approach which uses manually engineered features and a random forest classifier. The former achieves an accuracy of 93.61 percent and the latter an accuracy of 87.25 percent.
---
paper_title: Energy-Fluctuated Multiscale Feature Learning With Deep ConvNet for Intelligent Spindle Bearing Fault Diagnosis

paper_content:

Considering various health conditions under varying operational conditions, the mining sensitive feature from the measured signals is still a great challenge for intelligent fault diagnosis of spindle bearings. This paper proposed a novel energy-fluctuated multiscale feature mining approach based on wavelet packet energy (WPE) image and deep convolutional network (ConvNet) for spindle bearing fault diagnosis. Different from the vector characteristics applied in intelligent diagnosis of spindle bearings, wavelet packet transform is first combined with phase space reconstruction to rebuild a 2-D WPE image of the frequency subspaces. This special image can reconstruct the local relationship of the WP nodes and hold the energy fluctuation of the measured signal. Then, the identifiable characteristics can be further learned by a special architecture of the deep ConvNet. Other than the traditional neural network architecture, to maintain the global and local information simultaneously, deep ConvNet combines the skipping layer with the last convolutional layer as the input of the multiscale layer. The comparisons of clustering distribution and classification accuracy with six other features show that the proposed feature mining approach is quite suitable for spindle bearing fault diagnosis with multiclass classification regardless of the load fluctuation.
---
paper_title: Bearing Fault Detection by One-Dimensional Convolutional Neural Networks

paper_content:

Bearing faults are the biggest single source of motor failures. Artificial Neural Networks (ANNs) and other decision support systems are widely used for early detection of bearing faults. The typical decision support systems require feature extraction and classification as two distinct phases. Extracting fixed features each time may require a significant computational cost preventing their use in real-time applications. Furthermore, the selected features for the classification phase may not represent the most optimal choice. In this paper, the use of 1D Convolutional Neural Networks (CNNs) is proposed for a fast and accurate bearing fault detection system. The feature extraction and classification phases of the bearing fault detection are combined into a single learning body with the implementation of 1D CNN. The raw vibration data (signal) is fed into the proposed system as input eliminating the need for running a separate feature extraction algorithm each time vibration data is analyzed for classification. Implementation of 1D CNNs results in more efficient systems in terms of computational complexity. The classification performance of the proposed system with real bearing data demonstrates that the reduced computational complexity is achieved without a compromise in fault detection accuracy.
---
paper_title: ImageNet: A large-scale hierarchical image database

paper_content:

The explosion of image data on the Internet has the potential to foster more sophisticated and robust models and algorithms to index, retrieve, organize and interact with images and multimedia data. But exactly how such data can be harnessed and organized remains a critical problem. We introduce here a new database called “ImageNet”, a large-scale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. This will result in tens of millions of annotated images organized by the semantic hierarchy of WordNet. This paper offers a detailed analysis of ImageNet in its current state: 12 subtrees with 5247 synsets and 3.2 million images in total. We show that ImageNet is much larger in scale and diversity and much more accurate than the current image datasets. Constructing such a large-scale database is a challenging task. We describe the data collection scheme with Amazon Mechanical Turk. Lastly, we illustrate the usefulness of ImageNet through three simple applications in object recognition, image classification and automatic object clustering. We hope that the scale, accuracy, diversity and hierarchical structure of ImageNet can offer unparalleled opportunities to researchers in the computer vision community and beyond.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: A Generic Intelligent Bearing Fault Diagnosis System Using Compact Adaptive 1D CNN Classifier

paper_content:

Timely and accurate bearing fault detection and diagnosis is important for reliable and safe operation of industrial systems. In this study, performance of a generic real-time induction bearing fault diagnosis system employing compact adaptive 1D Convolutional Neural Network (CNN) classifier is extensively studied. In the literature, although many studies have developed highly accurate algorithms for detecting bearing faults, their results have generally been limited to relatively small train/test data sets. As opposed to conventional intelligent fault diagnosis systems that usually encapsulate feature extraction, feature selection and classification as distinct blocks, the proposed system takes directly raw time-series sensor data as input and it can efficiently learn optimal features with the proper training. The main advantages of the 1D CNN based approach are 1) its compact architecture configuration (rather than the complex deep architectures) which performs only 1D convolutions making it suitable for real-time fault detection and monitoring, 2) its cost effective and practical real-time hardware implementation, 3) its ability to work without any pre-determined transformation (such as FFT or DWT), hand-crafted feature extraction and feature selection, and 4) its capability to provide efficient training of the classifier with limited size of training data set and limited number of BP iterations. Effectiveness and feasibility of the 1D CNN based fault diagnosis method is validated by applying it to two commonly used benchmark real vibration data sets and comparing the results with the other competing intelligent fault diagnosis methods.
---
paper_title: A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load

paper_content:

Abstract In recent years, intelligent fault diagnosis algorithms using machine learning technique have achieved much success. However, due to the fact that in real world industrial applications, the working load is changing all the time and noise from the working environment is inevitable, degradation of the performance of intelligent fault diagnosis methods is very serious. In this paper, a new model based on deep learning is proposed to address the problem. Our contributions of include: First, we proposed an end-to-end method that takes raw temporal signals as inputs and thus doesn’t need any time consuming denoising preprocessing. The model can achieve pretty high accuracy under noisy environment. Second, the model does not rely on any domain adaptation algorithm or require information of the target domain. It can achieve high accuracy when working load is changed. To understand the proposed model, we will visualize the learned features, and try to analyze the reasons behind the high performance of the model.
---
paper_title: Bearings Fault Diagnosis Based on Convolutional Neural Networks with 2-D Representation of Vibration Signals as Input

paper_content:

Periodic vibration signals captured by the accelerometers carry rich information for bearing fault diagnosis. Existing methods mostly rely on hand-crafted time-consuming preprocessing of data to acquire suitable features. In this paper, we use an easy and effective method to transform the 1-D temporal vibration signal into a 2-D image. With the signal image, convolutional Neural Network (CNN) is used to train the raw vibration data. As powerful feature extractor and classifier for image recognition, CNN can learn to acquire features most suitable for the classification task by being trained. With the image format of vibration signals, the neuron in fully-connected layer of CNN can see farther and capture the periodic feature of signals. According to the results of the experiments, when fed in enough training samples, the proposed method outperforms other common methods. The proposed method can also be applied to solve intelligent diagnosis problems of other machine systems.
---
paper_title: Convolutional Neural Network Based Bearing Fault Diagnosis

paper_content:

In this paper, we propose a new bearing fault diagnosis method without the feature extraction, based on Convolutional Neural Network (CNN). The 1-D vibration signal is converted to 2-D data called vibration image. Then, the vibration images are fed into the CNN for bearing fault classification. Experiments are carried out with bearing data from the Case Western Reserve University Bearing Fault Database and its result are compared with the results of other methods to show the effectiveness of the proposed algorithm.
---
paper_title: 1-D CNNs for structural damage detection: Verification on a structural health monitoring benchmark data

paper_content:

Abstract Structural damage detection has been an interdisciplinary area of interest for various engineering fields. While the available damage detection methods have been in the process of adapting machine learning concepts, most machine learning based methods extract “hand-crafted” features which are fixed and manually selected in advance. Their performance varies significantly among various patterns of data depending on the particular structure under analysis. Convolutional neural networks (CNNs), on the other hand, can fuse and simultaneously optimize two major sets of an assessment task (feature extraction and classification) into a single learning block during the training phase. This ability not only provides an improved classification performance but also yields a superior computational efficiency. 1D CNNs have recently achieved state-of-the-art performance in vibration-based structural damage detection; however, it has been reported that the training of the CNNs requires significant amount of measurements especially in large structures. In order to overcome this limitation, this paper presents an enhanced CNN-based approach that requires only two measurement sets regardless of the size of the structure. This approach is verified using the experimental data of the Phase II benchmark problem of structural health monitoring which had been introduced by IASC-ASCE Structural Health Monitoring Task Group. As a result, it is shown that the enhanced CNN-based approach successfully estimated the actual amount of damage for the nine damage scenarios of the benchmark study.
---
paper_title: Speed Invariant Bearing Fault Characterization Using Convolutional Neural Networks

paper_content:

Unlike traditional machine learning techniques, convolutional neural networks (CNNs), one of deep learning methods, automate the feature extraction process required for an effective classification. In general, CNN based bearing fault diagnosis analyzes raw signals to classify the localized faults. However, bearings are subjected to non-stationary speeds due to various operating conditions, and thus, CNN cannot determine optimal features of the various conditions while analyzing raw signals, reducing classification accuracy. In this paper, we propose a pre-processing step to improve the performance of the CNN based fault diagnosis by extracting envelope spectrums (ES) on the raw signals. As ES demodulates the signals to provide the information inherent in defect frequency of faults and its variations to non-stationary speeds, CNN can learn to extract distinctive features to diagnose bearing defects effectively. The proposed method is evaluated on acoustic emission based low speed bearing data. The trained CNN model is tested on data with different revolutions per minute (RPM), and it achieves the classification accuracy greater than 94.8%.
---
paper_title: Real-Time Fault Detection and Identification for MMC Using 1-D Convolutional Neural Networks

paper_content:

Automated early detection and identification of switch faults are essential in high-voltage applications. Modular multilevel converter (MMC) is a new and promising topology for such applications. MMC is composed of many identical controlled voltage sources called modules or cells. Each cell may have one or more switches and a switch failure may occur in anyone of these cells. The steady-state normal and fault behavior of a cell voltage will also significantly vary according to the changes in the load current and the fault timing. This makes it a challenging problem to detect and identify such faults as soon as they occur. In this paper, we propose a real-time and highly accurate MMC circuit monitoring system for early fault detection and identification using adaptive one-dimensional convolutional neural networks. The proposed approach is directly applicable to the raw voltage and current data and thus eliminates the need for any feature extraction algorithm, resulting in a highly efficient and reliable system. Simulation results obtained using a four-cell, eight-switch MMC topology demonstrate that the proposed system has a high reliability to avoid any false alarm and achieves a detection probability of 0.989, and average identification probability of 0.997 in less than 100 ms.
---
paper_title: Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks

paper_content:

Early detection of the motor faults is essential and artificial neural networks are widely used for this purpose. The typical systems usually encapsulate two distinct blocks: feature extraction and classification. Such fixed and hand-crafted features may be a suboptimal choice and require a significant computational cost that will prevent their usage for real-time applications. In this paper, we propose a fast and accurate motor condition monitoring and early fault-detection system using 1-D convolutional neural networks that has an inherent adaptive design to fuse the feature extraction and classification phases of the motor fault detection into a single learning body. The proposed approach is directly applicable to the raw data (signal), and, thus, eliminates the need for a separate feature extraction algorithm resulting in more efficient systems in terms of both speed and hardware. Experimental results obtained using real motor data demonstrate the effectiveness of the proposed method for real-time motor condition monitoring.
---
paper_title: Efficiency validation of one dimensional convolutional neural networks for structural damage detection using a SHM benchmark data

paper_content:

In this paper, a novel one dimensional convolution neural network (1D-CNN) based structural damage assessment technique is validated with a benchmark study published by IASC-ASCE Structural Health Monitoring Task Group in 2003. In contrast with predominant machine learning based structural damage detection techniques of the literature, the technique shown in this paper runs without manual feature extraction or preprocessing stages. It runs directly on the raw vibration data. In CNNs, the stages of feature extraction and feature classification are merged into one stage; therefore, the proposed technique is efficient, feasible and economical. Utilizing the optimal features learned by 1D CNNs, the proposed CNN-based technique considerably improves the classification efficiency and accuracy. The performance improvement of the proposed technique is assessed by calculating the "Probability of Damage" values for damage estimations. The unseen structural damage cases between the two extreme end structural cases (zero damage and total damage) were successfully identified. Consequently, it is validated that the improved CNN-based technique is efficient since it predicted the level of damage consistently with the structural damage cases defined in the existing benchmark.
---
paper_title: Reducing the Dimensionality of Data with Neural Networks

paper_content:

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.
---
paper_title: Real-Time Patient-Specific ECG Classification by 1-D Convolutional Neural Networks

paper_content:

Goal: This paper presents a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system. Methods: An adaptive implementation of 1-D convolutional neural networks (CNNs) is inherently used to fuse the two major blocks of the ECG classification into a single learning body: feature extraction and classification. Therefore, for each patient, an individual and simple CNN will be trained by using relatively small common and patient-specific training data, and thus, such patient-specific feature extraction ability can further improve the classification performance. Since this also negates the necessity to extract hand-crafted manual features, once a dedicated CNN is trained for a particular patient, it can solely be used to classify possibly long ECG data stream in a fast and accurate manner or alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. Results: The results over the MIT-BIH arrhythmia benchmark database demonstrate that the proposed solution achieves a superior classification performance than most of the state-of-the-art methods for the detection of ventricular ectopic beats and supraventricular ectopic beats. Conclusion: Besides the speed and computational efficiency achieved, once a dedicated CNN is trained for an individual patient, it can solely be used to classify his/her long ECG records such as Holter registers in a fast and accurate manner. Significance: Due to its simple and parameter invariant nature, the proposed system is highly generic, and, thus, applicable to any ECG dataset.
---
paper_title: Gradient-Based Learning Applied to Document Recognition

paper_content:

Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: Gradient-Based Learning Applied to Document Recognition

paper_content:

Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: Personalized Monitoring and Advance Warning System for Cardiac Arrhythmias

paper_content:

Each year more than 7 million people die from cardiac arrhythmias. Yet no robust solution exists today to detect such heart anomalies right at the moment they occur. The purpose of this study was to design a personalized health monitoring system that can detect early occurrences of arrhythmias from an individual’s electrocardiogram (ECG) signal. We first modelled the common causes of arrhythmias in the signal domain as a degradation of normal ECG beats to abnormal beats. Using the degradation models, we performed abnormal beat synthesis which created potential abnormal beats from the average normal beat of the individual. Finally, a Convolutional Neural Network (CNN) was trained using real normal and synthesized abnormal beats. As a personalized classifier, the trained CNN can monitor ECG beats in real time for arrhythmia detection. Over 34 patients’ ECG records with a total of 63,341 ECG beats from the MIT-BIH arrhythmia benchmark database, we have shown that the probability of detecting one or more abnormal ECG beats among the first three occurrences is higher than 99.4% with a very low false-alarm rate.
---
paper_title: Convolutional Neural Networks for patient-specific ECG classification

paper_content:

We propose a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system using an adaptive implementation of 1D Convolutional Neural Networks (CNNs) that can fuse feature extraction and classification into a unified learner. In this way, a dedicated CNN will be trained for each patient by using relatively small common and patient-specific training data and thus it can also be used to classify long ECG records such as Holter registers in a fast and accurate manner. Alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. The experimental results demonstrate that the proposed system achieves a superior classification performance for the detection of ventricular ectopic beats (VEB) and supraventricular ectopic beats (SVEB).
---
paper_title: Real-Time Patient-Specific ECG Classification by 1-D Convolutional Neural Networks

paper_content:

Goal: This paper presents a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system. Methods: An adaptive implementation of 1-D convolutional neural networks (CNNs) is inherently used to fuse the two major blocks of the ECG classification into a single learning body: feature extraction and classification. Therefore, for each patient, an individual and simple CNN will be trained by using relatively small common and patient-specific training data, and thus, such patient-specific feature extraction ability can further improve the classification performance. Since this also negates the necessity to extract hand-crafted manual features, once a dedicated CNN is trained for a particular patient, it can solely be used to classify possibly long ECG data stream in a fast and accurate manner or alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. Results: The results over the MIT-BIH arrhythmia benchmark database demonstrate that the proposed solution achieves a superior classification performance than most of the state-of-the-art methods for the detection of ventricular ectopic beats and supraventricular ectopic beats. Conclusion: Besides the speed and computational efficiency achieved, once a dedicated CNN is trained for an individual patient, it can solely be used to classify his/her long ECG records such as Holter registers in a fast and accurate manner. Significance: Due to its simple and parameter invariant nature, the proposed system is highly generic, and, thus, applicable to any ECG dataset.
---
paper_title: Wireless and real-time structural damage detection: A novel decentralized method for wireless sensor networks

paper_content:

Abstract Being an alternative to conventional wired sensors, wireless sensor networks (WSNs) are extensively used in Structural Health Monitoring (SHM) applications. Most of the Structural Damage Detection (SDD) approaches available in the SHM literature are centralized as they require transferring data from all sensors within the network to a single processing unit to evaluate the structural condition. These methods are found predominantly feasible for wired SHM systems; however, transmission and synchronization of huge data sets in WSNs has been found to be arduous. As such, the application of centralized methods with WSNs has been a challenge for engineers. In this paper, the authors are presenting a novel application of 1D Convolutional Neural Networks (1D CNNs) on WSNs for SDD purposes. The SDD is successfully performed completely wireless and real-time under ambient conditions. As a result of this, a decentralized damage detection method suitable for wireless SHM systems is proposed. The proposed method is based on 1D CNNs and it involves training an individual 1D CNN for each wireless sensor in the network in a format where each CNN is assigned to process the locally-available data only, eliminating the need for data transmission and synchronization. The proposed damage detection method operates directly on the raw ambient vibration condition signals without any filtering or preprocessing. Moreover, the proposed approach requires minimal computational time and power since 1D CNNs merge both feature extraction and classification tasks into a single learning block. This ability is prevailingly cost-effective and evidently practical in WSNs considering the hardware systems have been occasionally reported to suffer from limited power supply in these networks. To display the capability and verify the success of the proposed method, large-scale experiments conducted on a laboratory structure equipped with a state-of-the-art WSN are reported.
---
paper_title: Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks

paper_content:

Abstract Structural health monitoring (SHM) and vibration-based structural damage detection have been a continuous interest for civil, mechanical and aerospace engineers over the decades. Early and meticulous damage detection has always been one of the principal objectives of SHM applications. The performance of a classical damage detection system predominantly depends on the choice of the features and the classifier. While the fixed and hand-crafted features may either be a sub-optimal choice for a particular structure or fail to achieve the same level of performance on another structure, they usually require a large computation power which may hinder their usage for real-time structural damage detection. This paper presents a novel, fast and accurate structural damage detection system using 1D Convolutional Neural Networks (CNNs) that has an inherent adaptive design to fuse both feature extraction and classification blocks into a single and compact learning body. The proposed method performs vibration-based damage detection and localization of the damage in real-time. The advantage of this approach is its ability to extract optimal damage-sensitive features automatically from the raw acceleration signals. Large-scale experiments conducted on a grandstand simulator revealed an outstanding performance and verified the computational efficiency of the proposed real-time damage detection method.
---
paper_title: 1-D CNNs for structural damage detection: Verification on a structural health monitoring benchmark data

paper_content:

Abstract Structural damage detection has been an interdisciplinary area of interest for various engineering fields. While the available damage detection methods have been in the process of adapting machine learning concepts, most machine learning based methods extract “hand-crafted” features which are fixed and manually selected in advance. Their performance varies significantly among various patterns of data depending on the particular structure under analysis. Convolutional neural networks (CNNs), on the other hand, can fuse and simultaneously optimize two major sets of an assessment task (feature extraction and classification) into a single learning block during the training phase. This ability not only provides an improved classification performance but also yields a superior computational efficiency. 1D CNNs have recently achieved state-of-the-art performance in vibration-based structural damage detection; however, it has been reported that the training of the CNNs requires significant amount of measurements especially in large structures. In order to overcome this limitation, this paper presents an enhanced CNN-based approach that requires only two measurement sets regardless of the size of the structure. This approach is verified using the experimental data of the Phase II benchmark problem of structural health monitoring which had been introduced by IASC-ASCE Structural Health Monitoring Task Group. As a result, it is shown that the enhanced CNN-based approach successfully estimated the actual amount of damage for the nine damage scenarios of the benchmark study.
---
paper_title: Real-Time Fault Detection and Identification for MMC Using 1-D Convolutional Neural Networks

paper_content:

Automated early detection and identification of switch faults are essential in high-voltage applications. Modular multilevel converter (MMC) is a new and promising topology for such applications. MMC is composed of many identical controlled voltage sources called modules or cells. Each cell may have one or more switches and a switch failure may occur in anyone of these cells. The steady-state normal and fault behavior of a cell voltage will also significantly vary according to the changes in the load current and the fault timing. This makes it a challenging problem to detect and identify such faults as soon as they occur. In this paper, we propose a real-time and highly accurate MMC circuit monitoring system for early fault detection and identification using adaptive one-dimensional convolutional neural networks. The proposed approach is directly applicable to the raw voltage and current data and thus eliminates the need for any feature extraction algorithm, resulting in a highly efficient and reliable system. Simulation results obtained using a four-cell, eight-switch MMC topology demonstrate that the proposed system has a high reliability to avoid any false alarm and achieves a detection probability of 0.989, and average identification probability of 0.997 in less than 100 ms.
---
paper_title: Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks

paper_content:

Early detection of the motor faults is essential and artificial neural networks are widely used for this purpose. The typical systems usually encapsulate two distinct blocks: feature extraction and classification. Such fixed and hand-crafted features may be a suboptimal choice and require a significant computational cost that will prevent their usage for real-time applications. In this paper, we propose a fast and accurate motor condition monitoring and early fault-detection system using 1-D convolutional neural networks that has an inherent adaptive design to fuse the feature extraction and classification phases of the motor fault detection into a single learning body. The proposed approach is directly applicable to the raw data (signal), and, thus, eliminates the need for a separate feature extraction algorithm resulting in more efficient systems in terms of both speed and hardware. Experimental results obtained using real motor data demonstrate the effectiveness of the proposed method for real-time motor condition monitoring.
---
paper_title: Efficiency validation of one dimensional convolutional neural networks for structural damage detection using a SHM benchmark data

paper_content:

In this paper, a novel one dimensional convolution neural network (1D-CNN) based structural damage assessment technique is validated with a benchmark study published by IASC-ASCE Structural Health Monitoring Task Group in 2003. In contrast with predominant machine learning based structural damage detection techniques of the literature, the technique shown in this paper runs without manual feature extraction or preprocessing stages. It runs directly on the raw vibration data. In CNNs, the stages of feature extraction and feature classification are merged into one stage; therefore, the proposed technique is efficient, feasible and economical. Utilizing the optimal features learned by 1D CNNs, the proposed CNN-based technique considerably improves the classification efficiency and accuracy. The performance improvement of the proposed technique is assessed by calculating the "Probability of Damage" values for damage estimations. The unseen structural damage cases between the two extreme end structural cases (zero damage and total damage) were successfully identified. Consequently, it is validated that the improved CNN-based technique is efficient since it predicted the level of damage consistently with the structural damage cases defined in the existing benchmark.
---
paper_title: Real-Time Patient-Specific ECG Classification by 1-D Convolutional Neural Networks

paper_content:

Goal: This paper presents a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system. Methods: An adaptive implementation of 1-D convolutional neural networks (CNNs) is inherently used to fuse the two major blocks of the ECG classification into a single learning body: feature extraction and classification. Therefore, for each patient, an individual and simple CNN will be trained by using relatively small common and patient-specific training data, and thus, such patient-specific feature extraction ability can further improve the classification performance. Since this also negates the necessity to extract hand-crafted manual features, once a dedicated CNN is trained for a particular patient, it can solely be used to classify possibly long ECG data stream in a fast and accurate manner or alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. Results: The results over the MIT-BIH arrhythmia benchmark database demonstrate that the proposed solution achieves a superior classification performance than most of the state-of-the-art methods for the detection of ventricular ectopic beats and supraventricular ectopic beats. Conclusion: Besides the speed and computational efficiency achieved, once a dedicated CNN is trained for an individual patient, it can solely be used to classify his/her long ECG records such as Holter registers in a fast and accurate manner. Significance: Due to its simple and parameter invariant nature, the proposed system is highly generic, and, thus, applicable to any ECG dataset.
---
paper_title: Personalized Monitoring and Advance Warning System for Cardiac Arrhythmias

paper_content:

Each year more than 7 million people die from cardiac arrhythmias. Yet no robust solution exists today to detect such heart anomalies right at the moment they occur. The purpose of this study was to design a personalized health monitoring system that can detect early occurrences of arrhythmias from an individual’s electrocardiogram (ECG) signal. We first modelled the common causes of arrhythmias in the signal domain as a degradation of normal ECG beats to abnormal beats. Using the degradation models, we performed abnormal beat synthesis which created potential abnormal beats from the average normal beat of the individual. Finally, a Convolutional Neural Network (CNN) was trained using real normal and synthesized abnormal beats. As a personalized classifier, the trained CNN can monitor ECG beats in real time for arrhythmia detection. Over 34 patients’ ECG records with a total of 63,341 ECG beats from the MIT-BIH arrhythmia benchmark database, we have shown that the probability of detecting one or more abnormal ECG beats among the first three occurrences is higher than 99.4% with a very low false-alarm rate.
---
paper_title: Automated detection of arrhythmias using different intervals of tachycardia ECG segments with convolutional neural network

paper_content:

Classification of normal and tachycardia arrhythmias ECG segments.Two and five seconds ECG segments are considered.Convolutional neural network is employed.QRS detection is not performed.Accuracy of 92.50% and 94.9% obtained for two and five seconds respectively. Our cardiovascular system weakens and is more prone to arrhythmia as we age. An arrhythmia is an abnormal heartbeat rhythm which can be life-threatening. Atrial fibrillation (Afib), atrial flutter (Afl), and ventricular fibrillation (Vfib) are the recurring life-threatening arrhythmias that affect the elderly population. An electrocardiogram (ECG) is the principal diagnostic tool employed to record and interpret ECG signals. These signals contain information about the different types of arrhythmias. However, due to the complexity and non-linearity of ECG signals, it is difficult to manually analyze these signals. Moreover, the interpretation of ECG signals is subjective and might vary between the experts. Hence, a computer-aided diagnosis (CAD) system is proposed. The CAD system will ensure that the assessment of ECG signals is objective and accurate. In this work, we present a convolutional neural network (CNN) technique to automatically detect the different ECG segments. Our algorithm consists of an eleven-layer deep CNN with the output layer of four neurons, each representing the normal (Nsr), Afib, Afl, and Vfib ECG class. In this work, we have used ECG signals of two seconds and five seconds durations without QRS detection. We achieved an accuracy, sensitivity, and specificity of 92.50%, 98.09%, and 93.13% respectively for two seconds of ECG segments. We obtained an accuracy of 94.90%, the sensitivity of 99.13%, and specificity of 81.44% for five seconds of ECG duration. This proposed algorithm can serve as an adjunct tool to assist clinicians in confirming their diagnosis. Display Omitted
---
paper_title: A deep convolutional neural network model to classify heartbeats

paper_content:

The electrocardiogram (ECG) is a standard test used to monitor the activity of the heart. Many cardiac abnormalities will be manifested in the ECG including arrhythmia which is a general term that refers to an abnormal heart rhythm. The basis of arrhythmia diagnosis is the identification of normal versus abnormal individual heart beats, and their correct classification into different diagnoses, based on ECG morphology. Heartbeats can be sub-divided into five categories namely non-ectopic, supraventricular ectopic, ventricular ectopic, fusion, and unknown beats. It is challenging and time-consuming to distinguish these heartbeats on ECG as these signals are typically corrupted by noise. We developed a 9-layer deep convolutional neural network (CNN) to automatically identify 5 different categories of heartbeats in ECG signals. Our experiment was conducted in original and noise attenuated sets of ECG signals derived from a publicly available database. This set was artificially augmented to even out the number of instances the 5 classes of heartbeats and filtered to remove high-frequency noise. The CNN was trained using the augmented data and achieved an accuracy of 94.03% and 93.47% in the diagnostic classification of heartbeats in original and noise free ECGs, respectively. When the CNN was trained with highly imbalanced data (original dataset), the accuracy of the CNN reduced to 89.07%% and 89.3% in noisy and noise-free ECGs. When properly trained, the proposed CNN model can serve as a tool for screening of ECG to quickly identify different types and frequency of arrhythmic heartbeats. Display Omitted Convolutional neural network (CNN) is used to classify 5 ECG classes.9-layer deep CNN is implemented.Generated synthetic data to overcome imbalance problem.Accuracy of 94.03% and 93.47% with and without noise removal respectively.
---
paper_title: Robust ECG signal classification for detection of atrial fibrillation using a novel neural network

paper_content:

Electrocardiograms (ECG) provide a non-invasive approach for clinical diagnosis in patients with cardiac problems, particularly atrial fibrillation (AF). Robust, automatic AF detection in clinics remains challenging. Deep learning has emerged as an effective tool for handling complex data analysis with minimal pre- and post-processing. A 16-layer 1D Convolutional Neural Network (CNN) was designed to classify the ECGs including AF. One of the key advances of the proposed CNN was that skip connections were employed to enhance the rate of information transfer throughout the network by connecting layers earlier in the network with layers later in the network. Skip connections led to a significant increase in the feature learning capabilities of the CNN as well as speeding up the training time. For comparisons, we also have implemented recurrent neural networks (RNN) and spectrogram learning. The CNN was trained on 8,528 ECGs and tested on 3,685 ECGs ranging from 9 to 60 seconds in length. The proposed 16-layer CNN outperformed RNNs and spectrogram learning. The training of the CNN took 2 hours on a Titan XPascal GPU (NVidia) with 3840 cores. The testing accuracy for the CNN was 82% and the runtime was ∼0.01 seconds for each signal classification. Particularly, the proposed CNN identified normal rhythm, AF and other rhythms with an accuracy of 90%, 82% and 75% respectively. We have demonstrated a novel CNN with skip connections to perform efficient, automatic ECG signal classification that could potentially aid robust patient diagnosis in real time.
---
paper_title: Convolutional recurrent neural networks for electrocardiogram classification

paper_content:

We propose two deep neural network architectures for classification of arbitrary-length electrocardiogram (ECG) recordings and evaluate them on the atrial fibrillation (AF) classification data set provided by the PhysioNet/CinC Challenge 2017. The first architecture is a deep convolutional neural network (CNN) with averaging-based feature aggregation across time. The second architecture combines convolutional layers for feature extraction with long-short term memory (LSTM) layers for temporal aggregation of features. As a key ingredient of our training procedure we introduce a simple data augmentation scheme for ECG data and demonstrate its effectiveness in the AF classification task at hand. The second architecture was found to outperform the first one, obtaining an $F_1$ score of $82.1$% on the hidden challenge testing set.
---
paper_title: Multidimensional Particle Swarm Optimization for Machine Learning and Pattern Recognition

paper_content:

For many engineering problems we require optimization processes with dynamic adaptation as we aim to establish the dimension of the search space where the optimum solution resides and develop robust techniques to avoid the local optima usually associated with multimodal problems. This book explores multidimensional particle swarm optimization, a technique developed by the authors that addresses these requirements in a well-defined algorithmic approach. After an introduction to the key optimization techniques, the authors introduce their unified framework and demonstrate its advantages in challenging application domains, focusing on the state of the art of multidimensional extensions such as global convergence in particle swarm optimization, dynamic data clustering, evolutionary neural networks, biomedical applications and personalized ECG classification, content-based image classification and retrieval, and evolutionary feature synthesis. The content is characterized by strong practical considerations, and the book is supported with fully documented source code for all applications presented, as well as many sample datasets. The book will be of benefit to researchers and practitioners working in the areas of machine intelligence, signal processing, pattern recognition, and data mining, or using principles from these areas in their application domains. It may also be used as a reference text for graduate courses on swarm optimization, data clustering and classification, content-based multimedia search, and biomedical signal processing applications.
---
paper_title: Personalized Monitoring and Advance Warning System for Cardiac Arrhythmias

paper_content:

Each year more than 7 million people die from cardiac arrhythmias. Yet no robust solution exists today to detect such heart anomalies right at the moment they occur. The purpose of this study was to design a personalized health monitoring system that can detect early occurrences of arrhythmias from an individual’s electrocardiogram (ECG) signal. We first modelled the common causes of arrhythmias in the signal domain as a degradation of normal ECG beats to abnormal beats. Using the degradation models, we performed abnormal beat synthesis which created potential abnormal beats from the average normal beat of the individual. Finally, a Convolutional Neural Network (CNN) was trained using real normal and synthesized abnormal beats. As a personalized classifier, the trained CNN can monitor ECG beats in real time for arrhythmia detection. Over 34 patients’ ECG records with a total of 63,341 ECG beats from the MIT-BIH arrhythmia benchmark database, we have shown that the probability of detecting one or more abnormal ECG beats among the first three occurrences is higher than 99.4% with a very low false-alarm rate.
---
paper_title: Convolutional Neural Networks for patient-specific ECG classification

paper_content:

We propose a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system using an adaptive implementation of 1D Convolutional Neural Networks (CNNs) that can fuse feature extraction and classification into a unified learner. In this way, a dedicated CNN will be trained for each patient by using relatively small common and patient-specific training data and thus it can also be used to classify long ECG records such as Holter registers in a fast and accurate manner. Alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. The experimental results demonstrate that the proposed system achieves a superior classification performance for the detection of ventricular ectopic beats (VEB) and supraventricular ectopic beats (SVEB).
---
paper_title: Real-Time Patient-Specific ECG Classification by 1-D Convolutional Neural Networks

paper_content:

Goal: This paper presents a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system. Methods: An adaptive implementation of 1-D convolutional neural networks (CNNs) is inherently used to fuse the two major blocks of the ECG classification into a single learning body: feature extraction and classification. Therefore, for each patient, an individual and simple CNN will be trained by using relatively small common and patient-specific training data, and thus, such patient-specific feature extraction ability can further improve the classification performance. Since this also negates the necessity to extract hand-crafted manual features, once a dedicated CNN is trained for a particular patient, it can solely be used to classify possibly long ECG data stream in a fast and accurate manner or alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. Results: The results over the MIT-BIH arrhythmia benchmark database demonstrate that the proposed solution achieves a superior classification performance than most of the state-of-the-art methods for the detection of ventricular ectopic beats and supraventricular ectopic beats. Conclusion: Besides the speed and computational efficiency achieved, once a dedicated CNN is trained for an individual patient, it can solely be used to classify his/her long ECG records such as Holter registers in a fast and accurate manner. Significance: Due to its simple and parameter invariant nature, the proposed system is highly generic, and, thus, applicable to any ECG dataset.
---
paper_title: Wireless and real-time structural damage detection: A novel decentralized method for wireless sensor networks

paper_content:

Abstract Being an alternative to conventional wired sensors, wireless sensor networks (WSNs) are extensively used in Structural Health Monitoring (SHM) applications. Most of the Structural Damage Detection (SDD) approaches available in the SHM literature are centralized as they require transferring data from all sensors within the network to a single processing unit to evaluate the structural condition. These methods are found predominantly feasible for wired SHM systems; however, transmission and synchronization of huge data sets in WSNs has been found to be arduous. As such, the application of centralized methods with WSNs has been a challenge for engineers. In this paper, the authors are presenting a novel application of 1D Convolutional Neural Networks (1D CNNs) on WSNs for SDD purposes. The SDD is successfully performed completely wireless and real-time under ambient conditions. As a result of this, a decentralized damage detection method suitable for wireless SHM systems is proposed. The proposed method is based on 1D CNNs and it involves training an individual 1D CNN for each wireless sensor in the network in a format where each CNN is assigned to process the locally-available data only, eliminating the need for data transmission and synchronization. The proposed damage detection method operates directly on the raw ambient vibration condition signals without any filtering or preprocessing. Moreover, the proposed approach requires minimal computational time and power since 1D CNNs merge both feature extraction and classification tasks into a single learning block. This ability is prevailingly cost-effective and evidently practical in WSNs considering the hardware systems have been occasionally reported to suffer from limited power supply in these networks. To display the capability and verify the success of the proposed method, large-scale experiments conducted on a laboratory structure equipped with a state-of-the-art WSN are reported.
---
paper_title: Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks

paper_content:

Abstract Structural health monitoring (SHM) and vibration-based structural damage detection have been a continuous interest for civil, mechanical and aerospace engineers over the decades. Early and meticulous damage detection has always been one of the principal objectives of SHM applications. The performance of a classical damage detection system predominantly depends on the choice of the features and the classifier. While the fixed and hand-crafted features may either be a sub-optimal choice for a particular structure or fail to achieve the same level of performance on another structure, they usually require a large computation power which may hinder their usage for real-time structural damage detection. This paper presents a novel, fast and accurate structural damage detection system using 1D Convolutional Neural Networks (CNNs) that has an inherent adaptive design to fuse both feature extraction and classification blocks into a single and compact learning body. The proposed method performs vibration-based damage detection and localization of the damage in real-time. The advantage of this approach is its ability to extract optimal damage-sensitive features automatically from the raw acceleration signals. Large-scale experiments conducted on a grandstand simulator revealed an outstanding performance and verified the computational efficiency of the proposed real-time damage detection method.
---
paper_title: Structural Damage Detection Using Convolutional Neural Networks

paper_content:

Detection of the deficiencies affecting the performance of the structures has been studied over the past few decades. However, with the long-term data collection from dense sensor arrays, accurate damage diagnosis has become computationally challenging task. To address such problem, this paper introduces convolutional neural network (CNN), which has led to breakthrough results in computer vision, to the damage detection challenge. CNN technique has the ability to discover abstract features which are able to discriminate various aspect of interest. In our case, these features are used to classify “damaged” and “healthy” samples modeled through the finite element simulations. CNN is performed by using a Python library called Theano with the graphics processing unit (GPU) to achieve higher performance of these data-intensive calculations. The accuracy and sensitivity of the proposed technique are assessed with a cracked steel gusset connection model with multiplicative noise. During the training procedure, strain distributions generated from different crack and loading scenarios are adopted. Completely unseen damage setups are introduced to the simulations while testing. Based on the findings of the proposed study, high accuracy, robustness and computational efficiency are succeeded for the damage diagnosis.
---
paper_title: Experimental Phase II of the Structural Health Monitoring Benchmark Problem

paper_content:

This paper introduces the second experimental phase of the activities of the IASC-ASCE Structural Health Monitoring Task Group, involving the application of structural health monitoring techniques to data obtained from a four story steel frame structure tested in August 2002 at the University of British Columbia. These Phase II experimental studies follow a series of analytical studies focusing on a model of the same structure. In the experiment, damage was simulated by removing bracing or loosening bolts within the structure. Three types of excitation were considered: electrodynamic shaker, impact hammer, and ambient vibration. In the shaker tests an electrodynamic shaker on the top floor of the frame was used to excite the structure. Accelerometers were placed throughout the structure to provide measurements of the structural responses. The data and a complete description of the experimental setup are also available at http://wusceel.cive.wustl.edu/asce.shm/ for potential participants to download and examine. Subsequent papers in this session will consider solution procedures for this problem.
---
paper_title: 1-D CNNs for structural damage detection: Verification on a structural health monitoring benchmark data

paper_content:

Abstract Structural damage detection has been an interdisciplinary area of interest for various engineering fields. While the available damage detection methods have been in the process of adapting machine learning concepts, most machine learning based methods extract “hand-crafted” features which are fixed and manually selected in advance. Their performance varies significantly among various patterns of data depending on the particular structure under analysis. Convolutional neural networks (CNNs), on the other hand, can fuse and simultaneously optimize two major sets of an assessment task (feature extraction and classification) into a single learning block during the training phase. This ability not only provides an improved classification performance but also yields a superior computational efficiency. 1D CNNs have recently achieved state-of-the-art performance in vibration-based structural damage detection; however, it has been reported that the training of the CNNs requires significant amount of measurements especially in large structures. In order to overcome this limitation, this paper presents an enhanced CNN-based approach that requires only two measurement sets regardless of the size of the structure. This approach is verified using the experimental data of the Phase II benchmark problem of structural health monitoring which had been introduced by IASC-ASCE Structural Health Monitoring Task Group. As a result, it is shown that the enhanced CNN-based approach successfully estimated the actual amount of damage for the nine damage scenarios of the benchmark study.
---
paper_title: Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks

paper_content:

A number of image processing techniques IPTs have been implemented for detecting civil infrastructure defects to partially replace human-conducted onsite inspections. These IPTs are primarily used to manipulate images to extract defect features, such as cracks in concrete and steel surfaces. However, the extensively varying real-world situations e.g., lighting and shadow changes can lead to challenges to the wide adoption of IPTs. To overcome these challenges, this article proposes a vision-based method using a deep architecture of convolutional neural networks CNNs for detecting concrete cracks without calculating the defect features. As CNNs are capable of learning image features automatically, the proposed method works without the conjugation of IPTs for extracting features. The designed CNN is trained on 40 K images of 256 × 256 pixel resolutions and, consequently, records with about 98% accuracy. The trained CNN is combined with a sliding window technique to scan any image size larger than 256 × 256 pixel resolutions. The robustness and adaptability of the proposed approach are tested on 55 images of 5,888 × 3,584 pixel resolutions taken from a different structure which is not used for training and validation processes under various conditions e.g., strong light spot, shadows, and very thin cracks. Comparative studies are conducted to examine the performance of the proposed CNN using traditional Canny and Sobel edge detection methods. The results show that the proposed method shows quite better performances and can indeed find concrete cracks in realistic situations.
---
paper_title: A novel deep learning-based method for damage identification of smart building structures

paper_content:

In the past few years, intelligent structural damage identification algorithms based on machine learning techniques have been developed and obtained considerable attentions worldwide, due to the ad...
---
paper_title: Efficiency validation of one dimensional convolutional neural networks for structural damage detection using a SHM benchmark data

paper_content:

In this paper, a novel one dimensional convolution neural network (1D-CNN) based structural damage assessment technique is validated with a benchmark study published by IASC-ASCE Structural Health Monitoring Task Group in 2003. In contrast with predominant machine learning based structural damage detection techniques of the literature, the technique shown in this paper runs without manual feature extraction or preprocessing stages. It runs directly on the raw vibration data. In CNNs, the stages of feature extraction and feature classification are merged into one stage; therefore, the proposed technique is efficient, feasible and economical. Utilizing the optimal features learned by 1D CNNs, the proposed CNN-based technique considerably improves the classification efficiency and accuracy. The performance improvement of the proposed technique is assessed by calculating the "Probability of Damage" values for damage estimations. The unseen structural damage cases between the two extreme end structural cases (zero damage and total damage) were successfully identified. Consequently, it is validated that the improved CNN-based technique is efficient since it predicted the level of damage consistently with the structural damage cases defined in the existing benchmark.
---
paper_title: Bearing Fault Detection by One-Dimensional Convolutional Neural Networks

paper_content:

Bearing faults are the biggest single source of motor failures. Artificial Neural Networks (ANNs) and other decision support systems are widely used for early detection of bearing faults. The typical decision support systems require feature extraction and classification as two distinct phases. Extracting fixed features each time may require a significant computational cost preventing their use in real-time applications. Furthermore, the selected features for the classification phase may not represent the most optimal choice. In this paper, the use of 1D Convolutional Neural Networks (CNNs) is proposed for a fast and accurate bearing fault detection system. The feature extraction and classification phases of the bearing fault detection are combined into a single learning body with the implementation of 1D CNN. The raw vibration data (signal) is fed into the proposed system as input eliminating the need for running a separate feature extraction algorithm each time vibration data is analyzed for classification. Implementation of 1D CNNs results in more efficient systems in terms of computational complexity. The classification performance of the proposed system with real bearing data demonstrates that the reduced computational complexity is achieved without a compromise in fault detection accuracy.
---
paper_title: Bearing damage detection via wavelet packet decomposition of the stator current

paper_content:

Bearing faults are one of the major causes of motor failures. The bearing defects induce vibration, resulting in the modulation of the stator current. In this paper, the stator current is analyzed via wavelet packet decomposition to detect bearing defects. The proposed method enables the analysis of frequency bands that can accommodate the rotational speed dependence of the bearing defect frequencies. The wavelet packet decomposition also provides a better treatment of nonstationary stator current than currently used Fourier techniques.
---
paper_title: A Generic Intelligent Bearing Fault Diagnosis System Using Compact Adaptive 1D CNN Classifier

paper_content:

Timely and accurate bearing fault detection and diagnosis is important for reliable and safe operation of industrial systems. In this study, performance of a generic real-time induction bearing fault diagnosis system employing compact adaptive 1D Convolutional Neural Network (CNN) classifier is extensively studied. In the literature, although many studies have developed highly accurate algorithms for detecting bearing faults, their results have generally been limited to relatively small train/test data sets. As opposed to conventional intelligent fault diagnosis systems that usually encapsulate feature extraction, feature selection and classification as distinct blocks, the proposed system takes directly raw time-series sensor data as input and it can efficiently learn optimal features with the proper training. The main advantages of the 1D CNN based approach are 1) its compact architecture configuration (rather than the complex deep architectures) which performs only 1D convolutions making it suitable for real-time fault detection and monitoring, 2) its cost effective and practical real-time hardware implementation, 3) its ability to work without any pre-determined transformation (such as FFT or DWT), hand-crafted feature extraction and feature selection, and 4) its capability to provide efficient training of the classifier with limited size of training data set and limited number of BP iterations. Effectiveness and feasibility of the 1D CNN based fault diagnosis method is validated by applying it to two commonly used benchmark real vibration data sets and comparing the results with the other competing intelligent fault diagnosis methods.
---
paper_title: Early fault diagnosis of rotating machinery based on wavelet packets—Empirical mode decomposition feature extraction and neural network

paper_content:

Abstract After analyzing the shortcomings of current feature extraction and fault diagnosis technologies, a new approach based on wavelet packet decomposition (WPD) and empirical mode decomposition (EMD) are combined to extract fault feature frequency and neural network for rotating machinery early fault diagnosis is proposed. Acquisition signals with fault frequency feature are decomposed into a series of narrow bandwidth using WPD method for de-noising, then, the intrinsic mode functions (IMFs), which usually denoted the features of corresponding frequency bandwidth can be obtained by applying EMD method. Thus, the component of IMF with signal feature can be separated from all IMFs and the energy moment of IMFs is proposed as eigenvector to effectively express the failure feature. The classical three layers BP neural network model taking the fault feature frequency as target input of neural network, the 5 spectral bandwidth energy of vibration signal spectrum as characteristic parameter, and the 10 types of representative rotor fault as output can be established to identify the fault pattern of a machine. Lastly, the fault identification model of rotating machinery with rotor lateral early crack based on BP neural network is taken as an example. The results show that the proposed method can effectively get the signal feature to diagnose the occurrence of early fault of rotating machinery.
---
paper_title: Neural network based motor bearing fault detection

paper_content:

Bearing faults are the biggest single cause of motor failures. The bearing defects induce vibration resulting in the modulation of the stator current. The stator current can be analyzed via wavelet packet decomposition to detect bearing defects. This method enables the analysis of frequency bands that can accommodate the rotational speed dependence of the bearing defect frequencies. In this study, radial basis function neural networks are used to improve bearing fault detection procedure.
---
paper_title: Model-Based Analysis of Spur Gears’ Dynamic Behavior in the Presence of Multiple Cracks

paper_content:

Early detection of tooth cracks is crucial for effective condition-based monitoring and decision making. The scope of this work was to bring more insight into the vibration behavior of spur gears in the presence of single and multiple simultaneous tooth cracks. The investigation was conducted in both time and frequency domains. A finite element analysis was performed to determine the variation in stiffness with respect to the angular position for different combinations of crack lengths. A simplified nonlinear lumped parameter model of a one-stage gearbox with six degrees of freedom was then developed to simulate the vibration response of faulty external spur gears. Four different multiple-crack scenarios were proposed and studied. The performances of various statistical fault detection indicators were considered and investigated. The simulation results obtained via MATLAB indicated that, as the severity of a single crack increases, the values of the time domain statistical indicators increase also, but at different rates. Moreover, the number of cracks was found to have a negative effect on the values of all the performance indicators, except for the RMS. The number and amplitude of the sidebands in the frequency spectrum were also considered, while assessing the severity of the faults in each scenario. It was observed that, in the case of consecutive tooth cracks, the number of spectrum peaks and the number of cracks were consistent in the frequency range of 4-5 kHz. The main finding of this study was that the peak spectral amplitude was the most sensitive indicator of the number and severity of cracks.
---
paper_title: Real-Time Fault Detection and Identification for MMC Using 1-D Convolutional Neural Networks

paper_content:

Automated early detection and identification of switch faults are essential in high-voltage applications. Modular multilevel converter (MMC) is a new and promising topology for such applications. MMC is composed of many identical controlled voltage sources called modules or cells. Each cell may have one or more switches and a switch failure may occur in anyone of these cells. The steady-state normal and fault behavior of a cell voltage will also significantly vary according to the changes in the load current and the fault timing. This makes it a challenging problem to detect and identify such faults as soon as they occur. In this paper, we propose a real-time and highly accurate MMC circuit monitoring system for early fault detection and identification using adaptive one-dimensional convolutional neural networks. The proposed approach is directly applicable to the raw voltage and current data and thus eliminates the need for any feature extraction algorithm, resulting in a highly efficient and reliable system. Simulation results obtained using a four-cell, eight-switch MMC topology demonstrate that the proposed system has a high reliability to avoid any false alarm and achieves a detection probability of 0.989, and average identification probability of 0.997 in less than 100 ms.
---
paper_title: Models for Bearing Damage Detection in Induction Motors Using Stator Current Monitoring

paper_content:

This paper describes new models for the influence of rolling-element bearing faults on induction motor stator current. Bearing problems are one major cause for drive failures. Their detection is possible by vibration monitoring of characteristic bearing frequencies. As it is possible to detect other machine faults by monitoring the stator current, a great interest exists in applying the same method for bearing fault detection. After a presentation of the existing fault model, a new detailed approach is proposed. It is based on two effects of a bearing fault: the introduction of a particular radial rotor movement and load torque variations caused by the bearing fault. The theoretical study results in new expressions for the stator current frequency content. Experimental tests with artificial and realistic bearing damage were conducted by measuring vibration, torque and stator current. The obtained results by spectral analysis of the measured quantities validate the proposed theoretical approach.
---
paper_title: Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks

paper_content:

Early detection of the motor faults is essential and artificial neural networks are widely used for this purpose. The typical systems usually encapsulate two distinct blocks: feature extraction and classification. Such fixed and hand-crafted features may be a suboptimal choice and require a significant computational cost that will prevent their usage for real-time applications. In this paper, we propose a fast and accurate motor condition monitoring and early fault-detection system using 1-D convolutional neural networks that has an inherent adaptive design to fuse the feature extraction and classification phases of the motor fault detection into a single learning body. The proposed approach is directly applicable to the raw data (signal), and, thus, eliminates the need for a separate feature extraction algorithm resulting in more efficient systems in terms of both speed and hardware. Experimental results obtained using real motor data demonstrate the effectiveness of the proposed method for real-time motor condition monitoring.
---
paper_title: Real-Time Fault Detection and Identification for MMC Using 1-D Convolutional Neural Networks

paper_content:

Automated early detection and identification of switch faults are essential in high-voltage applications. Modular multilevel converter (MMC) is a new and promising topology for such applications. MMC is composed of many identical controlled voltage sources called modules or cells. Each cell may have one or more switches and a switch failure may occur in anyone of these cells. The steady-state normal and fault behavior of a cell voltage will also significantly vary according to the changes in the load current and the fault timing. This makes it a challenging problem to detect and identify such faults as soon as they occur. In this paper, we propose a real-time and highly accurate MMC circuit monitoring system for early fault detection and identification using adaptive one-dimensional convolutional neural networks. The proposed approach is directly applicable to the raw voltage and current data and thus eliminates the need for any feature extraction algorithm, resulting in a highly efficient and reliable system. Simulation results obtained using a four-cell, eight-switch MMC topology demonstrate that the proposed system has a high reliability to avoid any false alarm and achieves a detection probability of 0.989, and average identification probability of 0.997 in less than 100 ms.
---
paper_title: Kernel Multilayer Perceptron

paper_content:

We enhance the Multi layer Perceptron to map a feature vector not only from the original d-dimensional feature space, but from an intermediate implicit Hilbert feature space in which kernels calculate inner products. The kernel substitutes the usual inner product between weight vectors and the input vector (or the feature vector of the hidden layer). The objective is to boost the generalization capability of this universal function approximator even more. Classification experiments with standard Machine Learning data sets are shown. We are able to improve the classification accuracy performance criterion for certain kernel types and their intrinsic parameters for the majority of the data sets.
---
paper_title: Complexity Analysis of Multilayer Perceptron Neural Network Embedded into a Wireless Sensor Network

paper_content:

Abstract This paper presents computational and message complexity analysis for a multi-layer perceptron neural network, which is implemented in fully distributed and parallel form across a wireless sensor network. Wireless sensor networks offer a promising platform for parallel and distributed neurocomputing as well as potentially benefiting from artificial neural networks for enhancing their adaptation abilities and computational intelligence. Multilayer perceptron (MLP) neural networks are generic function approximators and classifiers with countless domain-specific applications as reported in the literature. Accordingly, embedding a multilayer perceptron neural network in a wireless sensor network in parallel and distributed mode offers synergy and is very promising. Accordingly, assessing the computational and communication complexity of such hybrid designs, namely an artificial neural network such as a multilayer perceptron network embedded within a wireless sensor network, of interest. This paper presents bounds and results of empirical study on the time, space and message complexity aspects of a wireless sensor network and multilayer perceptron neural network design.
---
paper_title: Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks

paper_content:

Abstract Structural health monitoring (SHM) and vibration-based structural damage detection have been a continuous interest for civil, mechanical and aerospace engineers over the decades. Early and meticulous damage detection has always been one of the principal objectives of SHM applications. The performance of a classical damage detection system predominantly depends on the choice of the features and the classifier. While the fixed and hand-crafted features may either be a sub-optimal choice for a particular structure or fail to achieve the same level of performance on another structure, they usually require a large computation power which may hinder their usage for real-time structural damage detection. This paper presents a novel, fast and accurate structural damage detection system using 1D Convolutional Neural Networks (CNNs) that has an inherent adaptive design to fuse both feature extraction and classification blocks into a single and compact learning body. The proposed method performs vibration-based damage detection and localization of the damage in real-time. The advantage of this approach is its ability to extract optimal damage-sensitive features automatically from the raw acceleration signals. Large-scale experiments conducted on a grandstand simulator revealed an outstanding performance and verified the computational efficiency of the proposed real-time damage detection method.
---
paper_title: Bearing damage detection via wavelet packet decomposition of the stator current

paper_content:

Bearing faults are one of the major causes of motor failures. The bearing defects induce vibration, resulting in the modulation of the stator current. In this paper, the stator current is analyzed via wavelet packet decomposition to detect bearing defects. The proposed method enables the analysis of frequency bands that can accommodate the rotational speed dependence of the bearing defect frequencies. The wavelet packet decomposition also provides a better treatment of nonstationary stator current than currently used Fourier techniques.
---
paper_title: Early fault diagnosis of rotating machinery based on wavelet packets—Empirical mode decomposition feature extraction and neural network

paper_content:

Abstract After analyzing the shortcomings of current feature extraction and fault diagnosis technologies, a new approach based on wavelet packet decomposition (WPD) and empirical mode decomposition (EMD) are combined to extract fault feature frequency and neural network for rotating machinery early fault diagnosis is proposed. Acquisition signals with fault frequency feature are decomposed into a series of narrow bandwidth using WPD method for de-noising, then, the intrinsic mode functions (IMFs), which usually denoted the features of corresponding frequency bandwidth can be obtained by applying EMD method. Thus, the component of IMF with signal feature can be separated from all IMFs and the energy moment of IMFs is proposed as eigenvector to effectively express the failure feature. The classical three layers BP neural network model taking the fault feature frequency as target input of neural network, the 5 spectral bandwidth energy of vibration signal spectrum as characteristic parameter, and the 10 types of representative rotor fault as output can be established to identify the fault pattern of a machine. Lastly, the fault identification model of rotating machinery with rotor lateral early crack based on BP neural network is taken as an example. The results show that the proposed method can effectively get the signal feature to diagnose the occurrence of early fault of rotating machinery.
---
paper_title: Neural network based motor bearing fault detection

paper_content:

Bearing faults are the biggest single cause of motor failures. The bearing defects induce vibration resulting in the modulation of the stator current. The stator current can be analyzed via wavelet packet decomposition to detect bearing defects. This method enables the analysis of frequency bands that can accommodate the rotational speed dependence of the bearing defect frequencies. In this study, radial basis function neural networks are used to improve bearing fault detection procedure.
---
paper_title: Models for Bearing Damage Detection in Induction Motors Using Stator Current Monitoring

paper_content:

This paper describes new models for the influence of rolling-element bearing faults on induction motor stator current. Bearing problems are one major cause for drive failures. Their detection is possible by vibration monitoring of characteristic bearing frequencies. As it is possible to detect other machine faults by monitoring the stator current, a great interest exists in applying the same method for bearing fault detection. After a presentation of the existing fault model, a new detailed approach is proposed. It is based on two effects of a bearing fault: the introduction of a particular radial rotor movement and load torque variations caused by the bearing fault. The theoretical study results in new expressions for the stator current frequency content. Experimental tests with artificial and realistic bearing damage were conducted by measuring vibration, torque and stator current. The obtained results by spectral analysis of the measured quantities validate the proposed theoretical approach.
---
paper_title: Bearing damage detection via wavelet packet decomposition of the stator current

paper_content:

Bearing faults are one of the major causes of motor failures. The bearing defects induce vibration, resulting in the modulation of the stator current. In this paper, the stator current is analyzed via wavelet packet decomposition to detect bearing defects. The proposed method enables the analysis of frequency bands that can accommodate the rotational speed dependence of the bearing defect frequencies. The wavelet packet decomposition also provides a better treatment of nonstationary stator current than currently used Fourier techniques.
---
paper_title: Early fault diagnosis of rotating machinery based on wavelet packets—Empirical mode decomposition feature extraction and neural network

paper_content:

Abstract After analyzing the shortcomings of current feature extraction and fault diagnosis technologies, a new approach based on wavelet packet decomposition (WPD) and empirical mode decomposition (EMD) are combined to extract fault feature frequency and neural network for rotating machinery early fault diagnosis is proposed. Acquisition signals with fault frequency feature are decomposed into a series of narrow bandwidth using WPD method for de-noising, then, the intrinsic mode functions (IMFs), which usually denoted the features of corresponding frequency bandwidth can be obtained by applying EMD method. Thus, the component of IMF with signal feature can be separated from all IMFs and the energy moment of IMFs is proposed as eigenvector to effectively express the failure feature. The classical three layers BP neural network model taking the fault feature frequency as target input of neural network, the 5 spectral bandwidth energy of vibration signal spectrum as characteristic parameter, and the 10 types of representative rotor fault as output can be established to identify the fault pattern of a machine. Lastly, the fault identification model of rotating machinery with rotor lateral early crack based on BP neural network is taken as an example. The results show that the proposed method can effectively get the signal feature to diagnose the occurrence of early fault of rotating machinery.
---
paper_title: Neural network based motor bearing fault detection

paper_content:

Bearing faults are the biggest single cause of motor failures. The bearing defects induce vibration resulting in the modulation of the stator current. The stator current can be analyzed via wavelet packet decomposition to detect bearing defects. This method enables the analysis of frequency bands that can accommodate the rotational speed dependence of the bearing defect frequencies. In this study, radial basis function neural networks are used to improve bearing fault detection procedure.
---
paper_title: Models for Bearing Damage Detection in Induction Motors Using Stator Current Monitoring

paper_content:

This paper describes new models for the influence of rolling-element bearing faults on induction motor stator current. Bearing problems are one major cause for drive failures. Their detection is possible by vibration monitoring of characteristic bearing frequencies. As it is possible to detect other machine faults by monitoring the stator current, a great interest exists in applying the same method for bearing fault detection. After a presentation of the existing fault model, a new detailed approach is proposed. It is based on two effects of a bearing fault: the introduction of a particular radial rotor movement and load torque variations caused by the bearing fault. The theoretical study results in new expressions for the stator current frequency content. Experimental tests with artificial and realistic bearing damage were conducted by measuring vibration, torque and stator current. The obtained results by spectral analysis of the measured quantities validate the proposed theoretical approach.
---
paper_title: Real-Time Motor Fault Detection by 1-D Convolutional Neural Networks

paper_content:

Early detection of the motor faults is essential and artificial neural networks are widely used for this purpose. The typical systems usually encapsulate two distinct blocks: feature extraction and classification. Such fixed and hand-crafted features may be a suboptimal choice and require a significant computational cost that will prevent their usage for real-time applications. In this paper, we propose a fast and accurate motor condition monitoring and early fault-detection system using 1-D convolutional neural networks that has an inherent adaptive design to fuse the feature extraction and classification phases of the motor fault detection into a single learning body. The proposed approach is directly applicable to the raw data (signal), and, thus, eliminates the need for a separate feature extraction algorithm resulting in more efficient systems in terms of both speed and hardware. Experimental results obtained using real motor data demonstrate the effectiveness of the proposed method for real-time motor condition monitoring.
---
paper_title: Generalized model of biological neural networks: Progressive operational perceptrons

paper_content:

Traditional Artificial Neural Networks (ANNs) such as Multi-Layer Perceptrons (MLPs) and Radial Basis Functions (RBFs) were designed to simulate biological neural networks; however, they are based only loosely on biology and only provide a crude model. This in turn yields well-known limitations and drawbacks on the performance and robustness. In this paper we shall address them by introducing a novel feed-forward ANN model, Generalized Operational Perceptrons (GOPs) that consist of neurons with distinct (non-)linear operators to achieve a generalized model of the biological neurons and ultimately a superior diversity. We modified the conventional back-propagation (BP) to train GOPs and furthermore, proposed Progressive Operational Perceptrons (POPs) to achieve self-organized and depth-adaptive GOPs according to the learning problem. The most crucial property of the POPs is their ability to simultaneously search for the optimal operator set and train each layer individually. The final POP is, therefore, formed layer by layer and this ability enables POPs with minimal network depth to attack the most challenging learning problems that cannot be learned by conventional ANNs even with a deeper and significantly complex configuration.
---
paper_title: Real-Time Patient-Specific ECG Classification by 1-D Convolutional Neural Networks

paper_content:

Goal: This paper presents a fast and accurate patient-specific electrocardiogram (ECG) classification and monitoring system. Methods: An adaptive implementation of 1-D convolutional neural networks (CNNs) is inherently used to fuse the two major blocks of the ECG classification into a single learning body: feature extraction and classification. Therefore, for each patient, an individual and simple CNN will be trained by using relatively small common and patient-specific training data, and thus, such patient-specific feature extraction ability can further improve the classification performance. Since this also negates the necessity to extract hand-crafted manual features, once a dedicated CNN is trained for a particular patient, it can solely be used to classify possibly long ECG data stream in a fast and accurate manner or alternatively, such a solution can conveniently be used for real-time ECG monitoring and early alert system on a light-weight wearable device. Results: The results over the MIT-BIH arrhythmia benchmark database demonstrate that the proposed solution achieves a superior classification performance than most of the state-of-the-art methods for the detection of ventricular ectopic beats and supraventricular ectopic beats. Conclusion: Besides the speed and computational efficiency achieved, once a dedicated CNN is trained for an individual patient, it can solely be used to classify his/her long ECG records such as Holter registers in a fast and accurate manner. Significance: Due to its simple and parameter invariant nature, the proposed system is highly generic, and, thus, applicable to any ECG dataset.
---
paper_title: Heterogeneous Multilayer Generalized Operational Perceptron

paper_content:

The traditional Multilayer Perceptron (MLP) using McCulloch-Pitts neuron model is inherently limited to a set of neuronal activities, i.e., linear weighted sum followed by nonlinear thresholding step. Previously, Generalized Operational Perceptron (GOP) was proposed to extend conventional perceptron model by defining a diverse set of neuronal activities to imitate a generalized model of biological neurons. Together with GOP, Progressive Operational Perceptron (POP) algorithm was proposed to optimize a pre-defined template of multiple homogeneous layers in a layerwise manner. In this paper, we propose an efficient algorithm to learn a compact, fully heterogeneous multilayer network that allows each individual neuron, regardless of the layer, to have distinct characteristics. Based on the complexity of the problem, the proposed algorithm operates in a progressive manner on a neuronal level, searching for a compact topology, not only in terms of depth but also width, i.e., the number of neurons in each layer. The proposed algorithm is shown to outperform other related learning methods in extensive experiments on several classification problems.
---
paper_title: Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks

paper_content:

Abstract Structural health monitoring (SHM) and vibration-based structural damage detection have been a continuous interest for civil, mechanical and aerospace engineers over the decades. Early and meticulous damage detection has always been one of the principal objectives of SHM applications. The performance of a classical damage detection system predominantly depends on the choice of the features and the classifier. While the fixed and hand-crafted features may either be a sub-optimal choice for a particular structure or fail to achieve the same level of performance on another structure, they usually require a large computation power which may hinder their usage for real-time structural damage detection. This paper presents a novel, fast and accurate structural damage detection system using 1D Convolutional Neural Networks (CNNs) that has an inherent adaptive design to fuse both feature extraction and classification blocks into a single and compact learning body. The proposed method performs vibration-based damage detection and localization of the damage in real-time. The advantage of this approach is its ability to extract optimal damage-sensitive features automatically from the raw acceleration signals. Large-scale experiments conducted on a grandstand simulator revealed an outstanding performance and verified the computational efficiency of the proposed real-time damage detection method.
---
paper_title: A logical calculus of the ideas immanent in nervous activity

paper_content:

Abstract Because of the “all-or-none” character of nervous activity, neural events and the relations among them can be treated by means of propositional logic. It is found that the behavior of every net can be described in these terms, with the addition of more complicated logical means for nets containing circles; and that for any logical expression satisfying certain conditions, one can find a net behaving in the fashion it describes. It is shown that many particular choices among possible neurophysiological assumptions are equivalent, in the sense that for every net behaving under one assumption, there exists another net which behaves under the other and gives the same results, although perhaps not in the same time. Various applications of the calculus are discussed.
---

Title: 1D Convolutional Neural Networks and Applications: A Survey

Section 1: Introduction
Description 1: Introduce the background and motivation for studying 1D Convolutional Neural Networks (CNNs), including historical context and foundational developments.

Section 2: Overview of Convolutional Neural Networks
Description 2: Provide a general background on Convolutional Neural Networks (CNNs), focusing on their importance and development over time.

Section 3: 2D Convolutional Neural Networks
Description 3: Detail the structure and functionality of traditional 2D CNNs, highlighting their components, advantages, and popular architectures.

Section 4: 1D Convolutional Neural Networks
Description 4: Introduce 1D CNNs and discuss their specific features, advantages, applications, and differences compared to 2D CNNs.

Section 5: Forward-and Back-Propagation in CNN-layers
Description 5: Explain the processes of Forward Propagation (FP) and Back-Propagation (BP) in the context of CNN layers, particularly focusing on the 1D CNNs.

Section 6: Applications of 1D CNNs
Description 6: Present various major engineering applications of 1D CNNs, providing detailed examples and studies across different domains.

Section 7: Computational Complexity Analysis of 1D-CNNs
Description 7: Analyze the computational complexity involved in both forward and backward propagation processes in 1D CNNs.

Section 8: Conclusions
Description 8: Summarize key points of the survey, highlight the significance of 1D CNNs, and suggest possible future directions for research and application.