A Comprehensive Survey of Prognostics and Health Management Based on Deep Learning for Autonomous Ships
---
paper_title: Analyzing the economic benefit of unmanned autonomous ships: An exploratory cost-comparison between an autonomous and a conventional bulk carrier

paper_content:

Abstract Unmanned autonomous ships are seen as a key element of a competitive and sustainable European shipping industry in future. But even if the technology to further automate ships will principally be available at some point, this does not imply that autonomous vessels are also the superior choice for the ship owner. In the end the success of autonomous vessels depends on their impact on the profitability of shipping companies. Following a structured approach this paper analyzes the costs of running an autonomous bulker and compares them against a conventional vessel in a cost-benefit analysis. Hereby it provides insights on the (economic) benefit of autonomous vessels for a first-time. Results principally confirm an economic potential. The expected present value of cost of owning and operating the autonomous bulker over a 25-year period is mUSD 4.3 lower than for a conventionally manned ship. Assuming identical cargo carrying capacity this means that the required freight rate of the autonomous bulker which produces a zero net present value is 3.4% lower than the required freight rate of the conventional vessel. This advantageousness is based on one aspect in particular as the paper argues. Besides cost savings associated with reducing crew levels an autonomous ship brings along additional benefits due to changes in ship design.
---
paper_title: Regional deep learning model for visual tracking

paper_content:

Deep learning has been successfully applied to visual tracking due to its powerful feature learning characteristic. However, existing deep learning trackers rely on single observation model and focus on the holistic representation of the tracking object. When occlusion occurs, the trackers suffer from the contaminated features obtained in occluded areas. In this paper, we propose a regional deep learning tracker that observes the target by multiple sub-regions and each region is observed by a deep learning model. In particular, we devise a stable factor, modeled as a hidden variable of the Factorial Hidden Markov Model, to characterize the stability of these sub-models. The stability indicator not only provides a confidence degree for the response score of each model during inference stage, but also determines the online training criteria for each deep learning model. This online training strategy enables the tracker to achieve more accurate local features compared with those fixed training trackers. In addition, to improve the computational efficiency, we exploit the structurized response property of the customized deep learning model to approximate the final tracking results by the weighted Gaussian Mixture Model under the particle filter framework. Qualitative and quantitative evaluations on the recent public benchmark dataset show that our approach outperforms most state-of-the-art trackers.
---
paper_title: A review of unsupervised feature learning and deep learning for time-series modeling

paper_content:

This paper gives a review of the recent developments in deep learning and unsupervised feature learning for time-series problems. While these techniques have shown promise for modeling static data, such as computer vision, applying them to time-series data is gaining increasing attention. This paper overviews the particular challenges present in time-series data and provides a review of the works that have either applied time-series data to unsupervised feature learning algorithms or alternatively have contributed to modifications of feature learning algorithms to take into account the challenges present in time-series data.
---
paper_title: Sequence to Sequence Learning with Neural Networks

paper_content:

Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this paper, we present a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure. Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector. Our main result is that on an English to French translation task from the WMT-14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did not have difficulty on long sentences. For comparison, a phrase-based SMT system achieves a BLEU score of 33.3 on the same dataset. When we used the LSTM to rerank the 1000 hypotheses produced by the aforementioned SMT system, its BLEU score increases to 36.5, which is close to the previous state of the art. The LSTM also learned sensible phrase and sentence representations that are sensitive to word order and are relatively invariant to the active and the passive voice. Finally, we found that reversing the order of the words in all source sentences (but not target sentences) improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
---
paper_title: Research on PHM technology application of ship maintenance program optimization

paper_content:

In order to change the current disadvantages of traditional ship maintenance concept, this article plans to apply the technical method of prognostics and health management (PHM) to improve equipment maintenance program and repair schedule. In the meantime, this thesis takes account into the example of the dredger power system of the Waterway Bureau to establish the PHM optimized system of life cycle ship maintenance program, which include the system architecture, work platforms and optimize processes.
---
paper_title: Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]

paper_content:

This article provides an overview of the mainstream deep learning approaches and research directions proposed over the past decade. It is important to emphasize that each approach has strengths and "weaknesses, depending on the application and context in "which it is being used. Thus, this article presents a summary on the current state of the deep machine learning field and some perspective into how it may evolve. Convolutional Neural Networks (CNNs) and Deep Belief Networks (DBNs) (and their respective variations) are focused on primarily because they are well established in the deep learning field and show great promise for future work.
---
paper_title: Big Data Deep Learning: Challenges and Perspectives

paper_content:

Deep learning is currently an extremely active research area in machine learning and pattern recognition society. It has gained huge successes in a broad area of applications such as speech recognition, computer vision, and natural language processing. With the sheer size of data available today, big data brings big opportunities and transformative potential for various sectors; on the other hand, it also presents unprecedented challenges to harnessing data and information. As the data keeps getting bigger, deep learning is coming to play a key role in providing big data predictive analytics solutions. In this paper, we provide a brief overview of deep learning, and highlight current research efforts and the challenges to big data, as well as the future trends.
---
paper_title: Data-driven approaches in health condition monitoring — A comparative study

paper_content:

In this paper, four data-driven classification approaches, that is, K-nearest neighbors (K-NN), self-organizing map (SOM), multi-layer perceptron (MLP), and Bayesian Network classifier (BNC), are applied to a health condition monitoring problem for the wearing cutter. The dataset is produced from a cutting machine using force sensing. A genetic algorithm (GA) based search is performed to select 3 dominant features from a 16-dimensional feature space, which is computed directly from the real dataset. Subsequently K-NN, SOM, MLP, and BNC algorithms are trained to predict the wearing status of the cutter, respectively. The suitability of the four data-driven approaches for the health condition monitoring are investigated and compared.
---
paper_title: Autonomous ships on the high seas

paper_content:

It's midnight on the North Atlantic, where a massive container ship receives the latest weather report. There's a nasty storm brewing ahead. Quietly, the ship changes course and speed, to skirt the worst of it and ensure an on-time arrival at its destination. The ship's owners and the harbormaster at its next port of call are advised of the revised route. And as it nears shore, the giant ship must correct course once again, this time to steer clear of a fishing vessel off its starboard bow.
---
paper_title: Towards autonomous maritime operations

paper_content:

The main purpose to push the development towards autonomous maritime operations in shipping and offshore installations, is to increase the performance of maritime activities; by social benefits for staff or other related personal groups; by economic benefits when the ability to increase the cargo or effectuate better space allocation; by environmental benefits that's allow the ship operations to be optimized for routing, speed, etc.; and finally by an increased safety benefit in all these aspects. The increased use of artificial intelligent based benefits is expected to increase the operational performance of all the above aspects in the sense that an overall quality in logical and knowledge based will be consistent. The increased complexity of maritime activities and the offshore business requires more precise and automated solutions to achieve the expectations from staff, stakeholders, and society.
---
paper_title: Failure diagnosis using deep belief learning based health state classification

paper_content:

Effective health diagnosis provides multifarious benefits such as improved safety, improved reliability and reduced costs for operation and maintenance of complex engineered systems. This paper presents a novel multi-sensor health diagnosis method using deep belief network (DBN). DBN has recently become a popular approach in machine learning for its promised advantages such as fast inference and the ability to encode richer and higher order network structures. The DBN employs a hierarchical structure with multiple stacked restricted Boltzmann machines and works through a layer by layer successive learning process. The proposed multi-sensor health diagnosis methodology using DBN based state classification can be structured in three consecutive stages: first, defining health states and preprocessing sensory data for DBN training and testing; second, developing DBN based classification models for diagnosis of predefined health states; third, validating DBN classification models with testing sensory dataset. Health diagnosis using DBN based health state classification technique is compared with four existing diagnosis techniques. Benchmark classification problems and two engineering health diagnosis applications: aircraft engine health diagnosis and electric power transformer health diagnosis are employed to demonstrate the efficacy of the proposed approach.
---
paper_title: System health monitoring and prognostics — a review of current paradigms and practices

paper_content:

System health monitoring is a set of activities performed on a system to maintain it in operable condition. Monitoring may be limited to the observation of current system states, with maintenance and repair actions prompted by these observations. Alternatively, monitoring of current system states is being augmented with prediction of future operating states and predictive diagnosis of future failure states.Such predictive diagnosis or prognosis is motivated by the need for manufacturers and other operators of complex systems to optimize equipment performance and reduce costs and unscheduled downtime. Prognosis is a difficult task requiring precise, adaptive and intuitive models to predict future machine health states. Numerous modeling techniques have been proposed in the literature and implemented in practice. This paper reviews the philosophies and techniques that focus on improving reliability and reducing unscheduled downtime by monitoring and predicting machine health.
---
paper_title: Remaining Useful Life Estimation in Prognostics Using Deep Convolution Neural Networks

paper_content:

Traditionally, system prognostics and health management (PHM) depends on sufficient prior knowledge of critical components degradation process in order to predict the remaining useful life (RUL). However, the accurate physical or expert models are not available in most cases. This paper proposes a new data-driven approach for prognostics using deep convolution neural networks (DCNN). Time window approach is employed for sample preparation in order for better feature extraction by DCNN. Raw collected data with normalization are directly used as inputs to the proposed network, and no prior expertise on prognostics and signal processing is required, that facilitates the application of the proposed method. In order to show the effectiveness of the proposed approach, experiments on the popular C-MAPSS dataset for aero-engine unit prognostics are carried out. High prognostic accuracy on the RUL estimation is achieved. The superiority of the proposed method is demonstrated by comparisons with other popular approaches and the state-of-the-art results on the same dataset. The results of this study suggest that the proposed data-driven prognostic method offers a new and promising approach.
---
paper_title: Comparison of Data-Driven Reconstruction Methods For Fault Detection

paper_content:

This work proposes a comparison of three data-driven signal reconstruction methods, which are Auto-Associative Kernel Regression (AAKR), Fuzzy Similarity (FS), and Elman Recurrent Neural Network (RNN), for fault detection based on the difference between the signal observations and the reconstructions of the signal in normal (typical) operating conditions. The aim is to show the capabilities and drawbacks of the methods, and propose a strategy for the aggregation of their outcomes, to overcome their limitations. For this purpose, the performance of each method is evaluated in terms of fault detection capability, considering accuracy, robustness, and resistance to the spillover effect of the obtained signal reconstructions. The comparison is supported by the application to a real industrial case study regarding temperature signals collected during operation of a rotating machine in an energy production plant. An ensemble of the three methods is proposed to overcome the limitations of the three methods.
---
paper_title: A survey of machine learning for big data processing

paper_content:

There is no doubt that big data are now rapidly expanding in all science and engineering domains. While the potential of these massive data is undoubtedly significant, fully making sense of them requires new ways of thinking and novel learning techniques to address the various challenges. In this paper, we present a literature survey of the latest advances in researches on machine learning for big data processing. First, we review the machine learning techniques and highlight some promising learning methods in recent studies, such as representation learning, deep learning, distributed and parallel learning, transfer learning, active learning, and kernel-based learning. Next, we focus on the analysis and discussions about the challenges and possible solutions of machine learning for big data. Following that, we investigate the close connections of machine learning with signal processing techniques for big data processing. Finally, we outline several open issues and research trends.
---
paper_title: Deep Learning and Its Applications to Signal and Information Processing

paper_content:

INTRODUCTION TO DEEP LEARNING Many traditional machine learning and signal processing techniques exploit shallow architectures, which contain a single layer of nonlinear feature transformation. Examples of shallow architectures are conventional hidden Markov models (HMMs), linear or nonlinear dynamical systems, conditional random fields (CRFs), maximum entropy (MaxEnt) models, support vector machines (SVMs), kernel regression, and multilayer perceptron (MLP) with a single hidden layer. A property common to these shallow learning models is the simple architecture that consists of only one layer responsible for transforming the raw input signals or features into a problem-specific feature space, which may be unobservable. Take the example of a support vector machine. It is a shallow linear separation model with one feature transformation layer when kernel trick is used, and with zero feature transformation layer when kernel trick is not used. Human information processing
---
paper_title: Prognostics and health management design for rotary machinery systems—Reviews, methodology and applications

paper_content:

Much research has been conducted in prognostics and health management (PHM), an emerging field in mechanical engineering that is gaining interest from both academia and industry. Most of these efforts have been in the area of machinery PHM, resulting in the development of many algorithms for this particular application. The majority of these algorithms concentrate on applications involving common rotary machinery components, such as bearings and gears. Knowledge of this prior work is a necessity for any future research efforts to be conducted; however, there has not been a comprehensive overview that details previous and on-going efforts in PHM. In addition, a systematic method for developing and deploying a PHM system has yet to be established. Such a method would enable rapid customization and integration of PHM systems for diverse applications. To address these gaps, this paper provides a comprehensive review of the PHM field, followed by an introduction of a systematic PHM design methodology, 5S methodology, for converting data to prognostics information. This methodology includes procedures for identifying critical components, as well as tools for selecting the most appropriate algorithms for specific applications. Visualization tools are presented for displaying prognostics information in an appropriate fashion for quick and accurate decision making. Industrial case studies are included in this paper to show how this methodology can help in the design of an effective PHM system.
---
paper_title: Generating Text with Recurrent Neural Networks

paper_content:

Recurrent Neural Networks (RNNs) are very powerful sequence models that do not enjoy widespread use because it is extremely difficult to train them properly. Fortunately, recent advances in Hessian-free optimization have been able to overcome the difficulties associated with training RNNs, making it possible to apply them successfully to challenging sequence problems. In this paper we demonstrate the power of RNNs trained with the new Hessian-Free optimizer (HF) by applying them to character-level language modeling tasks. The standard RNN architecture, while effective, is not ideally suited for such tasks, so we introduce a new RNN variant that uses multiplicative (or "gated") connections which allow the current input character to determine the transition matrix from one hidden state vector to the next. After training the multiplicative RNN with the HF optimizer for five days on 8 high-end Graphics Processing Units, we were able to surpass the performance of the best previous single method for character-level language modeling – a hierarchical non-parametric sequence model. To our knowledge this represents the largest recurrent neural network application to date.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: A Review on Basic Data-Driven Approaches for Industrial Process Monitoring

paper_content:

Recently, to ensure the reliability and safety of modern large-scale industrial processes, data-driven methods have been receiving considerably increasing attention, particularly for the purpose of process monitoring. However, great challenges are also met under different real operating conditions by using the basic data-driven methods. In this paper, widely applied data-driven methodologies suggested in the literature for process monitoring and fault diagnosis are surveyed from the application point of view. The major task of this paper is to sketch a basic data-driven design framework with necessary modifications under various industrial operating conditions, aiming to offer a reference for industrial process monitoring on large-scale industrial processes.
---
paper_title: Long Short-Term Memory Network for Remaining Useful Life estimation

paper_content:

Remaining Useful Life (RUL) of a component or a system is defined as the length from the current time to the end of the useful life. Accurate RUL estimation plays a critical role in Prognostics and Health Management(PHM). Data driven approaches for RUL estimation use sensor data and operational data to estimate RUL. Traditional regression based approaches and recent Convolutional Neural Network (CNN) approach use features created from sliding windows to build models. However, sequence information is not fully considered in these approaches. Sequence learning models such as Hidden Markov Models (HMMs) and Recurrent Neural Networks (RNNs) have flaws when modeling sequence information. HMMs are limited to discrete hidden states and are known to have issues when modeling long-term dependencies in the data. RNNs also have issues with long-term dependencies. In this work, we propose a Long Short-Term Memory (LSTM) approach for RUL estimation, which can make full use of the sensor sequence information and expose hidden patterns within sensor data with multiple operating conditions, fault and degradation models. Extensive experiments using three widely adopted Prognostics and Health Management data sets show that LSTM for RUL estimation significantly outperforms traditional approaches for RUL estimation as well as Convolutional Neural Network (CNN).
---
paper_title: Research advances in fault diagnosis and prognostic based on deep learning

paper_content:

Aiming to condition based maintenance for complex equipment, numerous intelligent fault diagnosis and prognostic methods based on machine learning have been researched. Compared with the traditional shallow models, which have problems of lacking expression capacity and existing the curse of dimensionality, using deep learning theory can effectively mine characteristics and accurately recognize the health condition. In consequence, fault diagnosis and prognostic based on deep learning have turned into an innovative and promising research field. This paper gives a review of fault diagnosis and prognostic based on deep learning. First of all, a brief introduction to deep learning architecture and the framework of fault diagnosis based on deep learning is described. Second, tracking describes the latest progress of fault diagnosis and prognostic based on deep learning in chronological order. In this section, the deep learning methods used in fault diagnosis and prognostic are discussed, including Deep Neural Network (DNN), Deep Belief Network (DBN) and Convolutional Neural Network (CNN). Then the engineering application fields are summarized, such as mechanical equipment diagnosis, electrical equipment diagnosis, etc. Finally, this paper indicates the potential future research issues in this field.
---
paper_title: A survey of deep neural network architectures and their applications

paper_content:

This work was supported in part the Royal Society of the UK, the National Natural Science Foundation of China under Grants 61329301, 61374010, and 61403319, and the Alexander von Humboldt Foundation of Germany.
---
paper_title: Building high-level features using large scale unsupervised learning

paper_content:

We consider the problem of building high-level, class-specific feature detectors from only unlabeled data. For example, is it possible to learn a face detector using only unlabeled images using unlabeled images? To answer this, we train a 9-layered locally connected sparse autoencoder with pooling and local contrast normalization on a large dataset of images (the model has 1 billion connections, the dataset has 10 million 200×200 pixel images downloaded from the Internet). We train this network using model parallelism and asynchronous SGD on a cluster with 1,000 machines (16,000 cores) for three days. Contrary to what appears to be a widely-held intuition, our experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not. Control experiments show that this feature detector is robust not only to translation but also to scaling and out-of-plane rotation. We also find that the same network is sensitive to other high-level concepts such as cat faces and human bodies. Starting with these learned features, we trained our network to obtain 15.8% accuracy in recognizing 20,000 object categories from ImageNet, a leap of 70% relative improvement over the previous state-of-the-art.
---
paper_title: Creating a closed loop evironment for condition based maintenance plus (CMB+) and prognostics health management

paper_content:

One important facet of CBM+ is to formulate the assessment of the system based upon objective evaluation of degraded performance or impending failure. Only when the objective evidence substantiates the need to perform a maintenance action would the system be pulled in for maintenance. Viability and acceptance of predictive measures and appropriate metrics to ensure success have crippled efforts to add prognostic systems in the past. Only a portion of the diagnostic information is available directly from the defense system. To get the full picture requires being aware of repair activity at all levels of maintenance to correlate initial conditions to the actual root cause of the failure. This is usually the downfall of most attempts at intelligent maintenance concepts. We have typically done a horrible job of keeping track of parts in maintenance throughout the enterprise. The US Army Research, Development, and Engineering Command (RDECOM), Aviation and Missile Research, Development and Engineering Center (AMRDEC), Diagnostic/Prognostic Laboratory has developed a closed loop environment to address critical aspects of system operation, fault classification, and prognostic health management (PHM). The environment was derived from our automatic test system (ATS) improvement program to develop a closed loop system to monitor ATS performance in the field remotely and be able to provide seamless updates to the operators around the world. Recognizing that today's complex defense system data sources are diverse in their form and their context requires special attention to feature extraction and fusion to reduce potential transmission overload. Periodicity related to the sampling of data and the correlation of the data set to particular incipient fault evidence is of interest to develop a rational architecture for a systems approach to health management. In many cases we monitor individual subsystems or components and singularly diagnose failures into ambiguity groups due to insufficient discrimination of diagnostic information. Therefore we blindly wait for a failure to occur, diagnose the problem, order the appropriate part, wait for its arrival, replace the component upon arrival, test the repaired system, and put it back into operational status. This is not satisfactory any longer.
---
paper_title: Diagnostics To Prognostics - A Product Availability Technology Evolution

paper_content:

From the customer's perspective, one of the most critical expectations of R&M in the new frontier focuses on product availability and cost of ownership. Whether you consider a personal automobile or a pilot's avionics, it must operate reliably, on demand, at an affordable cost. Consumer products such as automobiles, aircraft engines, medical diagnostic equipment, and computer server farms already employ remote diagnostics to monitor product health and optimize product availability. In this context, the next frontier is to evolve robust diagnostics into maturing prognostics and embed these technologies in our products in the form of health management system (HMS) capabilities. Using health based information for real time mission planning and autonomic maintenance is a key enabler for achieving ultra-reliability and availability in tomorrow's system of systems environment. The problem we face is how to best achieve these HMS capable products in a cost effective manner. Today's robust diagnostic technologies are proving to be key enablers for condition based maintenance, improved fault detection/fault isolation, and lower operational cost. Some success has been realized with the insertion of early prognostic technology, primarily in the area of mechanical PHM applications. By using a systematic methodology to develop, implement, and mature electronic prognostics into the next generation of HMS capable products, we are providing our customers (and most importantly the end user in the field) with new insight into the health and system readiness of our products. Being able to make health based decisions for mission planning, real time operational modes, and just-in-time maintenance will become the key to meeting product availability at an affordable cost
---
paper_title: Defining PHM, A Lexical Evolution of Maintenance and Logistics

paper_content:

Prognostics and health management (PHM) is an approach to system life-cycle support that seeks to reduce/eliminate inspections and time-based maintenance through accurate monitoring, incipient fault detection, and prediction of impending faults. Coupled with autonomic logistics for unprecedented responsiveness, cost effectiveness, and mission availability, PHM is largely automated in its application. Incorporating the principles of condition-based maintenance (CBM) along with the tenets of reliability-centered maintenance (RCM), the PHM paradigm extends these capabilities and provides a robust environment to optimize maintenance and logistics for increased operational availability (A0), and reduced life-cycle costs (LCC) while potentially increasing the reliability and life expectancy of mechanical, structural, and electronic systems. Driven by a demand for greater reliability at reduced cost and fueled by technological advancements, the PHM contribution to an already robust and confounding vocabulary surrounding maintenance and logistics is significant. As adopters of PHM technology attempt to define requirements and performance parameters, difficulties encountered with various non- standardized terminology indicate that the PHM vocabulary merits a lexical review. This paper will provide a compendium of PHM terminology along with definitions and examples, derived from the authors' experience in the implementation of PHM systems. Coalescing existing vocabularies and introducing, formally, the new lexicon of maintenance and logistics, the authors seek to aid in clarification of the emerging dialogue of life-cycle support.
---
paper_title: A review on machinery diagnostics and prognostics implementing condition-based maintenance

paper_content:

Condition-based maintenance (CBM) is a maintenance program that recommends maintenance decisions based on the information collected through condition monitoring. It consists of three main steps: data acquisition, data processing and maintenance decision-making. Diagnostics and prognostics are two important aspects of a CBM program. Research in the CBM area grows rapidly. Hundreds of papers in this area, including theory and practical applications, appear every year in academic journals, conference proceedings and technical reports. This paper attempts to summarise and review the recent research and developments in diagnostics and prognostics of mechanical systems implementing CBM with emphasis on models, algorithms and technologies for data processing and maintenance decision-making. Realising the increasing trend of using multiple sensors in condition monitoring, the authors also discuss different techniques for multiple sensor data fusion. The paper concludes with a brief discussion on current practices and possible future trends of CBM.
---
paper_title: Self-maintenance and engineering immune systems: Towards smarter machines and manufacturing systems

paper_content:

Abstract This paper discusses the state-of-the-art research in the areas of self-maintenance and engineering immune systems (EIS) for machines with smarter adaptability to operating regime changes in future manufacturing systems. Inspired by the biological immune and nervous systems, the authors are introducing the transformation of prognostics and health management (PHM) to engineering immune systems (EIS). First, an overview on PHM is introduced. Its transformation toward resilient systems, self-maintenance systems, and engineering immune systems is also discussed. Finally, new concepts in developing future biological-based smarter machines based on autonomic computing and cloud computing are discussed.
---
paper_title: Methodologies for Integration of PHM Systems with Maintenance Data

paper_content:

The automatic logistics program in the Air Force seeks to reduce development, production, and ownership costs for the next generation fighter aircraft by increasing system reliability while reducing maintenance requirements. A large number of technologies are becoming available within the Prognostics and Health Management (PHM) community that will lead to reduced cost and increased availability.12 The challenge is to develop advanced technology to integrate available PHM information from a variety of different sources into the maintenance and logistics infrastructure. PHM and maintenance/logistics systems must be thoroughly examined and tightly integrated in order to perform maintenance actions in the most efficient way to reduce ownership cost and increase availability. This paper presents multi-agent technology that integrates maintenance and PHM data to provide more effective maintenance identification and scheduling. The proposed methodologies will enable the maintenance and logistics infrastructure to fully benefit from newly developed PHM systems. Additionally, the PHM systems update themselves based on feedback obtained from the maintenance systems. The integration will utilize intelligent software agent technology in order to develop such solutions within open, highly dynamic, uncertain and complex environments with data distributed over a network. This provides benefits such as reusability, scalability, and continuous improvement with dynamically evolving ability.
---
paper_title: Remote prognosis, diagnosis and maintenance for automotive architecture based on least squares support vector machine and multiple classifiers

paper_content:

Software issues related to automotive controls account for an increasingly large percentage of the overall vehicles recalled. To alleviate this problem, vehicle diagnosis and maintenance systems are increasingly being performed remotely, that is while the vehicle is being driven without need for factory recall and there is strong consumer interest in Remote Diagnosis and Maintenance (RD&M) systems. Such systems are developed with different building blocks/elements and various capabilities. This paper presents a novel automotive RD&M system and prognosis architecture. The elements of the proposed system include vehicles, smart phones, maintenance service centers, vehicle manufacturer, RD&M experts, RD&M service centers, logistics carry centers, and emergency centers. The system promotes the role of smart phones used to run prognosis and diagnosis tools based on Least Squares Support Vector Machine (LS-SVM) multiple classifiers. During the prognosis phase, the smart phone stores the history of any forecast failures and send them, only if any failure already occurred during the diagnosis, to the RD&M service centre. The later will then forward it to RD&M experts as a real failure data to improve the training data used in prognosis classification and predication of the remaining useful life (RUL). The (LS-SVM) is used widely in prognostics and system health management of spacecraft in-orbit and it is applied to monitor spacecraft's performance, detect faults, identify the root cause of the fault, and predict RUL. The same approach is applied in this paper. Finally, the RD&M software architectures for the vehicle and the smart phone are presented.
---
paper_title: Prognostic modelling options for remaining useful life estimation by industry

paper_content:

Abstract Over recent years a significant amount of research has been undertaken to develop prognostic models that can be used to predict the remaining useful life of engineering assets. Implementations by industry have only had limited success. By design, models are subject to specific assumptions and approximations, some of which are mathematical, while others relate to practical implementation issues such as the amount of data required to validate and verify a proposed model. Therefore, appropriate model selection for successful practical implementation requires not only a mathematical understanding of each model type, but also an appreciation of how a particular business intends to utilise a model and its outputs. This paper discusses business issues that need to be considered when selecting an appropriate modelling approach for trial. It also presents classification tables and process flow diagrams to assist industry and research personnel select appropriate prognostic models for predicting the remaining useful life of engineering assets within their specific business environment. The paper then explores the strengths and weaknesses of the main prognostics model classes to establish what makes them better suited to certain applications than to others and summarises how each have been applied to engineering prognostics. Consequently, this paper should provide a starting point for young researchers first considering options for remaining useful life prediction. The models described in this paper are Knowledge-based (expert and fuzzy), Life expectancy (stochastic and statistical), Artificial Neural Networks, and Physical models.
---
paper_title: A review of diagnostic and prognostic capabilities and best practices for manufacturing

paper_content:

Prognostics and health management (PHM) technologies reduce time and costs for maintenance of products or processes through efficient and cost-effective diagnostic and prognostic activities. PHM systems use real-time and historical state information of subsystems and components to provide actionable information, enabling intelligent decision-making for improved performance, safety, reliability, and maintainability. However, PHM is still an emerging field, and much of the published work has been either too exploratory or too limited in scope. Future smart manufacturing systems will require PHM capabilities that overcome current challenges, while meeting future needs based on best practices, for implementation of diagnostics and prognostics. This paper reviews the challenges, needs, methods, and best practices for PHM within manufacturing systems. This includes PHM system development of numerous areas highlighted by diagnostics, prognostics, dependability analysis, data management, and business. Based on current capabilities, PHM systems are shown to benefit from open-system architectures, cost-benefit analyses, method verification and validation, and standards.
---
paper_title: Framework for post-prognostic decision support

paper_content:

This paper describes a decision support system (DSS) for use in operational decision making with PHM-specific data. Challenges arise from the large amount of different information pieces upon which a decision maker has to act. Conflicting information from on-board and off-board PHM modules, seemingly contradictory and changing requirements from operations as well as maintenance for a multitude of different systems within strict time constraints make operational decision-making a difficult undertaking. The DSS enables the user to make optimal decisions based on his expression of rigorous trade-offs between different prognostic and external information sources. This is accomplished through guided evaluation of different optimal decision alternatives under operational boundary conditions using user-specific and interactive collaboration. We present some preliminary results of the use of such a DSS for post-prognostics decision-making.
---
paper_title: Dependability of wireless sensor networks for industrial prognostics and health management

paper_content:

HighlightsState of the art in prognostics and health management.Limitations of prognostic models.Dependability of wireless sensor networks.Challenges of remaining useful life prediction. Maintenance is an important activity in industry. It is performed either to revive a machine/component or to prevent it from breaking down. Different strategies have evolved through time, bringing maintenance to its current state: condition-based and predictive maintenances. This evolution was due to the increasing demand of reliability in industry. The key process of condition-based and predictive maintenances is prognostics and health management, and it is a tool to predict the remaining useful life of engineering assets. Nowadays, plants are required to avoid shutdowns while offering safety and reliability. Nevertheless, planning a maintenance activity requires accurate information about the system/component health state. Such information is usually gathered by means of independent sensor nodes. In this study, we consider the case where the nodes are interconnected and form a wireless sensor network. As far as we know, no research work has considered such a case of study for prognostics. Regarding the importance of data accuracy, a good prognostics requires reliable sources of information. This is why, in this paper, we will first discuss the dependability of wireless sensor networks, and then present a state of the art in prognostic and health management activities.
---
paper_title: Defining PHM, A Lexical Evolution of Maintenance and Logistics

paper_content:

Prognostics and health management (PHM) is an approach to system life-cycle support that seeks to reduce/eliminate inspections and time-based maintenance through accurate monitoring, incipient fault detection, and prediction of impending faults. Coupled with autonomic logistics for unprecedented responsiveness, cost effectiveness, and mission availability, PHM is largely automated in its application. Incorporating the principles of condition-based maintenance (CBM) along with the tenets of reliability-centered maintenance (RCM), the PHM paradigm extends these capabilities and provides a robust environment to optimize maintenance and logistics for increased operational availability (A0), and reduced life-cycle costs (LCC) while potentially increasing the reliability and life expectancy of mechanical, structural, and electronic systems. Driven by a demand for greater reliability at reduced cost and fueled by technological advancements, the PHM contribution to an already robust and confounding vocabulary surrounding maintenance and logistics is significant. As adopters of PHM technology attempt to define requirements and performance parameters, difficulties encountered with various non- standardized terminology indicate that the PHM vocabulary merits a lexical review. This paper will provide a compendium of PHM terminology along with definitions and examples, derived from the authors' experience in the implementation of PHM systems. Coalescing existing vocabularies and introducing, formally, the new lexicon of maintenance and logistics, the authors seek to aid in clarification of the emerging dialogue of life-cycle support.
---
paper_title: Reasoning about uncertainty in prognosis: a confidence prediction neural network approach

paper_content:

Uncertainty representation and management is the Achilles heel of fault prognosis in condition based management systems. Long-term prediction of the time to failure of critical military and industrial systems entails large-grain uncertainty that must be represented effectively and managed efficiently, i.e. as more data becomes available, means must be devised to narrow or "shrink" the uncertainty bounds. Prediction accuracy and precision are typical performance metrics employed to access the performance of prognostic algorithms. That is, we would like the predicted time to failure to be as close as possible to the real one. Also, the bounds or limits of uncertainty must be as "narrow" as possible. This paper introduces a novel confidence prediction neural network construct with a confidence distribution node based on a Parzen estimate to represent uncertainty and a learning algorithm implemented as a lazy or Q-learning routine that improves online prognostics estimates. The approach is illustrated with test and simulation results obtained from a faulty helicopter planetary gear plate.
---
paper_title: Application of the wavelet transform in machine condition monitoring and fault diagnostics: a review with bibliography

paper_content:

Abstract The application of the wavelet transform for machine fault diagnostics has been developed for last 10 years at a very rapid rate. A review on all of the literature is certainly not possible. The purpose of this review is to present a summary about the application of the wavelet in machine fault diagnostics, including the following main aspects: the time–frequency analysis of signals, the fault feature extraction, the singularity detection for signals, the denoising and extraction of the weak signals, the compression of vibration signals and the system identification. Some other applications are introduced briefly as well, such as the wavelet networks, the wavelet-based frequency response function, etc. In addition, some problems in using the wavelet for machine fault diagnostics are analysed. The prospects of the wavelet analysis in solving non-linear problems are discussed.
---
paper_title: A review on machinery diagnostics and prognostics implementing condition-based maintenance

paper_content:

Condition-based maintenance (CBM) is a maintenance program that recommends maintenance decisions based on the information collected through condition monitoring. It consists of three main steps: data acquisition, data processing and maintenance decision-making. Diagnostics and prognostics are two important aspects of a CBM program. Research in the CBM area grows rapidly. Hundreds of papers in this area, including theory and practical applications, appear every year in academic journals, conference proceedings and technical reports. This paper attempts to summarise and review the recent research and developments in diagnostics and prognostics of mechanical systems implementing CBM with emphasis on models, algorithms and technologies for data processing and maintenance decision-making. Realising the increasing trend of using multiple sensors in condition monitoring, the authors also discuss different techniques for multiple sensor data fusion. The paper concludes with a brief discussion on current practices and possible future trends of CBM.
---
paper_title: Neuro-fuzzy pattern classification for fault diagnosis in nuclear components

paper_content:

Abstract The classification of patterns of measured quantities for diagnostic purposes is an important area of research with practical applications in a variety of fields ranging from industrial to medical. In particular, the classification of faults in nuclear components represents a fundamental task for the operation, control and accident management of nuclear power plants. In this paper, the fault diagnostic problem is tackled within a neuro-fuzzy approach to pattern classification. An important practical issue for the applicability of any diagnostic tool is the transparency of the underlying classification model, to allow for physical interpretation of the relationships between the underlying variables and for direct inspection for validation purposes. In this respect, the proposed neuro-fuzzy approach aims at obtaining not only a high rate of correct classification but also a transparent classification model, i.e., readable and easily interpretable from the physical point of view. For this reason, appropriate coverage and distinguishability constraints on the fuzzy input partitioning interface are introduced. The approach is tested on a diagnostic task concerning faults in the gland seals of the pumps of the primary heat transport system of the CANDU 6 nuclear reactor.
---
paper_title: Intelligent condition monitoring and prognostics system based on data-fusion strategy

paper_content:

This paper proposes an intelligent condition monitoring and prognostics system in condition-based maintenance architecture based on data-fusion strategy. Firstly, vibration signals are collected and trend features are extracted. Then features are normalized and sent into neural network for feature-level fusion. Next, data de-noising is conducted containing smoothing and wavelet decomposition to reduce the fluctuation and pick out trend information. The processed information is used for autonomic health degradation monitoring and data-driven prognostics. When the degradation curve crosses through the specified threshold of alarm, prognostics module is triggered and time-series prediction is performed using multi-nonlinear regression models. Furthermore, the predicted point estimate and interval estimate are fused, respectively. Finally, remaining useful life of operating machine, with its uncertainty interval, are assessed. The proposed system is evaluated by an experiment of health degradation monitoring and prognostics for a methane compressor. The experiment results show that the enhanced maintenance performances can be obtained, which make it suitable for advanced industry maintenance.
---
paper_title: A data-model-fusion prognostic framework for dynamic system state forecasting

paper_content:

A novel data-model-fusion prognostic framework is developed in this paper to improve the accuracy of system state long-horizon forecasting. This framework strategically integrates the strengths of the data-driven prognostic method and the model-based particle filtering approach in system state prediction while alleviating their limitations. In the proposed methodology, particle filtering is applied for system state estimation in parallel with parameter identification of the prediction model (with unknown parameters) based on Bayesian learning. Simultaneously, a data-driven predictor is employed to learn the system degradation pattern from history data so as to predict system evolution (or future measurements). An innovative feature of the proposed fusion prognostic framework is that the predicted measurements (with uncertainties) from the data-driven predictor will be properly managed and utilized by the particle filtering to further update the prediction model parameters, thereby enabling markedly better prognosis as well as improved forecasting transparency. As an application example, the developed fusion prognostic framework is employed to predict the remaining useful life of lithium ion batteries through electrochemical impedance spectroscopy tests. The investigation results demonstrate that the proposed fusion prognostic framework is an effective forecasting tool that can integrate the strengths of both the data-driven method and the particle filtering approach to achieve more accurate state forecasting.
---
paper_title: Web-Based and Model-Driven Decision Support Systems: Concepts and Issues

paper_content:

New technologies, especially the World-Wide Web technologies, have created many opportunities for research about Decision Support Systems. This paper reviews key concepts and technical issues. The technology of DSS is evolving rapidly and academic researchers need to catch-up with practitioners who are implementing innovative DSS. The paper briefly mentions a number of Web sites with innovative DSS that highlight current developments. Much remains however to be investigated and studied if DSS are to contribute to a company’s success.
---
paper_title: A survey of machine learning for big data processing

paper_content:

There is no doubt that big data are now rapidly expanding in all science and engineering domains. While the potential of these massive data is undoubtedly significant, fully making sense of them requires new ways of thinking and novel learning techniques to address the various challenges. In this paper, we present a literature survey of the latest advances in researches on machine learning for big data processing. First, we review the machine learning techniques and highlight some promising learning methods in recent studies, such as representation learning, deep learning, distributed and parallel learning, transfer learning, active learning, and kernel-based learning. Next, we focus on the analysis and discussions about the challenges and possible solutions of machine learning for big data. Following that, we investigate the close connections of machine learning with signal processing techniques for big data processing. Finally, we outline several open issues and research trends.
---
paper_title: Self-maintenance and engineering immune systems: Towards smarter machines and manufacturing systems

paper_content:

Abstract This paper discusses the state-of-the-art research in the areas of self-maintenance and engineering immune systems (EIS) for machines with smarter adaptability to operating regime changes in future manufacturing systems. Inspired by the biological immune and nervous systems, the authors are introducing the transformation of prognostics and health management (PHM) to engineering immune systems (EIS). First, an overview on PHM is introduced. Its transformation toward resilient systems, self-maintenance systems, and engineering immune systems is also discussed. Finally, new concepts in developing future biological-based smarter machines based on autonomic computing and cloud computing are discussed.
---
paper_title: A Data-Driven Failure Prognostics Method Based on Mixture of Gaussians Hidden Markov Models

paper_content:

This paper addresses a data-driven prognostics method for the estimation of the Remaining Useful Life (RUL) and the associated confidence value of bearings. The proposed method is based on the utilization of the Wavelet Packet Decomposition (WPD) technique, and the Mixture of Gaussians Hidden Markov Models (MoG-HMM). The method relies on two phases: an off-line phase, and an on-line phase. During the first phase, the raw data provided by the sensors are first processed to extract features in the form of WPD coefficients. The extracted features are then fed to dedicated learning algorithms to estimate the parameters of a corresponding MoG-HMM, which best fits the degradation phenomenon. The generated model is exploited during the second phase to continuously assess the current health state of the physical component, and to estimate its RUL value with the associated confidence. The developed method is tested on benchmark data taken from the “NASA prognostics data repository” related to several experiments of failures on bearings done under different operating conditions. Furthermore, the method is compared to traditional time-feature prognostics and simulation results are given at the end of the paper. The results of the developed prognostics method, particularly the estimation of the RUL, can help improving the availability, reliability, and security while reducing the maintenance costs. Indeed, the RUL and associated confidence value are relevant information which can be used to take appropriate maintenance and exploitation decisions. In practice, this information may help the maintainers to prepare the necessary material and human resources before the occurrence of a failure. Thus, the traditional maintenance policies involving corrective and preventive maintenance can be replaced by condition based maintenance.
---
paper_title: Benefits and Challenges of System Prognostics

paper_content:

Prognostics is an engineering discipline utilizing in-situ monitoring and analysis to assess system degradation trends, and determine remaining useful life. This paper discusses the benefits of prognostics in terms of system life-cycle processes, such as design and development, production, operations, logistics support, and maintenance. Challenges for prognostics technologies from the viewpoint of both system designers and users will be addressed. These challenges include implementing optimum sensor systems and settings, selecting applicable prognostics methods, addressing prognostic uncertainties, and estimating the cost-benefit implications of prognostics implementation. The research opportunities are summarized as well.
---
paper_title: An empirical evaluation of deep architectures on problems with many factors of variation

paper_content:

Recently, several learning algorithms relying on models with deep architectures have been proposed. Though they have demonstrated impressive performance, to date, they have only been evaluated on relatively simple problems such as digit recognition in a controlled environment, for which many machine learning algorithms already report reasonable results. Here, we present a series of experiments which indicate that these models show promise in solving harder learning problems that exhibit many factors of variation. These models are compared with well-established algorithms such as Support Vector Machines and single hidden-layer feed-forward neural networks.
---
paper_title: Efficient learning of sparse representations with an energy-based model

paper_content:

We describe a novel unsupervised method for learning sparse, overcomplete features. The model uses a linear encoder, and a linear decoder preceded by a sparsifying non-linearity that turns a code vector into a quasi-binary sparse code vector. Given an input, the optimal code minimizes the distance between the output of the decoder and the input patch while being as similar as possible to the encoder output. Learning proceeds in a two-phase EM-like fashion: (1) compute the minimum-energy code vector, (2) adjust the parameters of the encoder and decoder so as to decrease the energy. The model produces "stroke detectors" when trained on handwritten numerals, and Gabor-like filters when trained on natural image patches. Inference and learning are very fast, requiring no preprocessing, and no expensive sampling. Using the proposed unsupervised method to initialize the first layer of a convolutional network, we achieved an error rate slightly lower than the best reported result on the MNIST dataset. Finally, an extension of the method is described to learn topographical filter maps.
---
paper_title: A review of unsupervised feature learning and deep learning for time-series modeling

paper_content:

This paper gives a review of the recent developments in deep learning and unsupervised feature learning for time-series problems. While these techniques have shown promise for modeling static data, such as computer vision, applying them to time-series data is gaining increasing attention. This paper overviews the particular challenges present in time-series data and provides a review of the works that have either applied time-series data to unsupervised feature learning algorithms or alternatively have contributed to modifications of feature learning algorithms to take into account the challenges present in time-series data.
---
paper_title: Deep Learning in Neural Networks: An Overview

paper_content:

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.
---
paper_title: Sequence to Sequence Learning with Neural Networks

paper_content:

Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this paper, we present a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure. Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector. Our main result is that on an English to French translation task from the WMT-14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did not have difficulty on long sentences. For comparison, a phrase-based SMT system achieves a BLEU score of 33.3 on the same dataset. When we used the LSTM to rerank the 1000 hypotheses produced by the aforementioned SMT system, its BLEU score increases to 36.5, which is close to the previous state of the art. The LSTM also learned sensible phrase and sentence representations that are sensitive to word order and are relatively invariant to the active and the passive voice. Finally, we found that reversing the order of the words in all source sentences (but not target sentences) improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
---
paper_title: Circuit Complexity before the Dawn of the New Millennium

paper_content:

The 1980's saw rapid and exciting development of techniques for proving lower bounds in circuit complexity. This pace has slowed recently, and there has even been work indicating that quite different proof techniques must be employed to advance beyond the current frontier of circuit lower bounds. Although this has engendered pessimism in some quarters, there have in fact been many positive developments in the past few years showing that significant progress is possible on many fronts. This paper is a (necessarily incomplete) survey of the state of circuit complexity as we await the dawn of the new millennium.
---
paper_title: Signal Processing Techniques Restructure The Big Data Era

paper_content:

Big data science has been developed into a topic that attracts attention from industry, academia and governments. The main objective in Big Data science is to recognize and extract meaningful information from huge amounts of heterogeneous data and unstructured data (which constitute 95% of big data). Signal Processing (SP) techniques and related statistical learning (SL) tools such as Principal Component Analysis (PCA), R-PCA (Robust PCA), Compressive Sampling (CS), convex optimization (CO), stochastic approximation (SA), kernel based learning (KBL) tasks are used for robustness, compression and dimensionality reduction in Big Data arising challenges. This review paper introduces Big Data related SP techniques and presents applications of this emerging field.
---
paper_title: Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]

paper_content:

This article provides an overview of the mainstream deep learning approaches and research directions proposed over the past decade. It is important to emphasize that each approach has strengths and "weaknesses, depending on the application and context in "which it is being used. Thus, this article presents a summary on the current state of the deep machine learning field and some perspective into how it may evolve. Convolutional Neural Networks (CNNs) and Deep Belief Networks (DBNs) (and their respective variations) are focused on primarily because they are well established in the deep learning field and show great promise for future work.
---
paper_title: Big Data Deep Learning: Challenges and Perspectives

paper_content:

Deep learning is currently an extremely active research area in machine learning and pattern recognition society. It has gained huge successes in a broad area of applications such as speech recognition, computer vision, and natural language processing. With the sheer size of data available today, big data brings big opportunities and transformative potential for various sectors; on the other hand, it also presents unprecedented challenges to harnessing data and information. As the data keeps getting bigger, deep learning is coming to play a key role in providing big data predictive analytics solutions. In this paper, we provide a brief overview of deep learning, and highlight current research efforts and the challenges to big data, as well as the future trends.
---
paper_title: Learning Deep Architectures for AI

paper_content:

Can machine learning deliver AI? Theoretical results, inspiration from the brain and cognition, as well as machine learning experiments suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one would need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers, graphical models with many levels of latent variables, or in complicated propositional formulae re-using many sub-formulae. Each level of the architecture represents features at a different level of abstraction, defined as a composition of lower-level features. Searching the parameter space of deep architectures is a difficult task, but new algorithms have been discovered and a new sub-area has emerged in the machine learning community since 2006, following these discoveries. Learning algorithms such as those for Deep Belief Networks and other related unsupervised learning algorithms have recently been proposed to train deep architectures, yielding exciting results and beating the state-of-the-art in certain areas. Learning Deep Architectures for AI discusses the motivations for and principles of learning algorithms for deep architectures. By analyzing and comparing recent results with different learning algorithms for deep architectures, explanations for their success are proposed and discussed, highlighting challenges and suggesting avenues for future explorations in this area.
---
paper_title: Computer science: The learning machines

paper_content:

Using massive amounts of data to recognize photos and speech, deep-learning computers are taking a big step towards true artificial intelligence.
---
paper_title: Failure diagnosis using deep belief learning based health state classification

paper_content:

Effective health diagnosis provides multifarious benefits such as improved safety, improved reliability and reduced costs for operation and maintenance of complex engineered systems. This paper presents a novel multi-sensor health diagnosis method using deep belief network (DBN). DBN has recently become a popular approach in machine learning for its promised advantages such as fast inference and the ability to encode richer and higher order network structures. The DBN employs a hierarchical structure with multiple stacked restricted Boltzmann machines and works through a layer by layer successive learning process. The proposed multi-sensor health diagnosis methodology using DBN based state classification can be structured in three consecutive stages: first, defining health states and preprocessing sensory data for DBN training and testing; second, developing DBN based classification models for diagnosis of predefined health states; third, validating DBN classification models with testing sensory dataset. Health diagnosis using DBN based health state classification technique is compared with four existing diagnosis techniques. Benchmark classification problems and two engineering health diagnosis applications: aircraft engine health diagnosis and electric power transformer health diagnosis are employed to demonstrate the efficacy of the proposed approach.
---
paper_title: Hierarchical Bayesian inference in the visual cortex

paper_content:

Traditional views of visual processing suggest that early visual neurons in areas V1 and V2 are static spatiotemporal filters that extract local features from a visual scene. The extracted information is then channeled through a feedforward chain of modules in successively higher visual areas for further analysis. Recent electrophysiological recordings from early visual neurons in awake behaving monkeys reveal that there are many levels of complexity in the information processing of the early visual cortex, as seen in the long-latency responses of its neurons. These new findings suggest that activity in the early visual cortex is tightly coupled and highly interactive with the rest of the visual system. They lead us to propose a new theoretical setting based on the mathematical framework of hierarchical Bayesian inference for reasoning about the visual system. In this framework, the recurrent feedforward/feedback loops in the cortex serve to integrate top-down contextual priors and bottom-up observations so as to implement concurrent probabilistic inference along the visual hierarchy. We suggest that the algorithms of particle filtering and Bayesian-belief propagation might model these interactive cortical computations. We review some recent neurophysiological evidences that support the plausibility of these ideas.
---
paper_title: A survey of machine learning for big data processing

paper_content:

There is no doubt that big data are now rapidly expanding in all science and engineering domains. While the potential of these massive data is undoubtedly significant, fully making sense of them requires new ways of thinking and novel learning techniques to address the various challenges. In this paper, we present a literature survey of the latest advances in researches on machine learning for big data processing. First, we review the machine learning techniques and highlight some promising learning methods in recent studies, such as representation learning, deep learning, distributed and parallel learning, transfer learning, active learning, and kernel-based learning. Next, we focus on the analysis and discussions about the challenges and possible solutions of machine learning for big data. Following that, we investigate the close connections of machine learning with signal processing techniques for big data processing. Finally, we outline several open issues and research trends.
---
paper_title: An introduction to hidden Markov models

paper_content:

The basic theory of Markov chains has been known to mathematicians and engineers for close to 80 years, but it is only in the past decade that it has been applied explicitly to problems in speech processing. One of the major reasons why speech models, based on Markov chains, have not been developed until recently was the lack of a method for optimizing the parameters of the Markov model to match observed signal patterns. Such a method was proposed in the late 1960's and was immediately applied to speech processing in several research institutions. Continued refinements in the theory and implementation of Markov modelling techniques have greatly enhanced the method, leading to a wide range of applications of these models. It is the purpose of this tutorial paper to give an introduction to the theory of Markov models, and to illustrate how they have been applied to problems in speech recognition.
---
paper_title: Computational limitations of small-depth circuits

paper_content:

Proving lower bounds on the amount of resources needed to compute specific functions is one of the most active branches of theoretical computer science. Significant progress has been made recently in proving lower bounds in two restricted models of Boolean circuits. One is the model of small depth circuits, and in this book Johan Torkel Hastad has developed very powerful techniques for proving exponential lower bounds on the size of small depth circuits' computing functions.The techniques described in "Computational Limitations for Small Depth Circuits" can be used to demonstrate almost optimal lower bounds on the size of small depth circuits computing several different functions, such as parity and majority. The main tool used in the proof of the lower bounds is a lemma, stating that any AND of small fanout OR gates can be converted into an OR of small fanout AND gates with high probability when random values are substituted for the variables.Hastad also applies this tool to relativized complexity, and discusses in great detail the computation of parity and majority in small depth circuits.Contents: Introduction. Small Depth Circuits. Outline of Lower Bound Proofs. Main Lemma. Lower Bounds for Small Depth Circuits. Functions Requiring Depth k to Have Small Circuits. Applications to Relativized Complexity. How Well Can We Compute Parity in Small Depth? Is Majority Harder than Parity? Conclusions.John Hastad is a postdoctoral fellow in the Department of Mathematics at MIT C"omputational Limitations of Small Depth Circuits" is a winner of the 1986 ACM Doctoral Dissertation Award.
---
paper_title: Deep Learning and Its Applications to Signal and Information Processing

paper_content:

INTRODUCTION TO DEEP LEARNING Many traditional machine learning and signal processing techniques exploit shallow architectures, which contain a single layer of nonlinear feature transformation. Examples of shallow architectures are conventional hidden Markov models (HMMs), linear or nonlinear dynamical systems, conditional random fields (CRFs), maximum entropy (MaxEnt) models, support vector machines (SVMs), kernel regression, and multilayer perceptron (MLP) with a single hidden layer. A property common to these shallow learning models is the simple architecture that consists of only one layer responsible for transforming the raw input signals or features into a problem-specific feature space, which may be unobservable. Take the example of a support vector machine. It is a shallow linear separation model with one feature transformation layer when kernel trick is used, and with zero feature transformation layer when kernel trick is not used. Human information processing
---
paper_title: Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life

paper_content:

Prognostics technique aims to accurately estimate the Remaining Useful Life (RUL) of a subsystem or a component using sensor data, which has many real world applications. However, many of the existing algorithms are based on linear models, which cannot capture the complex relationship between the sensor data and RUL. Although Multilayer Perceptron (MLP) has been applied to predict RUL, it cannot learn salient features automatically, because of its network structure. A novel deep Convolutional Neural Network (CNN) based regression approach for estimating the RUL is proposed in this paper. Although CNN has been applied on tasks such as computer vision, natural language processing, speech recognition etc., this is the first attempt to adopt CNN for RUL estimation in prognostics. Different from the existing CNN structure for computer vision, the convolution and pooling filters in our approach are applied along the temporal dimension over the multi-channel sensor data to incorporate automated feature learning from raw sensor signals in a systematic way. Through the deep architecture, the learned features are the higher-level abstract representation of low-level raw sensor signals. Furthermore, feature learning and RUL estimation are mutually enhanced by the supervised feedback. We compared with several state-of-the-art algorithms on two publicly available data sets to evaluate the effectiveness of this proposed approach. The encouraging results demonstrate that our proposed deep convolutional neural network based regression approach for RUL estimation is not only more efficient but also more accurate.
---
paper_title: Representation Learning: A Review and New Perspectives

paper_content:

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks. This motivates longer term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation, and manifold learning.
---
paper_title: The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training ∗

paper_content:

Whereas theoretical work suggests that deep architectures might be more e cient at representing highly-varying functions, training deep architectures was unsuccessful until the recent advent of algorithms based on unsupervised pretraining. Even though these new algorithms have enabled training deep models, many questions remain as to the nature of this di cult learning problem. Answering these questions is important if learning in deep architectures is to be further improved. We attempt to shed some light on these questions through extensive simulations. The experiments confirm and clarify the advantage of unsupervised pre-training. They demonstrate the robustness of the training procedure with respect to the random initialization, the positive e ect of pre-training in terms of optimization and its role as a regularizer. We empirically show the influence of pre-training with respect to architecture depth, model capacity, and number of training examples.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: A Fast Learning Algorithm for Deep Belief Nets

paper_content:

We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.
---
paper_title: Support-vector networks

paper_content:

Thesupport-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data.High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.
---
paper_title: A survey of deep neural network architectures and their applications

paper_content:

This work was supported in part the Royal Society of the UK, the National Natural Science Foundation of China under Grants 61329301, 61374010, and 61403319, and the Alexander von Humboldt Foundation of Germany.
---
paper_title: Deep learning

paper_content:

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
---
paper_title: Prognostic and health management for engineering systems: a review of the data-driven approach and algorithms

paper_content:

Prognostics and health management (PHM) has become an important component of many engineering systems and products, where algorithms are used to detect anomalies, diagnose faults and predict remaining useful lifetime (RUL). PHM can provide many advantages to users and maintainers. Although primary goals are to ensure the safety, provide state of the health and estimate RUL of the components and systems, there are also financial benefits such as operational and maintenance cost reductions and extended lifetime. This study aims at reviewing the current status of algorithms and methods used to underpin different existing PHM approaches. The focus is on providing a structured and comprehensive classification of the existing state-of-the-art PHM approaches, data-driven approaches and algorithms.
---
paper_title: The role of the primary visual cortex in higher level vision

paper_content:

In the classical feed-forward, modular view of visual processing, the primary visual cortex (area V1) is a module that serves to extract local features such as edges and bars. Representation and recognition of objects are thought to be functions of higher extrastriate cortical areas. This paper presents neurophysiological data that show the later part of V1 neurons’ responses reflecting higher order perceptual computations related to Ullman’s (Cognition 1984;18:97‐159) visual routines and Marr’s (Vision NJ: Freeman 1982) full primal sketch, 2 1 D sketch and 3D model. Based on theoretical reasoning and the experimental evidence, we propose a possible reinterpretation of the functional role of V1. In this framework, because of V1 neurons’ precise encoding of orientation and spatial information, higher level perceptual computations and representations that involve high resolution details, fine geometry and spatial precision would necessarily involve V1 and be reflected in the later part of its neurons’ activities. © 1998 Elsevier Science Ltd. All rights reserved.
---
paper_title: Autonomous ships on the high seas

paper_content:

It's midnight on the North Atlantic, where a massive container ship receives the latest weather report. There's a nasty storm brewing ahead. Quietly, the ship changes course and speed, to skirt the worst of it and ensure an on-time arrival at its destination. The ship's owners and the harbormaster at its next port of call are advised of the revised route. And as it nears shore, the giant ship must correct course once again, this time to steer clear of a fishing vessel off its starboard bow.
---
paper_title: Towards autonomous maritime operations

paper_content:

The main purpose to push the development towards autonomous maritime operations in shipping and offshore installations, is to increase the performance of maritime activities; by social benefits for staff or other related personal groups; by economic benefits when the ability to increase the cargo or effectuate better space allocation; by environmental benefits that's allow the ship operations to be optimized for routing, speed, etc.; and finally by an increased safety benefit in all these aspects. The increased use of artificial intelligent based benefits is expected to increase the operational performance of all the above aspects in the sense that an overall quality in logical and knowledge based will be consistent. The increased complexity of maritime activities and the offshore business requires more precise and automated solutions to achieve the expectations from staff, stakeholders, and society.
---
paper_title: System health monitoring and prognostics — a review of current paradigms and practices

paper_content:

System health monitoring is a set of activities performed on a system to maintain it in operable condition. Monitoring may be limited to the observation of current system states, with maintenance and repair actions prompted by these observations. Alternatively, monitoring of current system states is being augmented with prediction of future operating states and predictive diagnosis of future failure states.Such predictive diagnosis or prognosis is motivated by the need for manufacturers and other operators of complex systems to optimize equipment performance and reduce costs and unscheduled downtime. Prognosis is a difficult task requiring precise, adaptive and intuitive models to predict future machine health states. Numerous modeling techniques have been proposed in the literature and implemented in practice. This paper reviews the philosophies and techniques that focus on improving reliability and reducing unscheduled downtime by monitoring and predicting machine health.
---
paper_title: Towards Design of Prognostics and Health Management Solutions for Maritime Assets

paper_content:

Abstract With increase in competition between OEMs of maritime assets and operators alike, the need to maximize the productivity of an equipment and increase operational efficiency and reliability is increasingly stringent and challenging. Also, with the adoption of availability contracts, maritime OEMs are becoming directly interested in understanding the health of their assets in order to maximize profits and to minimize the risk of a system's failure. The key to address these challenges and needs is performance optimization. For this to be possible it is important to understand that system failure can induce downtime which will increase the total cost of ownership, therefore it is important by all means to minimize unscheduled maintenance. If the state of health or condition of a system, subsystem or component is known, condition-based maintenance can be carried out and system design optimization can be achieved thereby reducing total cost of ownership. With the increasing competition with regards to the maritime industry, it is important that the state of health of a component/sub-system/system/asset is known before a vessel embarks on a mission. Any breakdown or malfunction in any part of any system or subsystem on board vessel during the operation offshore will lead to large economic losses and sometimes cause accidents. For example, damages to the fuel oil system of vessel's main engine can result in huge downtime as a result of the vessel not being in operation. This paper presents a prognostic and health management (PHM) development process applied on a fuel oil system powering diesel engines typically used in various cruise and fishing vessels, dredgers, pipe laying vessels and large oil tankers. This process will hopefully enable future PHM solutions for maritime assets to be designed in a more formal and systematic way.
---
paper_title: Autonomous ships on the high seas

paper_content:

It's midnight on the North Atlantic, where a massive container ship receives the latest weather report. There's a nasty storm brewing ahead. Quietly, the ship changes course and speed, to skirt the worst of it and ensure an on-time arrival at its destination. The ship's owners and the harbormaster at its next port of call are advised of the revised route. And as it nears shore, the giant ship must correct course once again, this time to steer clear of a fishing vessel off its starboard bow.
---
paper_title: Enhanced Restricted Boltzmann Machine With Prognosability Regularization for Prognostics and Health Assessment

paper_content:

In the Internet-of-Things environment, it is critical to bridge the gap between business decision-making and real-time factory data to let companies transfer from condition-based maintenance service to predictive maintenance service. Condition monitoring systems have been widely applied to many industries to acquire operation and equipment related data, through which machine health state can be evaluated. One of the challenges of predicting future machine health lies in extracting the right features that are correlated well with the fault progression/degradation. We propose an enhanced restricted Boltzmann machine with a novel regularization term to automatically generate features that are suitable for remaining useful life prediction. The regularization term tries to maximize the trendability of the output features, which potentially better represent the degradation pattern of a system. The proposed method is benchmarked with regular restricted Boltzmann machine algorithm and principal component analysis. The generated features are used as input to a similarity-based method for life prediction. Run-to-failure datasets collected from two rotating systems are used for validation.
---
paper_title: A review of unsupervised feature learning and deep learning for time-series modeling

paper_content:

This paper gives a review of the recent developments in deep learning and unsupervised feature learning for time-series problems. While these techniques have shown promise for modeling static data, such as computer vision, applying them to time-series data is gaining increasing attention. This paper overviews the particular challenges present in time-series data and provides a review of the works that have either applied time-series data to unsupervised feature learning algorithms or alternatively have contributed to modifications of feature learning algorithms to take into account the challenges present in time-series data.
---
paper_title: Learning Deep Architectures for AI

paper_content:

Can machine learning deliver AI? Theoretical results, inspiration from the brain and cognition, as well as machine learning experiments suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one would need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers, graphical models with many levels of latent variables, or in complicated propositional formulae re-using many sub-formulae. Each level of the architecture represents features at a different level of abstraction, defined as a composition of lower-level features. Searching the parameter space of deep architectures is a difficult task, but new algorithms have been discovered and a new sub-area has emerged in the machine learning community since 2006, following these discoveries. Learning algorithms such as those for Deep Belief Networks and other related unsupervised learning algorithms have recently been proposed to train deep architectures, yielding exciting results and beating the state-of-the-art in certain areas. Learning Deep Architectures for AI discusses the motivations for and principles of learning algorithms for deep architectures. By analyzing and comparing recent results with different learning algorithms for deep architectures, explanations for their success are proposed and discussed, highlighting challenges and suggesting avenues for future explorations in this area.
---
paper_title: Data-driven approaches in health condition monitoring — A comparative study

paper_content:

In this paper, four data-driven classification approaches, that is, K-nearest neighbors (K-NN), self-organizing map (SOM), multi-layer perceptron (MLP), and Bayesian Network classifier (BNC), are applied to a health condition monitoring problem for the wearing cutter. The dataset is produced from a cutting machine using force sensing. A genetic algorithm (GA) based search is performed to select 3 dominant features from a 16-dimensional feature space, which is computed directly from the real dataset. Subsequently K-NN, SOM, MLP, and BNC algorithms are trained to predict the wearing status of the cutter, respectively. The suitability of the four data-driven approaches for the health condition monitoring are investigated and compared.
---
paper_title: An approach to fault diagnosis of reciprocating compressor valves using Teager-Kaiser energy operator and deep belief networks

paper_content:

This paper presents an approach to implement vibration, pressure, and current signals for fault diagnosis of the valves in reciprocating compressors. Due to the complexity of structure and motion of such compressor, the acquired vibration signal normally involves transient impacts and noise. This causes the useful information to be corrupted and difficulty in accurately diagnosing the faults with traditional methods. To reveal the fault patterns contained in this signal, the Teager-Kaiser energy operation (TKEO) is proposed to estimate the amplitude envelopes. In case of pressure and current, the random noise is removed by using a denoising method based on wavelet transform. Subsequently, statistical measures are extracted from all signals to represent the characteristics of the valve conditions. In order to classify the faults of compressor valves, a new type of learning architecture for deep generative model called deep belief networks (DBNs) is applied. DBN employs a hierarchical structure with multiple stacked restricted Boltzmann machines (RBMs) and works through a greedy layer-by-layer learning algorithm. In pattern recognition research areas, DBN has proved to be very effective and provided with high performance for binary values. However, for implementing DBN to fault diagnosis where most of signals are real-valued, RBM with Bernoulli hidden units and Gaussian visible units is considered in this study. The proposed approach is validated with the signals from a two-stage reciprocating air compressor under different valve conditions. To confirm the superiority of DBN in fault classification, its performance is compared with that of relevant vector machine and back propagation neuron networks. The achieved accuracy indicates that the proposed approach is highly reliable and applicable in fault diagnosis of industrial reciprocating machinery.
---
paper_title: Fault Diagnosis for Rotating Machinery Using Vibration Measurement Deep Statistical Feature Learning

paper_content:

Fault diagnosis is important for the maintenance of rotating machinery. The detection of faults and fault patterns is a challenging part of machinery fault diagnosis. To tackle this problem, a model for deep statistical feature learning from vibration measurements of rotating machinery is presented in this paper. Vibration sensor signals collected from rotating mechanical systems are represented in the time, frequency, and time-frequency domains, each of which is then used to produce a statistical feature set. For learning statistical features, real-value Gaussian-Bernoulli restricted Boltzmann machines (GRBMs) are stacked to develop a Gaussian-Bernoulli deep Boltzmann machine (GDBM). The suggested approach is applied as a deep statistical feature learning tool for both gearbox and bearing systems. The fault classification performances in experiments using this approach are 95.17% for the gearbox, and 91.75% for the bearing system. The proposed approach is compared to such standard methods as a support vector machine, GRBM and a combination model. In experiments, the best fault classification rate was detected using the proposed model. The results show that deep learning with statistical feature extraction has an essential improvement potential for diagnosing rotating machinery faults.
---
paper_title: Failure diagnosis using deep belief learning based health state classification

paper_content:

Effective health diagnosis provides multifarious benefits such as improved safety, improved reliability and reduced costs for operation and maintenance of complex engineered systems. This paper presents a novel multi-sensor health diagnosis method using deep belief network (DBN). DBN has recently become a popular approach in machine learning for its promised advantages such as fast inference and the ability to encode richer and higher order network structures. The DBN employs a hierarchical structure with multiple stacked restricted Boltzmann machines and works through a layer by layer successive learning process. The proposed multi-sensor health diagnosis methodology using DBN based state classification can be structured in three consecutive stages: first, defining health states and preprocessing sensory data for DBN training and testing; second, developing DBN based classification models for diagnosis of predefined health states; third, validating DBN classification models with testing sensory dataset. Health diagnosis using DBN based health state classification technique is compared with four existing diagnosis techniques. Benchmark classification problems and two engineering health diagnosis applications: aircraft engine health diagnosis and electric power transformer health diagnosis are employed to demonstrate the efficacy of the proposed approach.
---
paper_title: Reducing the Dimensionality of Data with Neural Networks

paper_content:

High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data.
---
paper_title: Using Deep Learning-Based Approach to Predict Remaining Useful Life of Rotating Components

paper_content:

In the age of Internet of Things and Industrial 4.0, prognostic and health management (PHM) systems are used to collect massive real-time data from mechanical equipment. PHM big data has the characteristics of large-volume, diversity, and high-velocity. Effectively mining features from such data and accurately predicting the remaining useful life (RUL) of the rotating components with new advanced methods become issues in PHM. Traditional data driven prognostics is based on shallow learning architectures, requires establishing explicit model equations and much prior knowledge about signal processing techniques and prognostic expertise, and therefore is limited in the age of big data. This paper presents a deep learning-based approach for RUL prediction of rotating components with big data. The presented approach is tested and validated using data collected from a gear test rig and bearing run-to-failure tests and compared with existing PHM methods. The test results show the promising RUL prediction performance of the deep learning-based approach.
---
paper_title: Time Series Prediction Using Restricted Boltzmann Machines and Backpropagation

paper_content:

Abstract Time series prediction appear in many real-world problems, e.g., financial market, signal processing, weather forecasting among others. The underlying models and time series data of those problems are generally complex in a way that reasonable accurate estimation cannot be easily achieved, thus requiring more advanced techniques. Statistical models are the classical approaches for tackling this problem. Many works extended different architectures of Artificial neural networks to work with time series prediction, such as Feedforward, Boltzmann Machines and Deep Belief Network. A Deep Belief Network based on hybridization between Gaussian-Bernoulli Restricted Boltzmann Machine and the Backpropagation algorithm is presented. The hybrid algorithm is tested on three time series databases showing promising results.
---
paper_title: The Difficulty of Training Deep Architectures and the Effect of Unsupervised Pre-Training ∗

paper_content:

Whereas theoretical work suggests that deep architectures might be more e cient at representing highly-varying functions, training deep architectures was unsuccessful until the recent advent of algorithms based on unsupervised pretraining. Even though these new algorithms have enabled training deep models, many questions remain as to the nature of this di cult learning problem. Answering these questions is important if learning in deep architectures is to be further improved. We attempt to shed some light on these questions through extensive simulations. The experiments confirm and clarify the advantage of unsupervised pre-training. They demonstrate the robustness of the training procedure with respect to the random initialization, the positive e ect of pre-training in terms of optimization and its role as a regularizer. We empirically show the influence of pre-training with respect to architecture depth, model capacity, and number of training examples.
---
paper_title: Unsupervised Learning of Distributions on Binary Vectors Using Two Layer Networks

paper_content:

We present a distribution model for binary vectors, called the influence combination model and show how this model can be used as the basis for unsupervised learning algorithms for feature selection. The model can be represented by a particular type of Boltzmann machine with a bipartite graph structure that we call the combination machine. This machine is closely related to the Harmonium model defined by Smolensky. In the first part of the paper we analyze properties of this distribution representation scheme. We show that arbitrary distributions of binary vectors can be approximated by the combination model. We show how the weight vectors in the model can be interpreted as high order correlation patterns among the input bits, and how the combination machine can be used as a mechanism for detecting these patterns. We compare the combination model with the mixture model and with principle component analysis. In the second part of the paper we present two algorithms for learning the combination model from examples. The first learning algorithm is the standard gradient ascent heuristic for computing maximum likelihood estimates for the parameters of the model. Here we give a closed form for this gradient that is significantly easier to compute than the corresponding gradient for the general Boltzmann machine. The second learning algorithm is a greedy method that creates the hidden units and computes their weights one at a time. This method is a variant of projection pursuit density estimation. In the third part of the paper we give experimental results for these learning methods on synthetic data and on natural data of handwritten digit images.
---
paper_title: Multiobjective Deep Belief Networks Ensemble for Remaining Useful Life Estimation in Prognostics

paper_content:

In numerous industrial applications where safety, efficiency, and reliability are among primary concerns, condition-based maintenance (CBM) is often the most effective and reliable maintenance policy. Prognostics, as one of the key enablers of CBM, involves the core task of estimating the remaining useful life (RUL) of the system. Neural networks-based approaches have produced promising results on RUL estimation, although their performances are influenced by handcrafted features and manually specified parameters. In this paper, we propose a multiobjective deep belief networks ensemble (MODBNE) method. MODBNE employs a multiobjective evolutionary algorithm integrated with the traditional DBN training technique to evolve multiple DBNs simultaneously subject to accuracy and diversity as two conflicting objectives. The eventually evolved DBNs are combined to establish an ensemble model used for RUL estimation, where combination weights are optimized via a single-objective differential evolution algorithm using a task-oriented objective function. We evaluate the proposed method on several prognostic benchmarking data sets and also compare it with some existing approaches. Experimental results demonstrate the superiority of our proposed method.
---
paper_title: Analysis of Feature Extracting Ability for Cutting State Monitoring Using Deep Belief Networks

paper_content:

Abstract Information extracting method from numerous measured signals is a critical technique for intelligent manufacturing application to further reduce the manpower cost and improve the productivity and workpiece quality. Manually defining signal features,as the common way, unfortunately will lose most of the information and the performance can’t be guaranteed. In the past few years, machine learning method with deep structure has been the most promising automatic feature extracting method which has made great breakthrough in computer vision and automatic speech recognition. In this paper, deep belief networksareemployed using vibration signal obtained from endmilling to build feature space for cutting states monitoring. Greedy layer-wise strategy is adopted to pre-train the network and standard samples are used for fine-tuning by applying back-propagation method. Comparisons are made with several manually defined features both in time and frequency domain, like MFCC and wavelet method. Different modeling methods are also employed in the research forcomparisons. Resultsshow that the deep learning method has similar ability to characterize the signal for cutting states monitoring compared to those manually defined features. And the modeling accuracy ismuch better thanother traditionalmodeling methods.Furthermore, benefitting fromthe potentialcapability in information fusion, deep learning method would be a promising solution for more complex applications, like tool wear monitoring, machining surface prediction et al.
---
paper_title: Time series prediction for evolutions of complex systems: A deep learning approach

paper_content:

Evolutions of complex systems are expressions of the dynamic and uncertain behaviors and ever-changing causalities of the potential multivariate processes in a system, which exhibit the characteristics of uncertainties, nonlinear dynamics, and aperiodic fluctuation. These characteristics pose a major challenge for predicting the states of evolutions of complex systems accurately. Supervised learning methods in artificial intelligence, such as the artificial neural networks (ANNs) and the support vector machine (SVM), are suitable for nonlinear time series prediction. However, due to suffering from the issue of feature extraction relying on prior knowledge, and the weakness of shallow learning, traditional ANNs and SVM are hard to guarantee the prediction accuracy and stability. In this study, we propose a deep learning approach, which hybridizes a deep belief networks (DBNs) and a nonlinear kernel-based parallel evolutionary SVM (ESVM), to predict evolution states of complex systems in a classification manner. The proposed approach is tested by two case studies regarding the mobile robot navigation and the early-warning-of-risk of harmful algal blooms. Computational results are compared with those of SVM and DBNs approaches, which demonstrate that the proposed approach outperforms them, and can offer an intelligent decision support tool for predicting evolutions of complex systems.
---
paper_title: A Fast Learning Algorithm for Deep Belief Nets

paper_content:

We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.
---
paper_title: Deep Learning: Methods and Applications

paper_content:

This monograph provides an overview of general deep learning methodology and its applications to a variety of signal and information processing tasks. The application areas are chosen with the following three criteria in mind: (1) expertise or knowledge of the authors; (2) the application areas that have already been transformed by the successful use of deep learning technology, such as speech recognition and computer vision; and (3) the application areas that have the potential to be impacted significantly by deep learning and that have been experiencing research growth, including natural language and text processing, information retrieval, and multimodal information processing empowered by multi-task deep learning.
---
paper_title: Time series forecasting using a deep belief network with restricted Boltzmann machines

paper_content:

Abstract Multi-layer perceptron (MLP) and other artificial neural networks (ANNs) have been widely applied to time series forecasting since 1980s. However, for some problems such as initialization and local optima existing in applications, the improvement of ANNs is, and still will be the most interesting study for not only time series forecasting but also other intelligent computing fields. In this study, we propose a method for time series prediction using Hinton and Salakhutdinov׳s deep belief nets (DBN) which are probabilistic generative neural network composed by multiple layers of restricted Boltzmann machine (RBM). We use a 3-layer deep network of RBMs to capture the feature of input space of time series data, and after pretraining of RBMs using their energy functions, gradient descent training, i.e., back-propagation learning algorithm is used for fine-tuning connection weights between “visible layers” and “hidden layers” of RBMs. To decide the sizes of neural networks and the learning rates, Kennedy and Eberhart׳s particle swarm optimization (PSO) is adopted during the training processes. Furthermore, “trend removal”, a preprocessing to the original data, is also approached in the forecasting experiment using CATS benchmark data. Additionally, approximating and short-term prediction of chaotic time series such as Lorenz chaos and logistic map were also applied by the proposed method.
---
paper_title: Research advances in fault diagnosis and prognostic based on deep learning

paper_content:

Aiming to condition based maintenance for complex equipment, numerous intelligent fault diagnosis and prognostic methods based on machine learning have been researched. Compared with the traditional shallow models, which have problems of lacking expression capacity and existing the curse of dimensionality, using deep learning theory can effectively mine characteristics and accurately recognize the health condition. In consequence, fault diagnosis and prognostic based on deep learning have turned into an innovative and promising research field. This paper gives a review of fault diagnosis and prognostic based on deep learning. First of all, a brief introduction to deep learning architecture and the framework of fault diagnosis based on deep learning is described. Second, tracking describes the latest progress of fault diagnosis and prognostic based on deep learning in chronological order. In this section, the deep learning methods used in fault diagnosis and prognostic are discussed, including Deep Neural Network (DNN), Deep Belief Network (DBN) and Convolutional Neural Network (CNN). Then the engineering application fields are summarized, such as mechanical equipment diagnosis, electrical equipment diagnosis, etc. Finally, this paper indicates the potential future research issues in this field.
---
paper_title: Deep Sparse Rectifier Neural Networks

paper_content:

While logistic sigmoid neurons are more biologically plausible than hyperbolic tangent neurons, the latter work better for training multi-layer neural networks. This paper shows that rectifying neurons are an even better model of biological neurons and yield equal or better performance than hyperbolic tangent networks in spite of the hard non-linearity and non-dierentiabil ity
---
paper_title: Damage propagation modeling for aircraft engine run-to-failure simulation

paper_content:

This paper describes how damage propagation can be modeled within the modules of aircraft gas turbine engines. To that end, response surfaces of all sensors are generated via a thermo-dynamical simulation model for the engine as a function of variations of flow and efficiency of the modules of interest. An exponential rate of change for flow and efficiency loss was imposed for each data set, starting at a randomly chosen initial deterioration set point. The rate of change of the flow and efficiency denotes an otherwise unspecified fault with increasingly worsening effect. The rates of change of the faults were constrained to an upper threshold but were otherwise chosen randomly. Damage propagation was allowed to continue until a failure criterion was reached. A health index was defined as the minimum of several superimposed operational margins at any given time instant and the failure criterion is reached when health index reaches zero. Output of the model was the time series (cycles) of sensed measurements typically available from aircraft gas turbine engines. The data generated were used as challenge data for the prognostics and health management (PHM) data competition at PHMpsila08.
---
paper_title: Rectified Linear Units Improve Restricted Boltzmann Machines

paper_content:

Restricted Boltzmann machines were developed using binary stochastic hidden units. These can be generalized by replacing each binary unit by an infinite number of copies that all have the same weights but have progressively more negative biases. The learning and inference rules for these "Stepped Sigmoid Units" are unchanged. They can be approximated efficiently by noisy, rectified linear units. Compared with binary units, these units learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset. Unlike binary units, rectified linear units preserve information about relative intensities as information travels through multiple layers of feature detectors.
---
paper_title: Ensemble deep learning for regression and time series forecasting

paper_content:

In this paper, for the first time, an ensemble of deep learning belief networks (DBN) is proposed for regression and time series forecasting. Another novel contribution is to aggregate the outputs from various DBNs by a support vector regression (SVR) model. We show the advantage of the proposed method on three electricity load demand datasets, one artificial time series dataset and three regression datasets over other benchmark methods.
---
paper_title: An empirical evaluation of deep architectures on problems with many factors of variation

paper_content:

Recently, several learning algorithms relying on models with deep architectures have been proposed. Though they have demonstrated impressive performance, to date, they have only been evaluated on relatively simple problems such as digit recognition in a controlled environment, for which many machine learning algorithms already report reasonable results. Here, we present a series of experiments which indicate that these models show promise in solving harder learning problems that exhibit many factors of variation. These models are compared with well-established algorithms such as Support Vector Machines and single hidden-layer feed-forward neural networks.
---
paper_title: Efficient learning of sparse representations with an energy-based model

paper_content:

We describe a novel unsupervised method for learning sparse, overcomplete features. The model uses a linear encoder, and a linear decoder preceded by a sparsifying non-linearity that turns a code vector into a quasi-binary sparse code vector. Given an input, the optimal code minimizes the distance between the output of the decoder and the input patch while being as similar as possible to the encoder output. Learning proceeds in a two-phase EM-like fashion: (1) compute the minimum-energy code vector, (2) adjust the parameters of the encoder and decoder so as to decrease the energy. The model produces "stroke detectors" when trained on handwritten numerals, and Gabor-like filters when trained on natural image patches. Inference and learning are very fast, requiring no preprocessing, and no expensive sampling. Using the proposed unsupervised method to initialize the first layer of a convolutional network, we achieved an error rate slightly lower than the best reported result on the MNIST dataset. Finally, an extension of the method is described to learn topographical filter maps.
---
paper_title: A review of unsupervised feature learning and deep learning for time-series modeling

paper_content:

This paper gives a review of the recent developments in deep learning and unsupervised feature learning for time-series problems. While these techniques have shown promise for modeling static data, such as computer vision, applying them to time-series data is gaining increasing attention. This paper overviews the particular challenges present in time-series data and provides a review of the works that have either applied time-series data to unsupervised feature learning algorithms or alternatively have contributed to modifications of feature learning algorithms to take into account the challenges present in time-series data.
---
paper_title: Learning Deep Architectures for AI

paper_content:

Can machine learning deliver AI? Theoretical results, inspiration from the brain and cognition, as well as machine learning experiments suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one would need deep architectures. Deep architectures are composed of multiple levels of non-linear operations, such as in neural nets with many hidden layers, graphical models with many levels of latent variables, or in complicated propositional formulae re-using many sub-formulae. Each level of the architecture represents features at a different level of abstraction, defined as a composition of lower-level features. Searching the parameter space of deep architectures is a difficult task, but new algorithms have been discovered and a new sub-area has emerged in the machine learning community since 2006, following these discoveries. Learning algorithms such as those for Deep Belief Networks and other related unsupervised learning algorithms have recently been proposed to train deep architectures, yielding exciting results and beating the state-of-the-art in certain areas. Learning Deep Architectures for AI discusses the motivations for and principles of learning algorithms for deep architectures. By analyzing and comparing recent results with different learning algorithms for deep architectures, explanations for their success are proposed and discussed, highlighting challenges and suggesting avenues for future explorations in this area.
---
paper_title: Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion

paper_content:

We explore an original strategy for building deep networks, based on stacking layers of denoising autoencoders which are trained locally to denoise corrupted versions of their inputs. The resulting algorithm is a straightforward variation on the stacking of ordinary autoencoders. It is however shown on a benchmark of classification problems to yield significantly lower classification error, thus bridging the performance gap with deep belief networks (DBN), and in several cases surpassing it. Higher level representations learnt in this purely unsupervised fashion also help boost the performance of subsequent SVM classifiers. Qualitative experiments show that, contrary to ordinary autoencoders, denoising autoencoders are able to learn Gabor-like edge detectors from natural image patches and larger stroke detectors from digit images. This work clearly establishes the value of using a denoising criterion as a tractable unsupervised objective to guide the learning of useful higher level representations.
---
paper_title: Intelligent fault diagnosis approach with unsupervised feature learning by stacked denoising autoencoder

paper_content:

Condition monitoring and fault diagnosis are important for maintaining the system performance and guaranteeing the operational safety. The traditional data-driven approaches mostly incorporate well-defined features and methodologies such as supervised artificial intelligence algorithms. Prior knowledge of possible features and a large quantity of labelled condition data are needed. Besides, many traditional approaches require rebuilding or a retraining of the original model to diagnosis new conditions. The present study proposes an intelligent fault diagnosis approach that uses a deep neural network (DNN) based on stacked denoising autoencoder. Representative features are learned by applying the denoising autoencoder to the unlabelled data in an unsupervised manner. A DNN is then constructed and fine-tuned with just a few items of labelled data. The trained DNN achieves high performance in fault classification. Furthermore, new conditions can be correctly classified by simply fine-tuning the trained DNN model using a small amount of labelled data under the new conditions. The effectiveness of the proposed approach is evaluated using a case study of fault diagnosis of a bearing unit. The results indicate that the proposed method can extract representative features from massive unlabelled data on the system condition and achieve high performance in fault diagnosis.
---
paper_title: Representation Learning: A Review and New Perspectives

paper_content:

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, autoencoders, manifold learning, and deep networks. This motivates longer term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation, and manifold learning.
---
paper_title: A Fast Learning Algorithm for Deep Belief Nets

paper_content:

We show how to use "complementary priors" to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modeled by long ravines in the free-energy landscape of the top-level associative memory, and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind.
---
paper_title: A sparse auto-encoder-based deep neural network approach for induction motor faults classification

paper_content:

Abstract This paper presents a deep neural network (DNN) approach for induction motor fault diagnosis. The approach utilizes sparse auto-encoder (SAE) to learn features, which belongs to unsupervised feature learning that only requires unlabeled measurement data. With the help of the denoising coding, partial corruption is added into the input of the SAE to improve robustness of feature representation. Features learned from the SAE are then used to train a neural network classifier for identifying induction motor faults. In addition, to prevent overfitting during the training process, a recently developed regularization method called “dropout” which has been proved to be very effective in neural network was employed. An experiment performed on a machine fault simulator indicates that compared with traditional neural network, the SAE-based DNN can achieve superior performance for feature learning and classification in the field of induction motor fault diagnosis.
---
paper_title: A survey of deep neural network architectures and their applications

paper_content:

This work was supported in part the Royal Society of the UK, the National Natural Science Foundation of China under Grants 61329301, 61374010, and 61403319, and the Alexander von Humboldt Foundation of Germany.
---
paper_title: Extracting and composing robust features with denoising autoencoders

paper_content:

Previous work has shown that the difficulties in learning deep generative or discriminative models can be overcome by an initial unsupervised learning step that maps inputs to useful intermediate representations. We introduce and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern. This approach can be used to train autoencoders, and these denoising autoencoders can be stacked to initialize deep architectures. The algorithm can be motivated from a manifold learning and information theoretic perspective or from a generative model perspective. Comparative experiments clearly show the surprising advantage of corrupting the input of autoencoders on a pattern classification benchmark suite.
---
paper_title: Generating feature sets for fault diagnosis using denoising stacked auto-encoder

paper_content:

Recent advancements in sensor technologies and data driven model based techniques have made intelligent diagnostic systems prominent in machine maintenance frameworks of industries. The performance of such systems immensely relies upon the quality of features extracted and the classifier model learned. Traditionally features were handcrafted, where engineers would manually design them with statistical parameters and signal transforms based energy distribution analysis. Recently, deep learning techniques have shown new ways of obtaining useful feature representation that provide state of the art results in image and speech processing applications. This paper first presents a brief survey of traditional handcrafted features and later presents a short analysis of handcrafted features v/s features learned by deep neural networks (DNN), for doing fault diagnosis. The DNN based features in this paper were generated in 3 phases: 1) extracted handcrafted features using traditional techniques 2) initialized the weights of DNN by learning de-noising sparse auto-encoders with the handcrafted features in unsupervised fashion and 3) applied two generic fine tuning heuristics that tailor DNN's weights to give good classification performance. The experimentation and analysis were performed on 5 datasets: one each on Air compressor monitoring, Drill bit monitoring and Steel plate monitoring, and two on bearing fault monitoring data. The results clearly show the prospects of DNN obtaining good feature representations and good classification performance. Further, it also finds that Fast Fourier Transform based features with DNN are more suited for Support Vector Machine as classifier than Random Forest.
---
paper_title: A novel feature extraction method using deep neural network for rolling bearing fault diagnosis

paper_content:

Rolling bearing fault diagnosis has received much attention because of its importance for the rotatory machinery. Feature extraction is the crucial part of rolling bearing fault diagnosis, which determines the diagnosis performance greatly. However, features extracted by many available methods cannot guarantee the sensitiveness to every interested fault category, which leads to incomplete diagnosis results and ability absence of handling with the situation that unknown-category fault appears. To solve this issue, the feature extraction method based on deep neural network (DNN) is proposed to extract a meaningful representation for bearing signal in this article. DNN is a new kind of machine learning tool with strong power of representation, which has been utilized as the feature extractors in lots of practical applications successfully. Afterwards, the effectiveness of this proposed approach is presented by using the actual rolling bearing data.
---
paper_title: Dropout: a simple way to prevent neural networks from overfitting

paper_content:

Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
---
paper_title: Intelligent fault diagnosis approach with unsupervised feature learning by stacked denoising autoencoder

paper_content:

Condition monitoring and fault diagnosis are important for maintaining the system performance and guaranteeing the operational safety. The traditional data-driven approaches mostly incorporate well-defined features and methodologies such as supervised artificial intelligence algorithms. Prior knowledge of possible features and a large quantity of labelled condition data are needed. Besides, many traditional approaches require rebuilding or a retraining of the original model to diagnosis new conditions. The present study proposes an intelligent fault diagnosis approach that uses a deep neural network (DNN) based on stacked denoising autoencoder. Representative features are learned by applying the denoising autoencoder to the unlabelled data in an unsupervised manner. A DNN is then constructed and fine-tuned with just a few items of labelled data. The trained DNN achieves high performance in fault classification. Furthermore, new conditions can be correctly classified by simply fine-tuning the trained DNN model using a small amount of labelled data under the new conditions. The effectiveness of the proposed approach is evaluated using a case study of fault diagnosis of a bearing unit. The results indicate that the proposed method can extract representative features from massive unlabelled data on the system condition and achieve high performance in fault diagnosis.
---
paper_title: Deep neural networks: A promising tool for fault characteristic mining and intelligent diagnosis of rotating machinery with massive data

paper_content:

Abstract Aiming to promptly process the massive fault data and automatically provide accurate diagnosis results, numerous studies have been conducted on intelligent fault diagnosis of rotating machinery. Among these studies, the methods based on artificial neural networks (ANNs) are commonly used, which employ signal processing techniques for extracting features and further input the features to ANNs for classifying faults. Though these methods did work in intelligent fault diagnosis of rotating machinery, they still have two deficiencies. (1) The features are manually extracted depending on much prior knowledge about signal processing techniques and diagnostic expertise. In addition, these manual features are extracted according to a specific diagnosis issue and probably unsuitable for other issues. (2) The ANNs adopted in these methods have shallow architectures, which limits the capacity of ANNs to learn the complex non-linear relationships in fault diagnosis issues. As a breakthrough in artificial intelligence, deep learning holds the potential to overcome the aforementioned deficiencies. Through deep learning, deep neural networks (DNNs) with deep architectures, instead of shallow ones, could be established to mine the useful information from raw data and approximate complex non-linear functions. Based on DNNs, a novel intelligent method is proposed in this paper to overcome the deficiencies of the aforementioned intelligent diagnosis methods. The effectiveness of the proposed method is validated using datasets from rolling element bearings and planetary gearboxes. These datasets contain massive measured signals involving different health conditions under various operating conditions. The diagnosis results show that the proposed method is able to not only adaptively mine available fault characteristics from the measured signals, but also obtain superior diagnosis accuracy compared with the existing methods.
---
paper_title: A sparse auto-encoder-based deep neural network approach for induction motor faults classification

paper_content:

Abstract This paper presents a deep neural network (DNN) approach for induction motor fault diagnosis. The approach utilizes sparse auto-encoder (SAE) to learn features, which belongs to unsupervised feature learning that only requires unlabeled measurement data. With the help of the denoising coding, partial corruption is added into the input of the SAE to improve robustness of feature representation. Features learned from the SAE are then used to train a neural network classifier for identifying induction motor faults. In addition, to prevent overfitting during the training process, a recently developed regularization method called “dropout” which has been proved to be very effective in neural network was employed. An experiment performed on a machine fault simulator indicates that compared with traditional neural network, the SAE-based DNN can achieve superior performance for feature learning and classification in the field of induction motor fault diagnosis.
---
paper_title: LSTM: A Search Space Odyssey

paper_content:

Several variants of the long short-term memory (LSTM) architecture for recurrent neural networks have been proposed since its inception in 1995. In recent years, these networks have become the state-of-the-art models for a variety of machine learning problems. This has led to a renewed interest in understanding the role and utility of various computational components of typical LSTM variants. In this paper, we present the first large-scale analysis of eight LSTM variants on three representative tasks: speech recognition, handwriting recognition, and polyphonic music modeling. The hyperparameters of all LSTM variants for each task were optimized separately using random search, and their importance was assessed using the powerful functional ANalysis Of VAriance framework. In total, we summarize the results of 5400 experimental runs ( $\approx 15$ years of CPU time), which makes our study the largest of its kind on LSTM networks. Our results show that none of the variants can improve upon the standard LSTM architecture significantly, and demonstrate the forget gate and the output activation function to be its most critical components. We further observe that the studied hyperparameters are virtually independent and derive guidelines for their efficient adjustment.
---
paper_title: Learning Long-Term Dependencies with Gradient Descent is Difficult

paper_content:

Recurrent neural networks can be used to map input sequences to output sequences, such as for recognition, production or prediction problems. However, practical difficulties have been reported in training recurrent neural networks to perform tasks in which the temporal contingencies present in the input/output sequences span long intervals. We show why gradient based learning algorithms face an increasingly difficult problem as the duration of the dependencies to be captured increases. These results expose a trade-off between efficient learning by gradient descent and latching on information for long periods. Based on an understanding of this problem, alternatives to standard gradient descent are considered. >
---
paper_title: Framewise phoneme classification with bidirectional LSTM networks

paper_content:

In this paper, we apply bidirectional training to a long short term memory (LSTM) network for the first time. We also present a modified, full gradient version of the LSTM learning algorithm. We discuss the significance of framewise phoneme classification to continuous speech recognition, and the validity of using bidirectional networks for online causal tasks. On the TIMIT speech database, we measure the framewise phoneme classification scores of bidirectional and unidirectional variants of both LSTM and conventional recurrent neural networks (RNNs). We find that bidirectional LSTM outperforms both RNNs and unidirectional LSTM.
---
paper_title: Long Short-Term Memory

paper_content:

Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
---
paper_title: Generating Text with Recurrent Neural Networks

paper_content:

Recurrent Neural Networks (RNNs) are very powerful sequence models that do not enjoy widespread use because it is extremely difficult to train them properly. Fortunately, recent advances in Hessian-free optimization have been able to overcome the difficulties associated with training RNNs, making it possible to apply them successfully to challenging sequence problems. In this paper we demonstrate the power of RNNs trained with the new Hessian-Free optimizer (HF) by applying them to character-level language modeling tasks. The standard RNN architecture, while effective, is not ideally suited for such tasks, so we introduce a new RNN variant that uses multiplicative (or "gated") connections which allow the current input character to determine the transition matrix from one hidden state vector to the next. After training the multiplicative RNN with the HF optimizer for five days on 8 high-end Graphics Processing Units, we were able to surpass the performance of the best previous single method for character-level language modeling – a hierarchical non-parametric sequence model. To our knowledge this represents the largest recurrent neural network application to date.
---
paper_title: Learning to forget: continual prediction with LSTM

paper_content:

Long short-term memory (LSTM) can solve many tasks not solvable by previous learning algorithms for recurrent neural networks (RNNs). We identify a weakness of LSTM networks processing continual input streams without explicitly marked sequence ends. Without resets, the internal state values may grow indefinitely and eventually cause the network to break down. Our remedy is an adaptive "forget gate" that enables an LSTM cell to learn to reset itself at appropriate times, thus releasing internal resources. We review an illustrative benchmark problem on which standard LSTM outperforms other RNN algorithms. All algorithms (including LSTM) fail to solve a continual version of that problem. LSTM with forget gates, however, easily solves it in an elegant way.
---
paper_title: Deep learning

paper_content:

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
---
paper_title: Sequence to Sequence Learning with Neural Networks

paper_content:

Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this paper, we present a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure. Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector. Our main result is that on an English to French translation task from the WMT-14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did not have difficulty on long sentences. For comparison, a phrase-based SMT system achieves a BLEU score of 33.3 on the same dataset. When we used the LSTM to rerank the 1000 hypotheses produced by the aforementioned SMT system, its BLEU score increases to 36.5, which is close to the previous state of the art. The LSTM also learned sensible phrase and sentence representations that are sensitive to word order and are relatively invariant to the active and the passive voice. Finally, we found that reversing the order of the words in all source sentences (but not target sentences) improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
---
paper_title: Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network

paper_content:

Aero engine is a kind of sophisticated and expensive industrial product. Accurate fault location and Remaining Useful Life (RUL) estimation for aero engine can lead to appropriate maintenance actions to avoid catastrophic failures and minimize economic losses. The aim of this paper is to propose utilizing Long Short-Term Memory (LSTM) neural network to get good diagnosis and prediction performance in the cases of complicated operations, hybrid faults and strong noises. The whole proposition is demonstrated and discussed by carrying out tests on a health monitoring dataset of aircraft turbofan engines provided by NASA. Performances of LSTM and some of its modifications were tested and contrasted. Experiment results show the standard LSTM outperformed others.
---
paper_title: Recurrent neural networks for remaining useful life estimation

paper_content:

This paper presents an approach and solution to the IEEE 2008 Prognostics and Health Management conference challenge problem. The solution utilizes an advanced recurrent neural network architecture to estimate the remaining useful life of the system. The recurrent neural network is trained with back-propagation through time gradient calculations, an Extended Kalman Filter training method, and evolutionary algorithms to generate an accurate and compact algorithm. This solution placed second overall in the competition with a very small margin between the first and second place finishers.
---
paper_title: Remaining useful life estimation of engineered systems using vanilla LSTM neural networks

paper_content:

Abstract Long Short-Term Memory (LSTM) networks are a significant branch of Recurrent Neural Networks (RNN), capable of learning long-term dependencies. In recent years, vanilla LSTM (a variation of original LSTM above) has become the state-of-the-art model for a variety of machine learning problems, especially Natural Language Processing (NLP). However, in industry, this powerful Deep Neural Network (DNN) has not aroused wide concern. In research focusing on Prognostics and Health Management (PHM) technology for complex engineered systems, Remaining Useful Life (RUL) estimation is one of the most challenging problems, which can lead to appropriate maintenance actions to be scheduled proactively to avoid catastrophic failures and minimize economic losses of the systems. Following that, this paper aims to propose utilizing vanilla LSTM neural networks to get good RUL prediction accuracy which makes the most of long short-term memory ability, in the cases of complicated operations, working conditions, model degradations and strong noises. In addition, to promote cognition ability about model degradation processes, a dynamic differential technology was proposed to extract inter-frame information. The whole proposition is illustrated and discussed by performing tests on a case of the health monitoring of aircraft turbofan engines which have four different issues. Performances of vanilla LSTM are benchmarked with standard RNN and Gated Recurrent Unit (GRU) LSTM. Results show the significance of performance improvement achieved by vanilla LSTM.
---
paper_title: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

paper_content:

In this paper, we propose a novel neural network model called RNN Encoder‐ Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixedlength vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder‐Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.
---
paper_title: Multi-Sensor Prognostics using an Unsupervised Health Index based on LSTM Encoder-Decoder.

paper_content:

Many approaches for estimation of Remaining Useful Life (RUL) of a machine, using its operational sensor data, make assumptions about how a system degrades or a fault evolves, e.g., exponential degradation. However, in many domains degradation may not follow a pattern. We propose a Long Short Term Memory based Encoder-Decoder (LSTM-ED) scheme to obtain an unsupervised health index (HI) for a system using multi-sensor time-series data. LSTM-ED is trained to reconstruct the time-series corresponding to healthy state of a system. The reconstruction error is used to compute HI which is then used for RUL estimation. We evaluate our approach on publicly available Turbofan Engine and Milling Machine datasets. We also present results on a real-world industry dataset from a pulverizer mill where we find significant correlation between LSTM-ED based HI and maintenance costs.
---
paper_title: Deep Learning for solar power forecasting — An approach using AutoEncoder and LSTM Neural Networks

paper_content:

Power forecasting of renewable energy power plants is a very active research field, as reliable information about the future power generation allow for a safe operation of the power grid and helps to minimize the operational costs of these energy sources. Deep Learning algorithms have shown to be very powerful in forecasting tasks, such as economic time series or speech recognition. Up to now, Deep Learning algorithms have only been applied sparsely for forecasting renewable energy power plants. By using different Deep Learning and Artificial Neural Network algorithms, such as Deep Belief Networks, AutoEncoder, and LSTM, we introduce these powerful algorithms in the field of renewable energy power forecasting. In our experiments, we used combinations of these algorithms to show their forecast strength compared to a standard MLP and a physical forecasting model in the forecasting the energy output of 21 solar power plants. Our results using Deep Learning algorithms show a superior forecasting performance compared to Artificial Neural Networks as well as other reference models such as physical models.
---
paper_title: Long Short-Term Memory Network for Remaining Useful Life estimation

paper_content:

Remaining Useful Life (RUL) of a component or a system is defined as the length from the current time to the end of the useful life. Accurate RUL estimation plays a critical role in Prognostics and Health Management(PHM). Data driven approaches for RUL estimation use sensor data and operational data to estimate RUL. Traditional regression based approaches and recent Convolutional Neural Network (CNN) approach use features created from sliding windows to build models. However, sequence information is not fully considered in these approaches. Sequence learning models such as Hidden Markov Models (HMMs) and Recurrent Neural Networks (RNNs) have flaws when modeling sequence information. HMMs are limited to discrete hidden states and are known to have issues when modeling long-term dependencies in the data. RNNs also have issues with long-term dependencies. In this work, we propose a Long Short-Term Memory (LSTM) approach for RUL estimation, which can make full use of the sensor sequence information and expose hidden patterns within sensor data with multiple operating conditions, fault and degradation models. Extensive experiments using three widely adopted Prognostics and Health Management data sets show that LSTM for RUL estimation significantly outperforms traditional approaches for RUL estimation as well as Convolutional Neural Network (CNN).
---
paper_title: Mechanical state prediction based on LSTM neural netwok

paper_content:

Effective mechanical state prediction systems are critical to modern manufacturing systems and industries. As a method of deep learning algorithm, Recurrent neural network, (RNN) has been playing an increasingly important role in the field of time series prediction. In order to solve the problem of hard training and gradient extinction of RNN model, a long short-term memory network (LSTM) algorithm is proposed and applied to the prediction of mechanical state (PMS). On the basis of the motor bearing data, the simulation is carried out. Aiming at the non-stationary of bearing data, the empirical mode decomposition (EMD) is used to decompose the bearing data into stationary singles, and the intrinsic mode function (IMF) energy entropy is calculated as the feature of mechanical state. In comparison with support vector regression machine (SVRM), LSTM achieves a better result in the single-step prediction of mechanical state. It shows that LSTM is superior in machine state prediction and monitoring.
---
paper_title: Damage propagation modeling for aircraft engine run-to-failure simulation

paper_content:

This paper describes how damage propagation can be modeled within the modules of aircraft gas turbine engines. To that end, response surfaces of all sensors are generated via a thermo-dynamical simulation model for the engine as a function of variations of flow and efficiency of the modules of interest. An exponential rate of change for flow and efficiency loss was imposed for each data set, starting at a randomly chosen initial deterioration set point. The rate of change of the flow and efficiency denotes an otherwise unspecified fault with increasingly worsening effect. The rates of change of the faults were constrained to an upper threshold but were otherwise chosen randomly. Damage propagation was allowed to continue until a failure criterion was reached. A health index was defined as the minimum of several superimposed operational margins at any given time instant and the failure criterion is reached when health index reaches zero. Output of the model was the time series (cycles) of sensed measurements typically available from aircraft gas turbine engines. The data generated were used as challenge data for the prognostics and health management (PHM) data competition at PHMpsila08.
---
paper_title: Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier]

paper_content:

This article provides an overview of the mainstream deep learning approaches and research directions proposed over the past decade. It is important to emphasize that each approach has strengths and "weaknesses, depending on the application and context in "which it is being used. Thus, this article presents a summary on the current state of the deep machine learning field and some perspective into how it may evolve. Convolutional Neural Networks (CNNs) and Deep Belief Networks (DBNs) (and their respective variations) are focused on primarily because they are well established in the deep learning field and show great promise for future work.
---
paper_title: Gradient-Based Learning Applied to Document Recognition

paper_content:

Multilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique. Given an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing. This paper reviews various methods applied to handwritten character recognition and compares them on a standard handwritten digit recognition task. Convolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques. Real-life document recognition systems are composed of multiple modules including field extraction, segmentation recognition, and language modeling. A new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure. Two systems for online handwriting recognition are described. Experiments demonstrate the advantage of global training, and the flexibility of graph transformer networks. A graph transformer network for reading a bank cheque is also described. It uses convolutional neural network character recognizers combined with global training techniques to provide record accuracy on business and personal cheques. It is deployed commercially and reads several million cheques per day.
---
paper_title: Imagenet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overriding in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: Phoneme recognition using time-delay neural networks

paper_content:

The authors present a time-delay neural network (TDNN) approach to phoneme recognition which is characterized by two important properties: (1) using a three-layer arrangement of simple computing units, a hierarchy can be constructed that allows for the formation of arbitrary nonlinear decision surfaces, which the TDNN learns automatically using error backpropagation; and (2) the time-delay arrangement enables the network to discover acoustic-phonetic features and the temporal relationships between them independently of position in time and therefore not blurred by temporal shifts in the input. As a recognition task, the speaker-dependent recognition of the phonemes B, D, and G in varying phonetic contexts was chosen. For comparison, several discrete hidden Markov models (HMM) were trained to perform the same task. Performance evaluation over 1946 testing tokens from three speakers showed that the TDNN achieves a recognition rate of 98.5% correct while the rate obtained by the best of the HMMs was only 93.7%. >
---
paper_title: Adam: A Method for Stochastic Optimization

paper_content:

We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.
---
paper_title: Dropout: a simple way to prevent neural networks from overfitting

paper_content:

Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
---
paper_title: Remaining Useful Life Estimation in Prognostics Using Deep Convolution Neural Networks

paper_content:

Traditionally, system prognostics and health management (PHM) depends on sufficient prior knowledge of critical components degradation process in order to predict the remaining useful life (RUL). However, the accurate physical or expert models are not available in most cases. This paper proposes a new data-driven approach for prognostics using deep convolution neural networks (DCNN). Time window approach is employed for sample preparation in order for better feature extraction by DCNN. Raw collected data with normalization are directly used as inputs to the proposed network, and no prior expertise on prognostics and signal processing is required, that facilitates the application of the proposed method. In order to show the effectiveness of the proposed approach, experiments on the popular C-MAPSS dataset for aero-engine unit prognostics are carried out. High prognostic accuracy on the RUL estimation is achieved. The superiority of the proposed method is demonstrated by comparisons with other popular approaches and the state-of-the-art results on the same dataset. The results of this study suggest that the proposed data-driven prognostic method offers a new and promising approach.
---
paper_title: DeepFace: Closing the Gap to Human-Level Performance in Face Verification

paper_content:

In modern face recognition, the conventional pipeline consists of four stages: detect => align => represent => classify. We revisit both the alignment step and the representation step by employing explicit 3D face modeling in order to apply a piecewise affine transformation, and derive a face representation from a nine-layer deep neural network. This deep network involves more than 120 million parameters using several locally connected layers without weight sharing, rather than the standard convolutional layers. Thus we trained it on the largest facial dataset to-date, an identity labeled dataset of four million facial images belonging to more than 4, 000 identities. The learned representations coupling the accurate model-based alignment with the large facial database generalize remarkably well to faces in unconstrained environments, even with a simple classifier. Our method reaches an accuracy of 97.35% on the Labeled Faces in the Wild (LFW) dataset, reducing the error of the current state of the art by more than 27%, closely approaching human-level performance.
---
paper_title: Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life

paper_content:

Prognostics technique aims to accurately estimate the Remaining Useful Life (RUL) of a subsystem or a component using sensor data, which has many real world applications. However, many of the existing algorithms are based on linear models, which cannot capture the complex relationship between the sensor data and RUL. Although Multilayer Perceptron (MLP) has been applied to predict RUL, it cannot learn salient features automatically, because of its network structure. A novel deep Convolutional Neural Network (CNN) based regression approach for estimating the RUL is proposed in this paper. Although CNN has been applied on tasks such as computer vision, natural language processing, speech recognition etc., this is the first attempt to adopt CNN for RUL estimation in prognostics. Different from the existing CNN structure for computer vision, the convolution and pooling filters in our approach are applied along the temporal dimension over the multi-channel sensor data to incorporate automated feature learning from raw sensor signals in a systematic way. Through the deep architecture, the learned features are the higher-level abstract representation of low-level raw sensor signals. Furthermore, feature learning and RUL estimation are mutually enhanced by the supervised feedback. We compared with several state-of-the-art algorithms on two publicly available data sets to evaluate the effectiveness of this proposed approach. The encouraging results demonstrate that our proposed deep convolutional neural network based regression approach for RUL estimation is not only more efficient but also more accurate.
---
paper_title: Dislocated Time Series Convolutional Neural Architecture: An Intelligent Fault Diagnosis Approach for Electric Machine

paper_content:

In most current intelligent diagnosis methods, fault classifiers of electric machine are built based on complex handcrafted features extractor from raw signals, which depend on prior knowledge and is difficult to implement intelligentization authentically. In addition, the increasingly complicated industrial structures and data make handcrafted features extractors less suited. Convolutional neural network (CNN) provides an efficient method to act on raw signals directly by weight sharing and local connections without feature extractors. However, effective as CNN works on image recognition, it does not work well in industrial applications due to the differences between image and industrial signals. Inspired by the idea of CNN, we develop a novel diagnosis framework based on the characteristics of industrial vibration signals, which is called dislocated time series CNN (DTS-CNN). The DTS-CNN architecture is composed of dislocate layer, convolutional layer, sub-sampling layer and fully connected layer. By adding a dislocate layer, this model can extract the relationship between signals with different intervals in periodic mechanical signals, thereby overcome the weaknesses of traditional CNNs and is more applicable for modern electric machines, especially under nonstationary conditions. Experiments under constant and nonstationary conditions are performed on a machine fault simulator to validate the proposed framework. The results and comparison with respect to the state of the art in the field is illustrated in detail, which highlights the superiority of the proposed method in industrial applications.
---
paper_title: Long Short-Term Memory Network for Remaining Useful Life estimation

paper_content:

Remaining Useful Life (RUL) of a component or a system is defined as the length from the current time to the end of the useful life. Accurate RUL estimation plays a critical role in Prognostics and Health Management(PHM). Data driven approaches for RUL estimation use sensor data and operational data to estimate RUL. Traditional regression based approaches and recent Convolutional Neural Network (CNN) approach use features created from sliding windows to build models. However, sequence information is not fully considered in these approaches. Sequence learning models such as Hidden Markov Models (HMMs) and Recurrent Neural Networks (RNNs) have flaws when modeling sequence information. HMMs are limited to discrete hidden states and are known to have issues when modeling long-term dependencies in the data. RNNs also have issues with long-term dependencies. In this work, we propose a Long Short-Term Memory (LSTM) approach for RUL estimation, which can make full use of the sensor sequence information and expose hidden patterns within sensor data with multiple operating conditions, fault and degradation models. Extensive experiments using three widely adopted Prognostics and Health Management data sets show that LSTM for RUL estimation significantly outperforms traditional approaches for RUL estimation as well as Convolutional Neural Network (CNN).
---
paper_title: A convolutional neural network based feature learning and fault diagnosis method for the condition monitoring of gearbox

paper_content:

Abstract Feature extraction plays a vital role in intelligent fault diagnosis of mechanical system. Nevertheless, traditional feature extraction methods suffer from three problems, which are (1) the requirements of domain expertise and prior knowledge, (2) the sensitive to the changes of mechanical system and (3) the limitations of mining new features. It is attractive and meaningful to investigate an automatic feature extraction method, which can adaptively learn features from raw data and discover new fault-sensitive features. Deep learning has been widely used in image analysis and speech recognition with great success. The key advantage of this method lies into the ability of mining representative information and sensitive features from raw data. However, the application of deep learning in feature leaning for mechanical diagnosis is still few, and limited studies have been carried out to compare the effectiveness of feature leaning with various data types. This paper will focus on developing a convolutional neural network (CNN) to learn features directly from frequency data of vibration signals and testing the different performance of feature learning from raw data, frequency spectrum and combined time-frequency data. Manual features from time domain, frequency domain and wavelet domain as well as three common intelligent methods are used as comparisons. The effectiveness of the proposed method is validated through PHM 2009 gearbox challenge data and a planetary gearbox test rig. The results demonstrate that the proposed method is able to learn features adaptively from frequency data and achieve higher diagnosis accuracy than other comparative methods.
---
paper_title: A survey of deep neural network architectures and their applications

paper_content:

This work was supported in part the Royal Society of the UK, the National Natural Science Foundation of China under Grants 61329301, 61374010, and 61403319, and the Alexander von Humboldt Foundation of Germany.
---
paper_title: Deep learning

paper_content:

Deep learning is a form of machine learning that enables computers to learn from experience and understand the world in terms of a hierarchy of concepts. Because the computer gathers knowledge from experience, there is no need for a human computer operator to formally specify all the knowledge that the computer needs. The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones; a graph of these hierarchies would be many layers deep. This book introduces a broad range of topics in deep learning. The text offers mathematical and conceptual background, covering relevant concepts in linear algebra, probability theory and information theory, numerical computation, and machine learning. It describes deep learning techniques used by practitioners in industry, including deep feedforward networks, regularization, optimization algorithms, convolutional networks, sequence modeling, and practical methodology; and it surveys such applications as natural language processing, speech recognition, computer vision, online recommendation systems, bioinformatics, and videogames. Finally, the book offers research perspectives, covering such theoretical topics as linear factor models, autoencoders, representation learning, structured probabilistic models, Monte Carlo methods, the partition function, approximate inference, and deep generative models. Deep Learning can be used by undergraduate or graduate students planning careers in either industry or research, and by software engineers who want to begin using deep learning in their products or platforms. A website offers supplementary material for both readers and instructors.
---
paper_title: Computation of optical flow using a neural network

paper_content:

A method for computing optical flow using a neural network is presented. Usually, the measurement primitives used for computing optical flow from successive image frames are the image-intensity values and their spatial and temporal derivatives, and tokens such as edges, corners, and linear features. Conventional methods based on such primitives suffer from edge sparsity, noise distortion, or sensitivity to rotation. The authors first fit a 2-D polynomial to find a smooth continuous image-intensity function in a window and estimate the subpixel intensity values and their principal curvatures. Under the local rigidity assumption and smoothness constraints, a neural network is then used to implement the computing procedure based on the estimated intensity values and their principal curvatures. Owing to the dense measured primitives, a dense optical flow with subpixel accuracy is obtained with only a few iterations. Since intensity values and their principle curvatures are rotation-invariant, this method can detect both rotating and translating objects in the scene. Experimental results using synthetic image sequences demonstrate the efficacy of the method. >
---
paper_title: Fault Diagnosis for Rotating Machinery Using Vibration Measurement Deep Statistical Feature Learning

paper_content:

Fault diagnosis is important for the maintenance of rotating machinery. The detection of faults and fault patterns is a challenging part of machinery fault diagnosis. To tackle this problem, a model for deep statistical feature learning from vibration measurements of rotating machinery is presented in this paper. Vibration sensor signals collected from rotating mechanical systems are represented in the time, frequency, and time-frequency domains, each of which is then used to produce a statistical feature set. For learning statistical features, real-value Gaussian-Bernoulli restricted Boltzmann machines (GRBMs) are stacked to develop a Gaussian-Bernoulli deep Boltzmann machine (GDBM). The suggested approach is applied as a deep statistical feature learning tool for both gearbox and bearing systems. The fault classification performances in experiments using this approach are 95.17% for the gearbox, and 91.75% for the bearing system. The proposed approach is compared to such standard methods as a support vector machine, GRBM and a combination model. In experiments, the best fault classification rate was detected using the proposed model. The results show that deep learning with statistical feature extraction has an essential improvement potential for diagnosing rotating machinery faults.
---
paper_title: Remaining useful life estimation of engineered systems using vanilla LSTM neural networks

paper_content:

Abstract Long Short-Term Memory (LSTM) networks are a significant branch of Recurrent Neural Networks (RNN), capable of learning long-term dependencies. In recent years, vanilla LSTM (a variation of original LSTM above) has become the state-of-the-art model for a variety of machine learning problems, especially Natural Language Processing (NLP). However, in industry, this powerful Deep Neural Network (DNN) has not aroused wide concern. In research focusing on Prognostics and Health Management (PHM) technology for complex engineered systems, Remaining Useful Life (RUL) estimation is one of the most challenging problems, which can lead to appropriate maintenance actions to be scheduled proactively to avoid catastrophic failures and minimize economic losses of the systems. Following that, this paper aims to propose utilizing vanilla LSTM neural networks to get good RUL prediction accuracy which makes the most of long short-term memory ability, in the cases of complicated operations, working conditions, model degradations and strong noises. In addition, to promote cognition ability about model degradation processes, a dynamic differential technology was proposed to extract inter-frame information. The whole proposition is illustrated and discussed by performing tests on a case of the health monitoring of aircraft turbofan engines which have four different issues. Performances of vanilla LSTM are benchmarked with standard RNN and Gated Recurrent Unit (GRU) LSTM. Results show the significance of performance improvement achieved by vanilla LSTM.
---
paper_title: Using Deep Learning-Based Approach to Predict Remaining Useful Life of Rotating Components

paper_content:

In the age of Internet of Things and Industrial 4.0, prognostic and health management (PHM) systems are used to collect massive real-time data from mechanical equipment. PHM big data has the characteristics of large-volume, diversity, and high-velocity. Effectively mining features from such data and accurately predicting the remaining useful life (RUL) of the rotating components with new advanced methods become issues in PHM. Traditional data driven prognostics is based on shallow learning architectures, requires establishing explicit model equations and much prior knowledge about signal processing techniques and prognostic expertise, and therefore is limited in the age of big data. This paper presents a deep learning-based approach for RUL prediction of rotating components with big data. The presented approach is tested and validated using data collected from a gear test rig and bearing run-to-failure tests and compared with existing PHM methods. The test results show the promising RUL prediction performance of the deep learning-based approach.
---
paper_title: Remaining Useful Life Estimation in Prognostics Using Deep Convolution Neural Networks

paper_content:

Traditionally, system prognostics and health management (PHM) depends on sufficient prior knowledge of critical components degradation process in order to predict the remaining useful life (RUL). However, the accurate physical or expert models are not available in most cases. This paper proposes a new data-driven approach for prognostics using deep convolution neural networks (DCNN). Time window approach is employed for sample preparation in order for better feature extraction by DCNN. Raw collected data with normalization are directly used as inputs to the proposed network, and no prior expertise on prognostics and signal processing is required, that facilitates the application of the proposed method. In order to show the effectiveness of the proposed approach, experiments on the popular C-MAPSS dataset for aero-engine unit prognostics are carried out. High prognostic accuracy on the RUL estimation is achieved. The superiority of the proposed method is demonstrated by comparisons with other popular approaches and the state-of-the-art results on the same dataset. The results of this study suggest that the proposed data-driven prognostic method offers a new and promising approach.
---
paper_title: Intelligent fault diagnosis approach with unsupervised feature learning by stacked denoising autoencoder

paper_content:

Condition monitoring and fault diagnosis are important for maintaining the system performance and guaranteeing the operational safety. The traditional data-driven approaches mostly incorporate well-defined features and methodologies such as supervised artificial intelligence algorithms. Prior knowledge of possible features and a large quantity of labelled condition data are needed. Besides, many traditional approaches require rebuilding or a retraining of the original model to diagnosis new conditions. The present study proposes an intelligent fault diagnosis approach that uses a deep neural network (DNN) based on stacked denoising autoencoder. Representative features are learned by applying the denoising autoencoder to the unlabelled data in an unsupervised manner. A DNN is then constructed and fine-tuned with just a few items of labelled data. The trained DNN achieves high performance in fault classification. Furthermore, new conditions can be correctly classified by simply fine-tuning the trained DNN model using a small amount of labelled data under the new conditions. The effectiveness of the proposed approach is evaluated using a case study of fault diagnosis of a bearing unit. The results indicate that the proposed method can extract representative features from massive unlabelled data on the system condition and achieve high performance in fault diagnosis.
---
paper_title: Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life

paper_content:

Prognostics technique aims to accurately estimate the Remaining Useful Life (RUL) of a subsystem or a component using sensor data, which has many real world applications. However, many of the existing algorithms are based on linear models, which cannot capture the complex relationship between the sensor data and RUL. Although Multilayer Perceptron (MLP) has been applied to predict RUL, it cannot learn salient features automatically, because of its network structure. A novel deep Convolutional Neural Network (CNN) based regression approach for estimating the RUL is proposed in this paper. Although CNN has been applied on tasks such as computer vision, natural language processing, speech recognition etc., this is the first attempt to adopt CNN for RUL estimation in prognostics. Different from the existing CNN structure for computer vision, the convolution and pooling filters in our approach are applied along the temporal dimension over the multi-channel sensor data to incorporate automated feature learning from raw sensor signals in a systematic way. Through the deep architecture, the learned features are the higher-level abstract representation of low-level raw sensor signals. Furthermore, feature learning and RUL estimation are mutually enhanced by the supervised feedback. We compared with several state-of-the-art algorithms on two publicly available data sets to evaluate the effectiveness of this proposed approach. The encouraging results demonstrate that our proposed deep convolutional neural network based regression approach for RUL estimation is not only more efficient but also more accurate.
---
paper_title: Deep Learning for solar power forecasting — An approach using AutoEncoder and LSTM Neural Networks

paper_content:

Power forecasting of renewable energy power plants is a very active research field, as reliable information about the future power generation allow for a safe operation of the power grid and helps to minimize the operational costs of these energy sources. Deep Learning algorithms have shown to be very powerful in forecasting tasks, such as economic time series or speech recognition. Up to now, Deep Learning algorithms have only been applied sparsely for forecasting renewable energy power plants. By using different Deep Learning and Artificial Neural Network algorithms, such as Deep Belief Networks, AutoEncoder, and LSTM, we introduce these powerful algorithms in the field of renewable energy power forecasting. In our experiments, we used combinations of these algorithms to show their forecast strength compared to a standard MLP and a physical forecasting model in the forecasting the energy output of 21 solar power plants. Our results using Deep Learning algorithms show a superior forecasting performance compared to Artificial Neural Networks as well as other reference models such as physical models.
---
paper_title: Long Short-Term Memory Network for Remaining Useful Life estimation

paper_content:

Remaining Useful Life (RUL) of a component or a system is defined as the length from the current time to the end of the useful life. Accurate RUL estimation plays a critical role in Prognostics and Health Management(PHM). Data driven approaches for RUL estimation use sensor data and operational data to estimate RUL. Traditional regression based approaches and recent Convolutional Neural Network (CNN) approach use features created from sliding windows to build models. However, sequence information is not fully considered in these approaches. Sequence learning models such as Hidden Markov Models (HMMs) and Recurrent Neural Networks (RNNs) have flaws when modeling sequence information. HMMs are limited to discrete hidden states and are known to have issues when modeling long-term dependencies in the data. RNNs also have issues with long-term dependencies. In this work, we propose a Long Short-Term Memory (LSTM) approach for RUL estimation, which can make full use of the sensor sequence information and expose hidden patterns within sensor data with multiple operating conditions, fault and degradation models. Extensive experiments using three widely adopted Prognostics and Health Management data sets show that LSTM for RUL estimation significantly outperforms traditional approaches for RUL estimation as well as Convolutional Neural Network (CNN).
---
paper_title: A sparse auto-encoder-based deep neural network approach for induction motor faults classification

paper_content:

Abstract This paper presents a deep neural network (DNN) approach for induction motor fault diagnosis. The approach utilizes sparse auto-encoder (SAE) to learn features, which belongs to unsupervised feature learning that only requires unlabeled measurement data. With the help of the denoising coding, partial corruption is added into the input of the SAE to improve robustness of feature representation. Features learned from the SAE are then used to train a neural network classifier for identifying induction motor faults. In addition, to prevent overfitting during the training process, a recently developed regularization method called “dropout” which has been proved to be very effective in neural network was employed. An experiment performed on a machine fault simulator indicates that compared with traditional neural network, the SAE-based DNN can achieve superior performance for feature learning and classification in the field of induction motor fault diagnosis.
---
paper_title: Deep Learning in Neural Networks: An Overview

paper_content:

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.
---
paper_title: Generating feature sets for fault diagnosis using denoising stacked auto-encoder

paper_content:

Recent advancements in sensor technologies and data driven model based techniques have made intelligent diagnostic systems prominent in machine maintenance frameworks of industries. The performance of such systems immensely relies upon the quality of features extracted and the classifier model learned. Traditionally features were handcrafted, where engineers would manually design them with statistical parameters and signal transforms based energy distribution analysis. Recently, deep learning techniques have shown new ways of obtaining useful feature representation that provide state of the art results in image and speech processing applications. This paper first presents a brief survey of traditional handcrafted features and later presents a short analysis of handcrafted features v/s features learned by deep neural networks (DNN), for doing fault diagnosis. The DNN based features in this paper were generated in 3 phases: 1) extracted handcrafted features using traditional techniques 2) initialized the weights of DNN by learning de-noising sparse auto-encoders with the handcrafted features in unsupervised fashion and 3) applied two generic fine tuning heuristics that tailor DNN's weights to give good classification performance. The experimentation and analysis were performed on 5 datasets: one each on Air compressor monitoring, Drill bit monitoring and Steel plate monitoring, and two on bearing fault monitoring data. The results clearly show the prospects of DNN obtaining good feature representations and good classification performance. Further, it also finds that Fast Fourier Transform based features with DNN are more suited for Support Vector Machine as classifier than Random Forest.
---
paper_title: Dropout: a simple way to prevent neural networks from overfitting

paper_content:

Deep neural nets with a large number of parameters are very powerful machine learning systems. However, overfitting is a serious problem in such networks. Large networks are also slow to use, making it difficult to deal with overfitting by combining the predictions of many different large neural nets at test time. Dropout is a technique for addressing this problem. The key idea is to randomly drop units (along with their connections) from the neural network during training. This prevents units from co-adapting too much. During training, dropout samples from an exponential number of different "thinned" networks. At test time, it is easy to approximate the effect of averaging the predictions of all these thinned networks by simply using a single unthinned network that has smaller weights. This significantly reduces overfitting and gives major improvements over other regularization methods. We show that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
---
paper_title: Deep Learning for solar power forecasting — An approach using AutoEncoder and LSTM Neural Networks

paper_content:

Power forecasting of renewable energy power plants is a very active research field, as reliable information about the future power generation allow for a safe operation of the power grid and helps to minimize the operational costs of these energy sources. Deep Learning algorithms have shown to be very powerful in forecasting tasks, such as economic time series or speech recognition. Up to now, Deep Learning algorithms have only been applied sparsely for forecasting renewable energy power plants. By using different Deep Learning and Artificial Neural Network algorithms, such as Deep Belief Networks, AutoEncoder, and LSTM, we introduce these powerful algorithms in the field of renewable energy power forecasting. In our experiments, we used combinations of these algorithms to show their forecast strength compared to a standard MLP and a physical forecasting model in the forecasting the energy output of 21 solar power plants. Our results using Deep Learning algorithms show a superior forecasting performance compared to Artificial Neural Networks as well as other reference models such as physical models.
---
paper_title: Long Short-Term Memory Network for Remaining Useful Life estimation

paper_content:

Remaining Useful Life (RUL) of a component or a system is defined as the length from the current time to the end of the useful life. Accurate RUL estimation plays a critical role in Prognostics and Health Management(PHM). Data driven approaches for RUL estimation use sensor data and operational data to estimate RUL. Traditional regression based approaches and recent Convolutional Neural Network (CNN) approach use features created from sliding windows to build models. However, sequence information is not fully considered in these approaches. Sequence learning models such as Hidden Markov Models (HMMs) and Recurrent Neural Networks (RNNs) have flaws when modeling sequence information. HMMs are limited to discrete hidden states and are known to have issues when modeling long-term dependencies in the data. RNNs also have issues with long-term dependencies. In this work, we propose a Long Short-Term Memory (LSTM) approach for RUL estimation, which can make full use of the sensor sequence information and expose hidden patterns within sensor data with multiple operating conditions, fault and degradation models. Extensive experiments using three widely adopted Prognostics and Health Management data sets show that LSTM for RUL estimation significantly outperforms traditional approaches for RUL estimation as well as Convolutional Neural Network (CNN).
---
paper_title: A sparse auto-encoder-based deep neural network approach for induction motor faults classification

paper_content:

Abstract This paper presents a deep neural network (DNN) approach for induction motor fault diagnosis. The approach utilizes sparse auto-encoder (SAE) to learn features, which belongs to unsupervised feature learning that only requires unlabeled measurement data. With the help of the denoising coding, partial corruption is added into the input of the SAE to improve robustness of feature representation. Features learned from the SAE are then used to train a neural network classifier for identifying induction motor faults. In addition, to prevent overfitting during the training process, a recently developed regularization method called “dropout” which has been proved to be very effective in neural network was employed. An experiment performed on a machine fault simulator indicates that compared with traditional neural network, the SAE-based DNN can achieve superior performance for feature learning and classification in the field of induction motor fault diagnosis.
---
paper_title: Deep Learning in Neural Networks: An Overview

paper_content:

In recent years, deep artificial neural networks (including recurrent ones) have won numerous contests in pattern recognition and machine learning. This historical survey compactly summarizes relevant work, much of it from the previous millennium. Shallow and Deep Learners are distinguished by the depth of their credit assignment paths, which are chains of possibly learnable, causal links between actions and effects. I review deep supervised learning (also recapitulating the history of backpropagation), unsupervised learning, reinforcement learning & evolutionary computation, and indirect search for short programs encoding deep and large networks.
---
paper_title: Computer science: The learning machines

paper_content:

Using massive amounts of data to recognize photos and speech, deep-learning computers are taking a big step towards true artificial intelligence.
---
paper_title: A survey of machine learning for big data processing

paper_content:

There is no doubt that big data are now rapidly expanding in all science and engineering domains. While the potential of these massive data is undoubtedly significant, fully making sense of them requires new ways of thinking and novel learning techniques to address the various challenges. In this paper, we present a literature survey of the latest advances in researches on machine learning for big data processing. First, we review the machine learning techniques and highlight some promising learning methods in recent studies, such as representation learning, deep learning, distributed and parallel learning, transfer learning, active learning, and kernel-based learning. Next, we focus on the analysis and discussions about the challenges and possible solutions of machine learning for big data. Following that, we investigate the close connections of machine learning with signal processing techniques for big data processing. Finally, we outline several open issues and research trends.
---
paper_title: Towards Design of Prognostics and Health Management Solutions for Maritime Assets

paper_content:

Abstract With increase in competition between OEMs of maritime assets and operators alike, the need to maximize the productivity of an equipment and increase operational efficiency and reliability is increasingly stringent and challenging. Also, with the adoption of availability contracts, maritime OEMs are becoming directly interested in understanding the health of their assets in order to maximize profits and to minimize the risk of a system's failure. The key to address these challenges and needs is performance optimization. For this to be possible it is important to understand that system failure can induce downtime which will increase the total cost of ownership, therefore it is important by all means to minimize unscheduled maintenance. If the state of health or condition of a system, subsystem or component is known, condition-based maintenance can be carried out and system design optimization can be achieved thereby reducing total cost of ownership. With the increasing competition with regards to the maritime industry, it is important that the state of health of a component/sub-system/system/asset is known before a vessel embarks on a mission. Any breakdown or malfunction in any part of any system or subsystem on board vessel during the operation offshore will lead to large economic losses and sometimes cause accidents. For example, damages to the fuel oil system of vessel's main engine can result in huge downtime as a result of the vessel not being in operation. This paper presents a prognostic and health management (PHM) development process applied on a fuel oil system powering diesel engines typically used in various cruise and fishing vessels, dredgers, pipe laying vessels and large oil tankers. This process will hopefully enable future PHM solutions for maritime assets to be designed in a more formal and systematic way.
---
paper_title: Why deep neural nets cannot ever match biological intelligence and what to do about it

paper_content:

The recently introduced theory of practopoiesis offers an account on how adaptive intelligent systems are organized. According to that theory, biological agents adapt at three levels of organization and this structure applies also to our brains. This is referred to as tri-traversal theory of the organization of mind or for short, a T3-structure. To implement a similar T3-organization in an artificially intelligent agent, it is necessary to have multiple policies, as usually used as a concept in the theory of reinforcement learning. These policies have to form a hierarchy. We define adaptive practopoietic systems in terms of hierarchy of policies and calculate whether the total variety of behavior required by real-life conditions of an adult human can be satisfactorily accounted for by a traditional approach to artificial intelligence based on T2-agents, or whether a T3-agent is needed instead. We conclude that the complexity of real life can be dealt with appropriately only by a T3-agent. This means that the current approaches to artificial intelligence, such as deep architectures of neural networks, will not suffice with fixed network architectures. Rather, they will need to be equipped with intelligent mechanisms that rapidly alter the architectures of those networks.
---
paper_title: About The Importance of Autonomy and Digital Twins for the Future of Manufacturing

paper_content:

Abstract Industrie 4.0 - the “brand” name of the German initiative driving the future of manufacturing - is one of several initiatives around the globe emphasizing the importance of industrial manufacturing for economy and society. Besides the socio-economical if not political question which has to be answered - including the question about the future of labor - there are a couple of substantial technical and technological questions that have to be taken care of as well.
---

Title: A Comprehensive Survey of Prognostics and Health Management Based on Deep Learning for Autonomous Ships
Section 1: Background: Prognostics and Health Management and Deep Learning
Description 1: In this section, the necessary background on PHM and DL will be introduced, defining PHM in general, explaining each step, and presenting DL with its promising aspects.

Section 2: Autonomous Ships: Benefits and Challenges in Applying PHM Based on DL
Description 2: This section introduces and discusses the benefits and challenges in applying Prognostics and Health Management based on deep learning (PHMDL) in autonomous ships.

Section 3: Applications of Deep Learning to Prognostics and Health Management
Description 3: This section reviews recent applications of well-established DL techniques like AE, CNN, DBN, and LSTM to PHM, highlighting their strengths and weaknesses.

Section 4: Discussion
Description 4: This section gives more general discussions concerning suitable deep architectures, addresses existing problems, and introduces future research opportunities.
 
Section 5: Conclusion
Description 5: This section summarizes the survey, highlighting the main findings and discussing the potential of DL, RL, and DT in advancing the development of intelligent PHM systems for autonomous ships.