Machine Vision in the Context of Robotics: A Systematic Literature Review
---
paper_title: Guidelines for conducting systematic mapping studies in software engineering: An update

paper_content:

Abstract Context Systematic mapping studies are used to structure a research area, while systematic reviews are focused on gathering and synthesizing evidence. The most recent guidelines for systematic mapping are from 2008. Since that time, many suggestions have been made of how to improve systematic literature reviews (SLRs). There is a need to evaluate how researchers conduct the process of systematic mapping and identify how the guidelines should be updated based on the lessons learned from the existing systematic maps and SLR guidelines. Objective To identify how the systematic mapping process is conducted (including search, study selection, analysis and presentation of data, etc.); to identify improvement potentials in conducting the systematic mapping process and updating the guidelines accordingly. Method We conducted a systematic mapping study of systematic maps, considering some practices of systematic review guidelines as well (in particular in relation to defining the search and to conduct a quality assessment). Results In a large number of studies multiple guidelines are used and combined, which leads to different ways in conducting mapping studies. The reason for combining guidelines was that they differed in the recommendations given. Conclusion The most frequently followed guidelines are not sufficient alone. Hence, there was a need to provide an update of how to conduct systematic mapping studies. New guidelines have been proposed consolidating existing findings.
---
paper_title: Systematic mapping studies in software engineering

paper_content:

BACKGROUND: A software engineering systematic map is a defined method to build a classification scheme and structure a software engineering field of interest. The analysis of results focuses on frequencies of publications for categories within the scheme. Thereby, the coverage of the research field can be determined. Different facets of the scheme can also be combined to answer more specific research questions. ::: ::: OBJECTIVE: We describe how to conduct a systematic mapping study in software engineering and provide guidelines. We also compare systematic maps and systematic reviews to clarify how to chose between them. This comparison leads to a set of guidelines for systematic maps. ::: ::: METHOD: We have defined a systematic mapping process and applied it to complete a systematic mapping study. Furthermore, we compare systematic maps with systematic reviews by systematically analyzing existing systematic reviews. ::: ::: RESULTS: We describe a process for software engineering systematic mapping studies and compare it to systematic reviews. Based on this, guidelines for conducting systematic maps are defined. ::: ::: CONCLUSIONS: Systematic maps and reviews are different in terms of goals, breadth, validity issues and implications. Thus, they should be used complementarily and require different methods (e.g., for analysis).
---
paper_title: Computer vision for fruit harvesting robots – state of the art and challenges ahead

paper_content:

Despite extensive research conducted in machine vision for harvesting robots, practical success in this field of agrobotics is still limited. This article presents a comprehensive review of classical and state-of-the-art machine vision solutions employed in such systems, with special emphasis on the visual cues and machine vision algorithms used. We discuss the advantages and limitations of each approach and we examine these capacities in light of the challenges ahead. We conclude with suggested directions from the general computer vision literature which could assist our research community meet these challenges and bring us closer to the goal of practical selective fruit harvesting robots.
---
paper_title: Survey of single-target visual tracking methods based on online learning

paper_content:

Visual tracking is a popular and challenging topic in computer vision and robotics. Owing to changes in the appearance of the target and complicated variations that may occur in various scenes, online learning scheme is necessary for advanced visual tracking framework to adopt. This paper briefly introduces the challenges and applications of visual tracking and focuses on discussing the state-of-the-art online-learning-based tracking methods by category. We provide detail descriptions of representative methods in each category, and examine their pros and cons. Moreover, several most representative algorithms are implemented to provide quantitative reference. At last, we outline several trends for future visual tracking research.
---
paper_title: Review of Neural Network Techniques in the Verge of Image Processing

paper_content:

Image processing is a vast area in the research field nowadays. This paper is a fleeting review on various technologies implemented to satisfy different image processing tasks like image segmentation, enhancement, restoration, acquisition, compression, classification, and many more. Neural network is one of the major techniques which is emphasized here. Different types of techniques using neural networks and hybridizations of neural network are discussed here briefly which are used for many image processing applications.
---
paper_title: Systematic mapping studies in software engineering

paper_content:

BACKGROUND: A software engineering systematic map is a defined method to build a classification scheme and structure a software engineering field of interest. The analysis of results focuses on frequencies of publications for categories within the scheme. Thereby, the coverage of the research field can be determined. Different facets of the scheme can also be combined to answer more specific research questions. ::: ::: OBJECTIVE: We describe how to conduct a systematic mapping study in software engineering and provide guidelines. We also compare systematic maps and systematic reviews to clarify how to chose between them. This comparison leads to a set of guidelines for systematic maps. ::: ::: METHOD: We have defined a systematic mapping process and applied it to complete a systematic mapping study. Furthermore, we compare systematic maps with systematic reviews by systematically analyzing existing systematic reviews. ::: ::: RESULTS: We describe a process for software engineering systematic mapping studies and compare it to systematic reviews. Based on this, guidelines for conducting systematic maps are defined. ::: ::: CONCLUSIONS: Systematic maps and reviews are different in terms of goals, breadth, validity issues and implications. Thus, they should be used complementarily and require different methods (e.g., for analysis).
---
paper_title: Speeding up Mutual Information Computation Using NVIDIA CUDA Hardware

paper_content:

We present an efficient method for mutual information (MI) computation between images (2D or 3D) for NVIDIA's `compute unified device architecture' (CUDA) compatible devices. Efficient parallelization of MI is particularly challenging on a `graphics processor unit' (GPU) due to the need for histogram-based calculation of joint and marginal probability mass functions (pmfs) with large number of bins. The data-dependent (unpredictable) nature of the updates to the histogram, together with hardware limitations of the GPU (lack of synchronization primitives and limited memory caching mechanisms) can make GPU-based computation inefficient. To overcome these limitation, we approximate the pmfs, using a down-sampled version of the joint- histogram which avoids memory update problems. Our CUDA implementation improves the efficiency of MI calculations by a factor of 25 compared to a standard CPU- based implementation and can be used in MI-based image registration applications.
---
paper_title: Object recognition using laser range finder and machine learning techniques

paper_content:

In recent years, computer vision has been widely used on industrial environments, allowing robots to perform important tasks like quality control, inspection and recognition. Vision systems are typically used to determine the position and orientation of objects in the workstation, enabling them to be transported and assembled by a robotic cell (e.g. industrial manipulator). These systems commonly resort to CCD (Charge-Coupled Device) Cameras fixed and located in a particular work area or attached directly to the robotic arm (eye-in-hand vision system). Although it is a valid approach, the performance of these vision systems is directly influenced by the industrial environment lighting. Taking all these into consideration, a new approach is proposed for eye-on-hand systems, where the use of cameras will be replaced by the 2D Laser Range Finder (LRF). The LRF will be attached to a robotic manipulator, which executes a pre-defined path to produce grayscale images of the workstation. With this technique the environment lighting interference is minimized resulting in a more reliable and robust computer vision system. After the grayscale image is created, this work focuses on the recognition and classification of different objects using inherent features (based on the invariant moments of Hu) with the most well-known machine learning models: k-Nearest Neighbor (kNN), Neural Networks (NNs) and Support Vector Machines (SVMs). In order to achieve a good performance for each classification model, a wrapper method is used to select one good subset of features, as well as an assessment model technique called K-fold cross-validation to adjust the parameters of the classifiers. The performance of the models is also compared, achieving performances of 83.5% for kNN, 95.5% for the NN and 98.9% for the SVM (generalized accuracy). These high performances are related with the feature selection algorithm based on the simulated annealing heuristic, and the model assessment (k-fold cross-validation). It makes possible to identify the most important features in the recognition process, as well as the adjustment of the best parameters for the machine learning models, increasing the classification ratio of the work objects present in the robot's environment. Graphical abstractDisplay Omitted Highlights? This paper analyses the performance of laser range finder for object recognition. ? Comparison between laser range finder and CCD camera approach. ? Object classification performed by using machine learning techniques. ? The proposed technique proves to be superior in terms of robustness and reliability.
---
paper_title: Pose-variant face recognition based on an improved Lucas-Kanade algorithm

paper_content:

In this paper, a pose-variant face recognition system is presented for study of human-robot interaction design. An iterative fitting algorithm is proposed to extract feature-point positions based on active appearance model (AAM). Comparing with the traditional Lucas-Kanade algorithm, the proposed iterative algorithm improves the capability of correct convergence as a larger variation of head posture occurs. After obtaining the location of feature points, the dimension of texture model is reduced and then sent to a back propagation neural network (BPNN) to recognize family-members. The proposed pose-variant face recognition system has been implemented on an embedded image processing system and integrated with a pet robot. Experimental results show that the robot can interact with a person in a responding manner. Tested with the database from UMIST and the database built in the lab, the proposed method achieved average recognition rates of 91.0% and 95.6% respectively.
---
paper_title: Proto-object categorisation and local gist vision using low-level spatial features

paper_content:

Object categorisation is a research area with significant challenges, especially in conditions with bad lighting, occlusions, different poses and similar objects. This makes systems that rely on precise information unable to perform efficiently, like a robotic arm that needs to know which objects it can reach. We propose a biologically inspired object detection and categorisation framework that relies on robust low-level object shape. Using only edge conspicuity and disparity features for scene figure-ground segregation and object categorisation, a trained neural network classifier can quickly categorise broad object families and consequently bootstrap a low-level scene gist system. We argue that similar processing is possibly located in the parietal pathway leading to the LIP cortex and, via areas V5/MT and MST, providing useful information to the superior colliculus for eye and head control.
---
paper_title: Neural network Reinforcement Learning for visual control of robot manipulators

paper_content:

It is known that most of the key problems in visual servo control of robots are related to the performance analysis of the system considering measurement and modeling errors. In this paper, the development and performance evaluation of a novel intelligent visual servo controller for a robot manipulator using neural network Reinforcement Learning is presented. By implementing machine learning techniques into the vision based control scheme, the robot is enabled to improve its performance online and to adapt to the changing conditions in the environment. Two different temporal difference algorithms (Q-learning and SARSA) coupled with neural networks are developed and tested through different visual control scenarios. A database of representative learning samples is employed so as to speed up the convergence of the neural network and real-time learning of robot behavior. Moreover, the visual servoing task is divided into two steps in order to ensure the visibility of the features: in the first step centering behavior of the robot is conducted using neural network Reinforcement Learning controller, while the second step involves switching control between the traditional Image Based Visual Servoing and the neural network Reinforcement Learning for enabling approaching behavior of the manipulator. The correction in robot motion is achieved with the definition of the areas of interest for the image features independently in both control steps. Various simulations are developed in order to present the robustness of the developed system regarding calibration error, modeling error, and image noise. In addition, a comparison with the traditional Image Based Visual Servoing is presented. Real world experiments on a robot manipulator with the low cost vision system demonstrate the effectiveness of the proposed approach.
---
paper_title: Small Brains, Smart Machines: From Fly Vision to Robot Vision and Back Again

paper_content:

Neurobiological and neuroethological findings on insects can be used to design and construct small robots controlling their navigation on the basis of bio-inspired visual strategies and circuits. Animals' visual guidance is partly mediated by motion-sensitive neurons, which are responsible for gauging the optic flow. Although neurons of this kind were discovered in vertebrates' and invertebrates' visual systems more than 50 years ago, the principles and neural mechanisms involved have not yet been completely elucidated. Here, first, I propose to outline some of the findings we made during the last few decades by performing electrophysiological recordings on identified neurons in the housefly's eye while applying optical stimulation to identified photoreceptors. Whereas these findings shed light on the inner processing structure of an elementary motion detector (EMD), recent studies in which the latest genetic and neuroanatomical methods were applied to the fruitfly's visual system have identified some of the neurons in the visual chain which are possibly involved in the neural circuitry underlying a given EMD. Then, I will describe some of the proof-of-concept robots that we have developed on the basis of our biological findings. The 100-g robot OCTAVE, for example, is able to avoid the ground, react to wind, and land autonomously on a flat terrain without ever having to measure any state variables such as distances or speeds. The 100-g robots OSCAR 1 and OSCAR 2 inspired by the microscanner we discovered in the housefly's eye are able to stabilize their body using mainly visual means and track a moving edge with hyperacuity. These robots react to the optic flow, which is sensed by miniature optic flow sensors inspired by the housefly's EMDs. Constructing a “biorobot” gives us a unique opportunity of checking the soundness and robustness of a principle that is initially thought to be understood by bringing it face to face with the real physical world. Bio-inspired robotics not only help neurobiologists and neuroethologists to identify and investigate worthwhile problems in animals' sensory-motor systems, but they also provide engineers with ideas for developing novel devices and machines with promising future applications, in the field of smart autonomous vehicles and microvehicles, for example.
---
paper_title: Marginal Space Deep Learning: Efficient Architecture for Volumetric Image Parsing.

paper_content:

Robust and fast solutions for anatomical object detection and segmentation support the entire clinical workflow from diagnosis, patient stratification, therapy planning, intervention and follow-up. Current state-of-the-art techniques for parsing volumetric medical image data are typically based on machine learning methods that exploit large annotated image databases. Two main challenges need to be addressed, these are the efficiency in scanning high-dimensional parametric spaces and the need for representative image features which require significant efforts of manual engineering. We propose a pipeline for object detection and segmentation in the context of volumetric image parsing, solving a two-step learning problem: anatomical pose estimation and boundary delineation. For this task we introduce Marginal Space Deep Learning (MSDL), a novel framework exploiting both the strengths of efficient object parametrization in hierarchical marginal spaces and the automated feature design of Deep Learning (DL) network architectures. In the 3D context, the application of deep learning systems is limited by the very high complexity of the parametrization. More specifically 9 parameters are necessary to describe a restricted affine transformation in 3D, resulting in a prohibitive amount of billions of scanning hypotheses. The mechanism of marginal space learning provides excellent run-time performance by learning classifiers in clustered, high-probability regions in spaces of gradually increasing dimensionality. To further increase computational efficiency and robustness, in our system we learn sparse adaptive data sampling patterns that automatically capture the structure of the input. Given the object localization, we propose a DL-based active shape model to estimate the non-rigid object boundary. Experimental results are presented on the aortic valve in ultrasound using an extensive dataset of 2891 volumes from 869 patients, showing significant improvements of up to 45.2% over the state-of-the-art. To our knowledge, this is the first successful demonstration of the DL potential to detection and segmentation in full 3D data with parametrized representations.
---
paper_title: Deep learning based smart radar vision system for object recognition

paper_content:

The capability of manipulating objects based on vision system are essential to robotic applications. For a vision system, high accuracy and robustness is vital. However, due to external environment and interference influences, as well as to lighting conditions influences, the most existing image-based vision system are limited in applications. In this context, a novel and robust data-driven vision system for recognizing different types of objects in various external situations has been presented in this paper. The system consists of three parts, namely, integrated module based on X2 Ultra-wideband (UWB) radar, deep learning (DL) module and robot controller. Specifically, the introduction of X2 UWB radar aims to (1) capture and process radar echo signals of different objects in various environments; (2) extract Range-Doppler power spectrograms from the processed signals accurately. The second part combines the DL algorithm to take the spectrograms as the input from the former to realize object classification. And the last part is used to control the behavior of the robot based on the classification results. The experimental results show that the proposed vision system achieves very reliable performance in recognizing six types of objects in four kinds of external environment and lighting conditions with three deep learning network models. Besides, by experimental data comparison, the performance of the proposed system achieves higher than that of the image-based system in the same poor conditions and ResNet is the optimal model for our proposed system in all conditions.
---
paper_title: Flame detection using deep learning

paper_content:

Flame detection is an increasingly important issue in intelligent surveillance. In fire flame detection, we need to extract visual features from video frames for training and test. Based on them, a group of shallow learning models have been developed to detect flames, such as color-based model, fuzzy-based model, motion and shape-based model, etc. Deep learning is a novel method which could be much efficient and accurate in flame detection. In this paper, we use YOLO model to implement flame detection and compare it with those shallow learning methods so as to determine the most efficient one for flame detection. Our contribution of this paper is to make use of the optimized YOLO model for flame detection from video frames. We collected the dataset and trained them using Google platform TensorFlow, the obtained accuracy of our proposed flame detection is up to 76%.
---
paper_title: Vision Based Robot Control Using Position Specific Artificial Neural Network

paper_content:

A new simpler vision based robot control system is proposed characterized which position specific artificial neural network (ANN) and end-effecter integrated camera system. Position specific ANN avoids the difficulty of covering the whole joint space with changing parameters using one set of ANN, and end-effecter integrated camera system makes the image of an object consistent when the end-effecter approaches the object. The image can be directly used as feedback. Simulation shows that the system works smoothly, and converges in limited steps. The algorithm simplifies the model of vision based robot manipulator control system, and improves the control accuracy and response time.
---
paper_title: A vision-based robotic grasping system using deep learning for garbage sorting

paper_content:

This paper proposes a robotic grasping system for automatically sorting garbage based on machine vision. This system achieves the identification and positioning of target objects in complex background before using manipulator to automatically grab the sorting objects. The object identification in complex background is the key problem that machine vision algorithm is trying to solve. This paper uses the deep learning method to achieve the authenticity identification of target object in complex background. In order to achieve the accurate grabbing of target object, we apply the Region Proposal Generation (RPN) and the VGG-16 model for object recognition and pose estimation. The machine vision system sends the information of the geometric centre coordinates and the angle of the long side of the target object to the manipulator which completes the classification and grabbing of the target object. The results of sorting experiment of the bottles in the garbage show that the vision algorithm and the manipulator control method of the proposed system can achieve the garbage sorting efficiently.
---
paper_title: Analysis of Q-learning on ANNs for robot control using live video feed

paper_content:

Training of artificial neural networks (ANNs) using reinforcement learning (RL) techniques is being widely discussed in the robot learning literature. The high model complexity of ANNs along with the model-free nature of RL algorithms provides a desirable combination for many robotics applications. There is a huge need for algorithms that generalize using raw sensory inputs, such as vision, without any hand-engineered features or domain heuristics. In this paper, the standard control problem of line following robot was used as a test-bed, and an ANN controller for the robot was trained on images from a live video feed using Q-learning. A virtual agent was first trained in simulation environment and then deployed onto a robot's hardware. The robot successfully learns to traverse a wide range of curves and displays excellent generalization ability. Qualitative analysis of the evolution of policies, performance and weights of the network provide insights into the nature and convergence of the learning algorithm.
---
paper_title: Pose-variant face recognition based on an improved Lucas-Kanade algorithm

paper_content:

In this paper, a pose-variant face recognition system is presented for study of human-robot interaction design. An iterative fitting algorithm is proposed to extract feature-point positions based on active appearance model (AAM). Comparing with the traditional Lucas-Kanade algorithm, the proposed iterative algorithm improves the capability of correct convergence as a larger variation of head posture occurs. After obtaining the location of feature points, the dimension of texture model is reduced and then sent to a back propagation neural network (BPNN) to recognize family-members. The proposed pose-variant face recognition system has been implemented on an embedded image processing system and integrated with a pet robot. Experimental results show that the robot can interact with a person in a responding manner. Tested with the database from UMIST and the database built in the lab, the proposed method achieved average recognition rates of 91.0% and 95.6% respectively.
---
paper_title: Adaptive Deep Learning Through Visual Domain Localization

paper_content:

A commercial robot, trained by its manufacturer to recognize a predefined number and type of objects, might be used in many settings, that will in general differ in their illumination conditions, background, type and degree of clutter, and so on. Recent computer vision works tackle this generalization issue through domain adaptation methods, assuming as source the visual domain where the system is trained and as target the domain of deployment. All approaches assume to have access to images from all classes of the target during training, an unrealistic condition in robotics applications. We address this issue proposing an algorithm that takes into account the specific needs of robot vision. Our intuition is that the nature of the domain shift experienced mostly in robotics is local. We exploit this through the learning of maps that spatially ground the domain and quantify the degree of shift, embedded into an end-to-end deep domain adaptation architecture. By explicitly localizing the roots of the domain shift we significantly reduce the number of parameters of the architecture to tune, we gain the flexibility necessary to deal with subset of categories in the target domain at training time, and we provide a clear feedback on the rationale behind any classification decision, which can be exploited in human-robot interactions. Experiments on two different settings of the iCub World database confirm the suitability of our method for robot vision.
---
paper_title: Performance Analysis of Neural Network Model for Automated Visual Inspection with Robotic Arm Controller System

paper_content:

The concept of Automated Visual Inspection (AVI) have emerged as a powerful platform for industrial machine vision where it used to inspect a large number of products rapidly. However, a major problem with this kind of application is the quality produced by the recognition process. In this paper, a system with the capability of identifying and categorizing a product based on image processing has been implemented. The image was processed by using Radial Basis Function (RBF) based on output center and spread values optimization. Robotic arm controller developed for pick and place the product based on their categories. Two performance measures are used to validate the model classification range and the spread values. The results of this project indicate that the model used able to identify the product and classify it according to their shape.
---
paper_title: Input Displacement Neuro-fuzzy Control and Object Recognition by Compliant Multi-fingered Passively Adaptive Robotic Gripper

paper_content:

The requirement for new flexible adaptive grippers is the ability to detect and recognize objects in their environments. It is known that robotic manipulators are highly nonlinear systems, and an accurate mathematical model is difficult to obtain, thus making it difficult make decision strategies using conventional techniques. Here, an adaptive neuro fuzzy inference system (ANFIS) for controlling input displacement and object recognition of a new adaptive compliant gripper is presented. The grasping function of the proposed adaptive multi-fingered gripper relies on the physical contact of the finger with an object. This design of the each finger has embedded sensors as part of its structure. The use of embedded sensors in a robot gripper gives the control system the ability to control input displacement of the gripper and to recognize particular shapes of the grasping objects. Fuzzy based controllers develop a control signal according to grasping object shape which yields on the firing of the rule base. The selection of the proper rule base depending on the situation can be achieved by using an ANFIS strategy, which becomes an integrated method of approach for the control purposes. In the designed ANFIS scheme, neural network techniques are used to select a proper rule base, which is achieved using the back propagation algorithm. The simulation results presented in this paper show the effectiveness of the developed method.
---
paper_title: A Hand Shape Instruction Recognition and Learning System Using Growing SOM with Asymmetric Neighborhood Function

paper_content:

In this paper, we adopt an asymmetric neighborhood function proposed by Aoki and Aoyagi in to a PL-G-SOM to improve the learning performance of the hand shape instruction perspective and learning system. The asymmetric neighborhood function was used in a normal SOM and few applications can be found. The novel PL-G-SOM and its improved version are named as "AGSOM" and “IAGSOM” respectively. The effectiveness of the proposed method was confirmed by the experiments with 8 kinds of instructions.
---
paper_title: Object recognition algorithms implemented on NAO robot for children's visual learning enhancement

paper_content:

Schools in many parts of the world use robots as social peers in order to interact with children and young students for a rich experience. Such use has shown significant enhancement of children's learning. This project uses the humanoid robot NAO which provides object recognition of colours, shapes, typed words, and handwritten digits and operators. The recognition of typed words provides performance of the corresponding movements in the sign language. Five classifiers including neural networks are used for the handwritten recognition of digits and operators. The accuracy of the object recognition algorithms are within the range of 82%-91% when tested on images captured by the robot including the movements which represent words in the sign language. The five classifiers for handwritten recognition produce highly accurate results which are within the range of 87%-98%. This project will serve as a promising provision for an affective touch for children and young students.
---
paper_title: End edge feedback and RBF neural network based vibration control of flexible manipulator

paper_content:

A vision feedback based active vibration control system for flexible manipulator is proposed. Active vibration control is conducted by employing Radial Basis Function (RBF) neural networks to regulate the control parameters of the proportional and derivative (PD) controller. Firstly, the composition of the vision feedback based piezoelectric flexible manipulator system is described. Furthermore, vision based vibration measurement for the flexible manipulator's end side and image processing methods are introduced. Secondly, the system dynamics model is derived by using finite element method, and the equation of the state space representation for the lower mode vibration is provided. Then, the vision feedback based control method is expounded, using RBF neural networks to tune the control parameters dynamically. Finally, simulation and experiments are conducted. The results demonstrate that vision feedback based control system and the RBF neural networks control algorithm can suppress the vibration of the flexible manipulator effectively.
---
paper_title: Analysis of Different Sparsity Methods in Constrained RBM for Sparse Representation in Cognitive Robotic Perception

paper_content:

Cognitive robotic systems nowadays are intensively involving learning algorithms to achieve highly adaptive and intelligent behaviors, including actuation, sensing, perception and adaptive control. Deep learning has emerged as an effective approach in image-based robotic perception and actions. Towards cognitive robotic perception based on deep learning, this paper focuses the Constrained Restricted Boltzmann Machine (RBM) on visual images for sparse feature representation. Inspired by sparse coding, the sparse constraints are performed on the hidden layer of RBM to obtain sparse and effective feature representation from perceived visual images. The RBM with Sparse Constraint (RBMSC) is proposed with a generalized optimization problem, where the constraints are applied on the probability density of hidden units directly to obtain more sparse representation. This paper presents three novel RBM variants, namely L1-RBM, L2-RBM, and L1/2-RBM constrained by L1-norm, L2-norm, and L1/2-norm on RBM, respectively. A Deep Belief Network with two hidden layers is built for comparison between each RBM variants. The experiments on MNIST database (Mixed National Institute of Standards and Technology database) show that the L1/2-RBM can obtain more sparse representation than RBM, L1-RBM, L2-RBM, and Sparse-RBM (SRBM) in terms of sparseness metric. For further verification, the proposed methods are still tested on MNIST Variations dataset. The recognition results from perceived images in MNIST and MNIST Variations demonstrate that our proposed constrained RBM variants are feasible for object cognitive and perception, and the proposed L1/2-RBM and L1-RBM outperforms RBM and SRBM in terms of object recognition.
---
paper_title: CNN-Based Model for Pose Detection of Industrial PCB

paper_content:

For applications in robot manipulate with object, get the pose of objects is very important for controller's subsequent operations, especially in PCB feeding and blanking field, the grasp success rate will be enhanced if robot can get a exact pose of objects that relative to end manipulator. So in this paper we utilize the CNN model to build on a neural network for 3 tasks: object recognition, location and pose detection. This model treat pose detection as a classification problem and try to combine recognition, location at the same level. To validate the performance of the multi-task detection model, experiments and analysis of the model performance was carried out by the real-time PCB detection test. In the experiment, we use the PCB dataset comprised of 3 types which contains different poses made by ourselves as train/test samples. The number of object pose categories was divided into 8bins, 12bins and 36bins according to pose detection precision. We analysis the effect of the non-uniform datasets on training process and the final detect results shows that this CNN-based detection model can achieve high accuracy on PCB pose detection.
---
paper_title: Intelligent Control of Arc Welding Dynamics during Robotic Welding Process

paper_content:

This paper presents a sequential research works on visual information acquirement and intelligent control of arc weld pool dynamics and seam formation during pulsed GTAW (Gas Tungsten Arc Welding) in robotic welding process. The visual information acquirement methods are focused in computer vision sensing, image processing and characteristic extraction of the weld pool surface from the single-item pool images by particular algorithms for robotic welding process. Based on acquired visual characteristics of weld pool and established neural network and knowledge models for predicting dynamical characteristics of weld pool during robotic welding, corresponding control methods, such adaptive control, self-learning and other composite intelligent control strategies are developed to control welding pool dynamics during pulsed GTAW by welding robot. Some experiments and applications of intelligent control methods in welding robot systems are shown in the paper.
---
paper_title: Pose-variant face recognition based on an improved Lucas-Kanade algorithm

paper_content:

In this paper, a pose-variant face recognition system is presented for study of human-robot interaction design. An iterative fitting algorithm is proposed to extract feature-point positions based on active appearance model (AAM). Comparing with the traditional Lucas-Kanade algorithm, the proposed iterative algorithm improves the capability of correct convergence as a larger variation of head posture occurs. After obtaining the location of feature points, the dimension of texture model is reduced and then sent to a back propagation neural network (BPNN) to recognize family-members. The proposed pose-variant face recognition system has been implemented on an embedded image processing system and integrated with a pet robot. Experimental results show that the robot can interact with a person in a responding manner. Tested with the database from UMIST and the database built in the lab, the proposed method achieved average recognition rates of 91.0% and 95.6% respectively.
---
paper_title: Automatic Tool Landmark Detection for Stereo Vision in Robot-Assisted Retinal Surgery

paper_content:

Computer vision and robotics are being increasingly applied in medical interventions. Especially in interventions where extreme precision is required they could make a difference. One such application is robot-assisted retinal microsurgery. In recent works, such interventions are conducted under a stereo-microscope, and with a robot-controlled surgical tool. The complementarity of computer vision and robotics has however not yet been fully exploited. In order to improve the robot control we are interested in 3D reconstruction of the anatomy and in automatic tool localization using a stereo microscope. In this paper, we solve this problem for the first time using a single pipeline, starting from uncalibrated cameras to reach metric 3D reconstruction and registration, in retinal microsurgery. The key ingredients of our method are: (a) surgical tool landmark detection, and (b) 3D reconstruction with the stereo microscope, using the detected landmarks. To address the former, we propose a novel deep learning method that detects and recognizes keypoints in high definition images at higher than real-time speed. We use the detected 2D keypoints along with their corresponding 3D coordinates obtained from the robot sensors to calibrate the stereo microscope using an affine projection model. We design an online 3D reconstruction pipeline that makes use of smoothness constraints and performs robot-to-camera registration. The entire pipeline is extensively validated on open-sky porcine eye sequences. Quantitative and qualitative results are presented for all steps.
---
paper_title: Corner detection based real-time workpiece recognition for robot manipulation

paper_content:

Workpiece recognition is vital and essential for robot manipulation which is one of the most important skills for robot. In this paper, we present a corner detection based strategy to recognize the type of workpiece so as to offer important visual cues for robot manipulation. Our framework works by three steps. Firstly, the bounding-box of workpiece is detected using a multi-scale convolutional neural network according to the closed regions (enclosed by edges). Secondly, three types of corners (Y-type, A-type and L-type) are detected by employing a simple neural network and the results are further refined according to both the detection probability and geometric relationship between corners. Finally, the type of workpiece is recognized on the basis of the relative position of corners. Due to that the workpiece detection step greatly reduces the searching space for the corner detection, a real-time process is achieved. Another important characteristic of our method lies in that the detected corners can be further used as basic modelling element in reconstructing the 3D structure of the workpiece, which is beneficial for the robot to decide the grasping position and pose. With the PKU-HR6.0 platform, a manipulation controller is established based on the proposed approach. Experimental results show that our approach is comparable with some state-of-the-art work in precision of recognition, and our PKU-HR6.0 robot is able to precisely recognize and locate the Tetris-shaped building blocks so that to well accomplish the manipulation tasks.
---
paper_title: A research on autonomous position method for mobile robot manipulator based on fusion feature

paper_content:

The cooperative work process and the automation work flow of the no man keeping watch library which takes the robot as its center and uses the internet of things technology are firstly introduced in this paper. A new data format of the RFID electronic label for books is designed, and the robustness of positioning of the manipulator is improved by the information fusion of the RFID label source and the CCD sensor source installed on the manipulator. A position method named THREE STEPS by the combination of fuzzy position, area position and accurate position is advanced, and the complex books positioning and grasping question is transformed into the world coordinate solving question of some point on the objective book. The fuzzy CMAC (cerebella model articulation controller) neuron network is adapted to realize the non-line relationship of fusion feature and manipulator position. And direct vision servo control architecture is used when designing the manipulator position system. The experiment result has proved this method can help the manipulator realize the accurate position and rapid catching.
---
paper_title: A Neuro-Fuzzy Visual Servoing Controller for an Articulated Manipulator

paper_content:

The challenges of selecting appropriate image features, optimizing complex nonlinear computations, and minimizing the approximation errors always exist in visual servoing. A fuzzy neural network controller is developed for a six-degrees-of-freedom robot manipulator to perform visual servoing is proposed to tackle these problems. To increase the accuracy of the image preprocesses, a synthetic image process performs feature extraction for the controller. The method combines a support vector machine contour recognition algorithm and a color-based feature recognition algorithm. For visual servoing, a control method based on the fuzzy cerebellar model articulation controller with the Takagi–Sugeno framework is proposed to directly map an image feature error vector to a desired robot end-effector velocity. This approach achieves visual servoing control without the need of computing the inverse interaction matrix. The control variables are learned and updated by the T–S fuzzy inference. This simplifies the implementation of visual servoing in real-time applications. The proposed control method is used to control an articulated manipulator with an eye-in-hand configuration. The results of simulations and experiments demonstrate that the proposed visual servoing controller has good performance, in terms of stability and convergence.
---
paper_title: Review of Neural Network Techniques in the Verge of Image Processing

paper_content:

Image processing is a vast area in the research field nowadays. This paper is a fleeting review on various technologies implemented to satisfy different image processing tasks like image segmentation, enhancement, restoration, acquisition, compression, classification, and many more. Neural network is one of the major techniques which is emphasized here. Different types of techniques using neural networks and hybridizations of neural network are discussed here briefly which are used for many image processing applications.
---
paper_title: Exploring aspects of cell intelligence with artificial reaction networks

paper_content:

The Artificial Reaction Network (ARN) is a Cell Signalling Network inspired connectionist representation belonging to the branch of A-Life known as Artificial Chemistry. Its purpose is to represent chemical circuitry and to explore computational properties responsible for generating emergent high-level behaviour associated with cells. In this paper, the computational mechanisms involved in pattern recognition and spatio-temporal pattern generation are examined in robotic control tasks. The results show that the ARN has application in limbed robotic control and computational functionality in common with Artificial Neural Networks. Like spiking neural models, the ARN can combine pattern recognition and complex temporal control functionality in a single network, however it offers increased flexibility. Furthermore, the results illustrate parallels between emergent neural and cell intelligence.
---
paper_title: Auto-Sorting System Toward Smart Factory Based on Deep Learning for Image Segmentation

paper_content:

Machine part sorting is important and monotonous in smart factory. In this paper, an auto-sorting system is proposed based on the deep learning method. In the proposed system, an industrial objection detection network combined with a robotic arm controlling system is designed to automatically and efficiently complete machine part sorting. Region-based full convolutional network (R-FCN) is applied for locating and recognizing different types of images of industrial object models. After comparison and simulation analysis, it illustrated that the R-FCN model trained with enough labeled data can efficiently and accurately recognize the object from the images captured by visual sensors. Furthermore, with enough data, the network can be robust to view angle rotation both vertically and horizontally, and a small part of overlapping of object will not mislead the judgment of the network in most situations. The case study results illustrate that the position and type of objects can be successfully detected. The code will be available publicly at https://github.com/tianwangbuaa/ .
---
paper_title: A vision-based robotic grasping system using deep learning for garbage sorting

paper_content:

This paper proposes a robotic grasping system for automatically sorting garbage based on machine vision. This system achieves the identification and positioning of target objects in complex background before using manipulator to automatically grab the sorting objects. The object identification in complex background is the key problem that machine vision algorithm is trying to solve. This paper uses the deep learning method to achieve the authenticity identification of target object in complex background. In order to achieve the accurate grabbing of target object, we apply the Region Proposal Generation (RPN) and the VGG-16 model for object recognition and pose estimation. The machine vision system sends the information of the geometric centre coordinates and the angle of the long side of the target object to the manipulator which completes the classification and grabbing of the target object. The results of sorting experiment of the bottles in the garbage show that the vision algorithm and the manipulator control method of the proposed system can achieve the garbage sorting efficiently.
---
paper_title: Door and cabinet recognition using Convolutional Neural Nets and real-time method fusion for handle detection and grasping

paper_content:

In this paper we present a new method that robustly identifies doors, cabinets and their respective handles, with special emphasis on extracting useful features from handles to be then manipulated. The novelty of this system relies on the combination of a Convolutional Neural Net (CNN), as a form of reducing the search space, several methods to extract point cloud data and a mobile robot to interact with the objects. The framework consists of the following components: The implementation of a CNN to extract a Region of Interest (ROI) from an image corresponding to a door or cabinet. Several vision based techniques to detect handles inside the ROI and its 3D positioning. A complementary plane segmentation method to differentiate door/cabinet from the handle. An algorithm to fuse both approaches robustly and extract essential information from the handle for robotic grasping (i.e. handle point cloud, door plane model, grasping locations, turning orientation, orthogonal vector to door). A mobile robot for grasping the handle. The system assumes no prior knowledge of the environment.
---
paper_title: You Only Look Once: Unified, Real-Time Object Detection

paper_content:

We present YOLO, a new approach to object detection. Prior work on object detection repurposes classifiers to perform detection. Instead, we frame object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. A single neural network predicts bounding boxes and class probabilities directly from full images in one evaluation. Since the whole detection pipeline is a single network, it can be optimized end-to-end directly on detection performance. ::: Our unified architecture is extremely fast. Our base YOLO model processes images in real-time at 45 frames per second. A smaller version of the network, Fast YOLO, processes an astounding 155 frames per second while still achieving double the mAP of other real-time detectors. Compared to state-of-the-art detection systems, YOLO makes more localization errors but is far less likely to predict false detections where nothing exists. Finally, YOLO learns very general representations of objects. It outperforms all other detection methods, including DPM and R-CNN, by a wide margin when generalizing from natural images to artwork on both the Picasso Dataset and the People-Art Dataset.
---
paper_title: A machine vision based pistachio sorting using transferred mid-level image representation of Convolutional Neural Network

paper_content:

Convolutional neural networks have proved to be prominent in various fields of machine vision and image classification. Although it necessitates a large-scale dataset for promising performance, the mid-level representation of these networks can be exploited for specified tasks with smaller annotated image dataset. To this end, by evaluating the generality-specificity of the desired layer as a feature extractor layer, the parameters of Convolutional Neural Networks learned on massive-size dataset like ImageNet can be transferred to a new model. In this study, the images of different sort of pistachios including trashes have been acquired to feed into a new model using a support vector classifier. The ultimate goal of our machine vision system is to separate the desired open-shell pistachios from other defected pistachios as well as trashes. For image segmentation, we use active contour method to detect objects and form both new images of each object type and their augmented images. Since our dataset is not large-scale compared to ImageNet classes, a feature reduction method is performed after the feature extractor layer of pre-trained Convolutional Neural Network. The results show the better performance of the proposed approach in detection of desired-formed pistachio facing unseen test set of images compared to basic approaches.
---
paper_title: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

paper_content:

State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [1] and Fast R-CNN [2] have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. We further merge RPN and Fast R-CNN into a single network by sharing their convolutional features—using the recently popular terminology of neural networks with ’attention’ mechanisms, the RPN component tells the unified network where to look. For the very deep VGG-16 model [3] , our detection system has a frame rate of 5 fps ( including all steps ) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015 competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning entries in several tracks. Code has been made publicly available.
---
paper_title: Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

paper_content:

Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features. We also present experiments that provide insight into what the network learns, revealing a rich hierarchy of image features. Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.
---
paper_title: Comparison of faster R-CNN models for object detection

paper_content:

Object detection is one of the important problems for autonomous robots. Faster R-CNN, one of the state-of-the-art object detection methods, approaches real time application; nevertheless, computational time lies borderline of real time application, i.e. 5fps with VGG16 model in K40 GPU system in [1]. Moreover, computation time depends on model and image crop size, but precision is also affected; usually, time and precision have trade-off relation. By adjusting input image size in spite of downgrading performance, computation time meets criteria for one model. Therefore, selection of a model is one of the important problems when faster R-CNN based object detection system for an autonomous robot is constructed. In this paper, we convert several state-of-the-art models from convolution neural network (CNN) for image classification. Then, we compare converted models with several image crop size in terms of computation time and detection precision. We will utilize those comparison data for selecting a proper detection model in case a robot needs to perform an object detection task.
---
paper_title: Kiwifruit detection in field images using Faster R-CNN with ZFNet

paper_content:

Abstract A kiwifruit detection system for field images was developed based on the deep convolutional neural network, which has a good robustness against the subjectivity and limitation of the features selected artificially. Under different lighting conditions, 2,100 sub-images with 784×784 pixels were prepared by random sub-sampling from 700 field captured images with a pixel resolution of 2352×1568 pixels. Sub-images were used as network training and validation samples. A faster R-CNN was trained end-to-end by using back-propagation and stochastic gradient descent techniques with Zeiler and Fergus network (ZFNet). The average precision of the Faster R-CNN-based kiwifruit detector was 89.3%. Finally, another 100 images of kiwifruit canopies in the field environment (including 5,918 fruits) were used for testing the network. The test results showed that the recognition ratio of occluded fruit, overlapping fruit, adjacent fruit and separated fruit were 82.5%, 85.6%, 94.3% and 96.7%, respectively. Overall, the model reached a recognition rate of 92.3%. The technique took 0.274 s to process each image (for images with 2352×1568 pixels) and only 5.0 ms on average to detect a fruit. Comparing against the conventional methods, it suggested that the proposed method has higher recognition rate and faster speed. Especially, the proposed technique was able to simultaneously detect individual kiwifruit in clusters, which provides a promise for accurate yield mapping and multi-arm robotic harvesting.
---
paper_title: Semantic Scene Segmentation for Indoor Robot Navigation via Deep Learning

paper_content:

This paper presents a vision-based framework for indoor robot navigation which exploits semantic segmentation and deep learning towards accurate and efficient indoor scene mapping and collision-free navigation for hardware constrained robotics. Firstly, a scheme for accurate and efficient path extraction using deep convolutional neural networks (DCNNs) and transfer learning for semantic pixel-wise segmentation is put forward. Secondly, multiple DCNN architectures and semantic segmentation techniques are explored to highlight the challenges associated with implementation as well as the trade-offs between accuracy and efficiency associated with the state-of-the-art. Finally, the achieved models are deployed and experimentally validated. Experimental results highlight promising potential with good segmentation accuracies and real-time feasibility. Results further highlight significant accuracy-efficiency trade-offs which are strongly driven by model decoder sub-network design.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: Fast R-CNN

paper_content:

This paper proposes a Fast Region-based Convolutional Network method (Fast R-CNN) for object detection. Fast R-CNN builds on previous work to efficiently classify object proposals using deep convolutional networks. Compared to previous work, Fast R-CNN employs several innovations to improve training and testing speed while also increasing detection accuracy. Fast R-CNN trains the very deep VGG16 network 9x faster than R-CNN, is 213x faster at test-time, and achieves a higher mAP on PASCAL VOC 2012. Compared to SPPnet, Fast R-CNN trains VGG16 3x faster, tests 10x faster, and is more accurate. Fast R-CNN is implemented in Python and C++ (using Caffe) and is available under the open-source MIT License at https://github.com/rbgirshick/fast-rcnn.
---
paper_title: Deep learning based smart radar vision system for object recognition

paper_content:

The capability of manipulating objects based on vision system are essential to robotic applications. For a vision system, high accuracy and robustness is vital. However, due to external environment and interference influences, as well as to lighting conditions influences, the most existing image-based vision system are limited in applications. In this context, a novel and robust data-driven vision system for recognizing different types of objects in various external situations has been presented in this paper. The system consists of three parts, namely, integrated module based on X2 Ultra-wideband (UWB) radar, deep learning (DL) module and robot controller. Specifically, the introduction of X2 UWB radar aims to (1) capture and process radar echo signals of different objects in various environments; (2) extract Range-Doppler power spectrograms from the processed signals accurately. The second part combines the DL algorithm to take the spectrograms as the input from the former to realize object classification. And the last part is used to control the behavior of the robot based on the classification results. The experimental results show that the proposed vision system achieves very reliable performance in recognizing six types of objects in four kinds of external environment and lighting conditions with three deep learning network models. Besides, by experimental data comparison, the performance of the proposed system achieves higher than that of the image-based system in the same poor conditions and ResNet is the optimal model for our proposed system in all conditions.
---
paper_title: R-FCN: Object Detection via Region-based Fully Convolutional Networks

paper_content:

We present region-based, fully convolutional networks for accurate and efficient object detection. In contrast to previous region-based detectors such as Fast/Faster R-CNN that apply a costly per-region subnetwork hundreds of times, our region-based detector is fully convolutional with almost all computation shared on the entire image. To achieve this goal, we propose position-sensitive score maps to address a dilemma between translation-invariance in image classification and translation-variance in object detection. Our method can thus naturally adopt fully convolutional image classifier backbones, such as the latest Residual Networks (ResNets), for object detection. We show competitive results on the PASCAL VOC datasets (e.g., 83.6% mAP on the 2007 set) with the 101-layer ResNet. Meanwhile, our result is achieved at a test-time speed of 170ms per image, 2.5-20 times faster than the Faster R-CNN counterpart. Code is made publicly available at: https://github.com/daijifeng001/r-fcn.
---
paper_title: Auto-Sorting System Toward Smart Factory Based on Deep Learning for Image Segmentation

paper_content:

Machine part sorting is important and monotonous in smart factory. In this paper, an auto-sorting system is proposed based on the deep learning method. In the proposed system, an industrial objection detection network combined with a robotic arm controlling system is designed to automatically and efficiently complete machine part sorting. Region-based full convolutional network (R-FCN) is applied for locating and recognizing different types of images of industrial object models. After comparison and simulation analysis, it illustrated that the R-FCN model trained with enough labeled data can efficiently and accurately recognize the object from the images captured by visual sensors. Furthermore, with enough data, the network can be robust to view angle rotation both vertically and horizontally, and a small part of overlapping of object will not mislead the judgment of the network in most situations. The case study results illustrate that the position and type of objects can be successfully detected. The code will be available publicly at https://github.com/tianwangbuaa/ .
---
paper_title: Very Deep Convolutional Networks for Large-Scale Image Recognition

paper_content:

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.
---
paper_title: Object recognition using laser range finder and machine learning techniques

paper_content:

In recent years, computer vision has been widely used on industrial environments, allowing robots to perform important tasks like quality control, inspection and recognition. Vision systems are typically used to determine the position and orientation of objects in the workstation, enabling them to be transported and assembled by a robotic cell (e.g. industrial manipulator). These systems commonly resort to CCD (Charge-Coupled Device) Cameras fixed and located in a particular work area or attached directly to the robotic arm (eye-in-hand vision system). Although it is a valid approach, the performance of these vision systems is directly influenced by the industrial environment lighting. Taking all these into consideration, a new approach is proposed for eye-on-hand systems, where the use of cameras will be replaced by the 2D Laser Range Finder (LRF). The LRF will be attached to a robotic manipulator, which executes a pre-defined path to produce grayscale images of the workstation. With this technique the environment lighting interference is minimized resulting in a more reliable and robust computer vision system. After the grayscale image is created, this work focuses on the recognition and classification of different objects using inherent features (based on the invariant moments of Hu) with the most well-known machine learning models: k-Nearest Neighbor (kNN), Neural Networks (NNs) and Support Vector Machines (SVMs). In order to achieve a good performance for each classification model, a wrapper method is used to select one good subset of features, as well as an assessment model technique called K-fold cross-validation to adjust the parameters of the classifiers. The performance of the models is also compared, achieving performances of 83.5% for kNN, 95.5% for the NN and 98.9% for the SVM (generalized accuracy). These high performances are related with the feature selection algorithm based on the simulated annealing heuristic, and the model assessment (k-fold cross-validation). It makes possible to identify the most important features in the recognition process, as well as the adjustment of the best parameters for the machine learning models, increasing the classification ratio of the work objects present in the robot's environment. Graphical abstractDisplay Omitted Highlights? This paper analyses the performance of laser range finder for object recognition. ? Comparison between laser range finder and CCD camera approach. ? Object classification performed by using machine learning techniques. ? The proposed technique proves to be superior in terms of robustness and reliability.
---
paper_title: Pose-variant face recognition based on an improved Lucas-Kanade algorithm

paper_content:

In this paper, a pose-variant face recognition system is presented for study of human-robot interaction design. An iterative fitting algorithm is proposed to extract feature-point positions based on active appearance model (AAM). Comparing with the traditional Lucas-Kanade algorithm, the proposed iterative algorithm improves the capability of correct convergence as a larger variation of head posture occurs. After obtaining the location of feature points, the dimension of texture model is reduced and then sent to a back propagation neural network (BPNN) to recognize family-members. The proposed pose-variant face recognition system has been implemented on an embedded image processing system and integrated with a pet robot. Experimental results show that the robot can interact with a person in a responding manner. Tested with the database from UMIST and the database built in the lab, the proposed method achieved average recognition rates of 91.0% and 95.6% respectively.
---
paper_title: Automatic Tool Landmark Detection for Stereo Vision in Robot-Assisted Retinal Surgery

paper_content:

Computer vision and robotics are being increasingly applied in medical interventions. Especially in interventions where extreme precision is required they could make a difference. One such application is robot-assisted retinal microsurgery. In recent works, such interventions are conducted under a stereo-microscope, and with a robot-controlled surgical tool. The complementarity of computer vision and robotics has however not yet been fully exploited. In order to improve the robot control we are interested in 3D reconstruction of the anatomy and in automatic tool localization using a stereo microscope. In this paper, we solve this problem for the first time using a single pipeline, starting from uncalibrated cameras to reach metric 3D reconstruction and registration, in retinal microsurgery. The key ingredients of our method are: (a) surgical tool landmark detection, and (b) 3D reconstruction with the stereo microscope, using the detected landmarks. To address the former, we propose a novel deep learning method that detects and recognizes keypoints in high definition images at higher than real-time speed. We use the detected 2D keypoints along with their corresponding 3D coordinates obtained from the robot sensors to calibrate the stereo microscope using an affine projection model. We design an online 3D reconstruction pipeline that makes use of smoothness constraints and performs robot-to-camera registration. The entire pipeline is extensively validated on open-sky porcine eye sequences. Quantitative and qualitative results are presented for all steps.
---
paper_title: Visual Object Recognition and Pose Estimation Based on a Deep Semantic Segmentation Network

paper_content:

In recent years, deep learning-based object recognition algorithms become emerging in robotic vision applications. This paper addresses the design of a novel deep learning-based visual object recognition and pose estimation system for a robot manipulator to handle random object picking tasks. The proposed visual control system consists of a visual perception module, an object pose estimation module, a data argumentation module, and a robot manipulator controller. The visual perception module combines deep convolution neural networks (CNNs) and a fully connected conditional random field layer to realize an image semantic segmentation function, which can provide stable and accurate object classification results in cluttered environments. The object pose estimation module implements a model-based pose estimation method to estimate the 3D pose of the target for picking control. In addition, the proposed data argumentation module automatically generates training data for training the deep CNN. Experimental results show that the proposed scene segmentation method used in the data argumentation module reaches a high accuracy rate of 97.10% on average, which is higher than other state-of-the-art segment methods. Moreover, with the proposed data argumentation module, the visual perception module reaches an accuracy rate over than 80% and 72% in the case of detecting and recognizing one object and three objects, respectively. In addition, the proposed model-based pose estimation method provides accurate 3D pose estimation results. The average translation and rotation errors in the three axes are all smaller than 0.52 cm and 3.95 degrees, respectively. These advantages make the proposed visual control system suitable for applications of random object picking and manipulation.
---
paper_title: Action Recognition with 3D ConvNet-GRU Architecture

paper_content:

Video action recognition is widely applied in video indexing, intelligent surveil-lance, multimedia understanding, and other fields. Recently, it was greatly improved by incorporating the learning of deep information using convolutional neural network (ConvNet). In this paper, we proposed a 3D ConvNet-GRU architecture to learn deep information for action recognition. Specifically, we use 3D ConvNet to learn spatiotemporal information from short RGB clips and optical flow clips, and impose gated recurrent unit (GRU) on the spatiotemporal in-formation to model the temporal evolution for action recognition. The experimental results show that our 3D ConvNet-GRU method is effective to model temporal evolution for action and achieves recognition performance comparable to that of state-of-the-art methods.
---
paper_title: Adaptive Deep Learning Through Visual Domain Localization

paper_content:

A commercial robot, trained by its manufacturer to recognize a predefined number and type of objects, might be used in many settings, that will in general differ in their illumination conditions, background, type and degree of clutter, and so on. Recent computer vision works tackle this generalization issue through domain adaptation methods, assuming as source the visual domain where the system is trained and as target the domain of deployment. All approaches assume to have access to images from all classes of the target during training, an unrealistic condition in robotics applications. We address this issue proposing an algorithm that takes into account the specific needs of robot vision. Our intuition is that the nature of the domain shift experienced mostly in robotics is local. We exploit this through the learning of maps that spatially ground the domain and quantify the degree of shift, embedded into an end-to-end deep domain adaptation architecture. By explicitly localizing the roots of the domain shift we significantly reduce the number of parameters of the architecture to tune, we gain the flexibility necessary to deal with subset of categories in the target domain at training time, and we provide a clear feedback on the rationale behind any classification decision, which can be exploited in human-robot interactions. Experiments on two different settings of the iCub World database confirm the suitability of our method for robot vision.
---
paper_title: An indirect adaptive neural control of a visual-based quadrotor robot for pursuing a moving target.

paper_content:

This paper aims to use a visual-based control mechanism to control a quadrotor type aerial robot which is in pursuit of a moving target. The nonlinear nature of a quadrotor, on the one hand, and the difficulty of obtaining an exact model for it, on the other hand, constitute two serious challenges in designing a controller for this UAV. A potential solution for such problems is the use of intelligent control methods such as those that rely on artificial neural networks and other similar approaches. In addition to the two mentioned problems, another problem that emerges due to the moving nature of a target is the uncertainty that exists in the target image. By employing an artificial neural network with a Radial Basis Function (RBF) an indirect adaptive neural controller has been designed for a quadrotor robot in search of a moving target. The results of the simulation for different paths show that the quadrotor has efficiently tracked the moving target.
---
paper_title: Door and cabinet recognition using Convolutional Neural Nets and real-time method fusion for handle detection and grasping

paper_content:

In this paper we present a new method that robustly identifies doors, cabinets and their respective handles, with special emphasis on extracting useful features from handles to be then manipulated. The novelty of this system relies on the combination of a Convolutional Neural Net (CNN), as a form of reducing the search space, several methods to extract point cloud data and a mobile robot to interact with the objects. The framework consists of the following components: The implementation of a CNN to extract a Region of Interest (ROI) from an image corresponding to a door or cabinet. Several vision based techniques to detect handles inside the ROI and its 3D positioning. A complementary plane segmentation method to differentiate door/cabinet from the handle. An algorithm to fuse both approaches robustly and extract essential information from the handle for robotic grasping (i.e. handle point cloud, door plane model, grasping locations, turning orientation, orthogonal vector to door). A mobile robot for grasping the handle. The system assumes no prior knowledge of the environment.
---
paper_title: DPC-Net: Deep Pose Correction for Visual Localization

paper_content:

We present a novel method to fuse the power of deep networks with the computational efficiency of geometric and probabilistic localization algorithms. In contrast to other methods that completely replace a classical visual estimator with a deep network, we propose an approach that uses a convolutional neural network to learn difficult-to-model corrections to the estimator from ground-truth training data. To this end, we derive a novel loss function for learning SE(3) corrections based on a matrix Lie groups approach, with a natural formulation for balancing translation and rotation errors. We use this loss to train a deep pose correction network (DPC-Net) that predicts corrections for a particular estimator, sensor and environment. Using the KITTI odometry dataset, we demonstrate significant improvements to the accuracy of a computationally-efficient sparse stereo visual odometry pipeline, that render it as accurate as a modern computationally-intensive dense estimator. Further, we show how DPC-Net can be used to mitigate the effect of poorly calibrated lens distortion parameters.
---
paper_title: Applications of Artificial Intelligence in Safe Human–Robot Interactions

paper_content:

The integration of industrial robots into the human workspace presents a set of unique challenges. This paper introduces a new sensory system for modeling, tracking, and predicting human motions within a robot workspace. A reactive control scheme to modify a robot's operations for accommodating the presence of the human within the robot workspace is also presented. To this end, a special class of artificial neural networks, namely, self-organizing maps (SOMs), is employed for obtaining a superquadric-based model of the human. The SOM network receives information of the human's footprints from the sensory system and infers necessary data for rendering the human model. The model is then used in order to assess the danger of the robot operations based on the measured as well as predicted human motions. This is followed by the introduction of a new reactive control scheme that results in the least interferences between the human and robot operations. The approach enables the robot to foresee an upcoming danger and take preventive actions before the danger becomes imminent. Simulation and experimental results are presented in order to validate the effectiveness of the proposed method.
---
paper_title: Sensor substitution for video-based action recognition

paper_content:

There are many applications where domain-specific sensing, such as accelerometers, kinematics, or force sensing, provide unique and important information for control or for analysis of motion. However, it is not always the case that these sensors can be deployed or accessed beyond laboratory environments. For example, it is possible to instrument humans or robots to measure motion in the laboratory in ways that it is not possible to replicate in the wild. An alternative, which we explore in this paper, is to address situations where accurate sensing is available while training an algorithm, but for which only video is available for deployment. We present two examples of this sensory substitution methodology. The first variation trains a convolutional neural network to regress real-valued signals, including robot end-effector pose, from video. The second example regresses binary signals derived from accelerometer data which signifies when specific objects are in motion. We evaluate these on the JIGSAWS dataset for robotic surgery training assessment and the 50 Salads dataset for modeling complex structured cooking tasks. We evaluate the trained models for video-based action recognition and show that the trained models provide information that is comparable to the sensory signals they replace.
---
paper_title: Optical character recognition using deep recurrent attention model

paper_content:

We address the problem of recognizing multi-digit numbers in optical character images. Classical approaches to solve this problem include separate localization, segmentation and recognition steps. In this paper, an integrated approach to multi-digit recognition from raw pixels to ultimate multi class labeling is proposed by using recurrent attention model based on a spatial transformer model equipped with LSTM to localize digits individually and a subsequent deep convolutional neural network for actual recognition. The proposed method is evaluated on the publicly available SVHN dataset where it achieves enhanced recognition accuracy per individual digits and in complete street numbers. Further evaluation on the cluttered MNIST dataset shows results comparable to state of the art techniques but with simple and computationally effective network architecture.
---
paper_title: A robotic welding system using image processing techniques and a CAD model to provide information to a multi‐intelligent decision module

paper_content:

Purpose – The paper aims to propose a system that uses a combination of techniques to suggest weld requirements for ships parts. These suggestions are evaluated, decisions are made and then weld parameters are sent to a program generator.Design/methodology/approach – A pattern recognition system recognizes shipbuilding parts using shape contour information. Fourier‐descriptors provide information and neural networks make decisions about shapes.Findings – The system has distinguished between various parts and programs have been generated so that the methods have proved to be valid approaches.Practical implications – The new system used a rudimentary curvature metric that measured Euclidean distance between two points in a window but the improved accuracy and ease of implementation can benefit other applications concerning curve approximation, node tracing, and image processing, but especially in identifying images of manufactured parts with distinct corners.Originality/value – A new proposed system has bee...
---
paper_title: End edge feedback and RBF neural network based vibration control of flexible manipulator

paper_content:

A vision feedback based active vibration control system for flexible manipulator is proposed. Active vibration control is conducted by employing Radial Basis Function (RBF) neural networks to regulate the control parameters of the proportional and derivative (PD) controller. Firstly, the composition of the vision feedback based piezoelectric flexible manipulator system is described. Furthermore, vision based vibration measurement for the flexible manipulator's end side and image processing methods are introduced. Secondly, the system dynamics model is derived by using finite element method, and the equation of the state space representation for the lower mode vibration is provided. Then, the vision feedback based control method is expounded, using RBF neural networks to tune the control parameters dynamically. Finally, simulation and experiments are conducted. The results demonstrate that vision feedback based control system and the RBF neural networks control algorithm can suppress the vibration of the flexible manipulator effectively.
---
paper_title: You Only Look Once: Unified, Real-Time Object Detection

paper_content:

We present YOLO, a new approach to object detection. Prior work on object detection repurposes classifiers to perform detection. Instead, we frame object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. A single neural network predicts bounding boxes and class probabilities directly from full images in one evaluation. Since the whole detection pipeline is a single network, it can be optimized end-to-end directly on detection performance. ::: Our unified architecture is extremely fast. Our base YOLO model processes images in real-time at 45 frames per second. A smaller version of the network, Fast YOLO, processes an astounding 155 frames per second while still achieving double the mAP of other real-time detectors. Compared to state-of-the-art detection systems, YOLO makes more localization errors but is far less likely to predict false detections where nothing exists. Finally, YOLO learns very general representations of objects. It outperforms all other detection methods, including DPM and R-CNN, by a wide margin when generalizing from natural images to artwork on both the Picasso Dataset and the People-Art Dataset.
---
paper_title: A machine vision based pistachio sorting using transferred mid-level image representation of Convolutional Neural Network

paper_content:

Convolutional neural networks have proved to be prominent in various fields of machine vision and image classification. Although it necessitates a large-scale dataset for promising performance, the mid-level representation of these networks can be exploited for specified tasks with smaller annotated image dataset. To this end, by evaluating the generality-specificity of the desired layer as a feature extractor layer, the parameters of Convolutional Neural Networks learned on massive-size dataset like ImageNet can be transferred to a new model. In this study, the images of different sort of pistachios including trashes have been acquired to feed into a new model using a support vector classifier. The ultimate goal of our machine vision system is to separate the desired open-shell pistachios from other defected pistachios as well as trashes. For image segmentation, we use active contour method to detect objects and form both new images of each object type and their augmented images. Since our dataset is not large-scale compared to ImageNet classes, a feature reduction method is performed after the feature extractor layer of pre-trained Convolutional Neural Network. The results show the better performance of the proposed approach in detection of desired-formed pistachio facing unseen test set of images compared to basic approaches.
---
paper_title: Visualizing and Understanding Convolutional Networks

paper_content:

Large Convolutional Network models have recently demonstrated impressive classification performance on the ImageNet benchmark Krizhevsky et al. [18]. However there is no clear understanding of why they perform so well, or how they might be improved. In this paper we explore both issues. We introduce a novel visualization technique that gives insight into the function of intermediate feature layers and the operation of the classifier. Used in a diagnostic role, these visualizations allow us to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. We also perform an ablation study to discover the performance contribution from different model layers. We show our ImageNet model generalizes well to other datasets: when the softmax classifier is retrained, it convincingly beats the current state-of-the-art results on Caltech-101 and Caltech-256 datasets.
---
paper_title: Door recognition and deep learning algorithm for visual based robot navigation

paper_content:

In this paper, a new method based on deep learning for robotics autonomous navigation is presented. Different from the most traditional methods based on fixed models, a convolutional neural network (CNN) modelling technique in Deep learning is selected to extract the feature inspired by the working pattern of the biological brain. This neural network model has muti-layer features where the ambient scenes can be recognized and useful information such as the location of door can be identified. The extracted information can be used for robot navigation, so does the robot can approach the target accurately. In the field experiments, detecting doors and predicting the door poses such tasks are designed in the indoor environment to verify the proposed method. The experimental results demonstrate that the doors can be identified with good performance and the deep learning model is suitable for robot navigation.
---
paper_title: Kiwifruit detection in field images using Faster R-CNN with ZFNet

paper_content:

Abstract A kiwifruit detection system for field images was developed based on the deep convolutional neural network, which has a good robustness against the subjectivity and limitation of the features selected artificially. Under different lighting conditions, 2,100 sub-images with 784×784 pixels were prepared by random sub-sampling from 700 field captured images with a pixel resolution of 2352×1568 pixels. Sub-images were used as network training and validation samples. A faster R-CNN was trained end-to-end by using back-propagation and stochastic gradient descent techniques with Zeiler and Fergus network (ZFNet). The average precision of the Faster R-CNN-based kiwifruit detector was 89.3%. Finally, another 100 images of kiwifruit canopies in the field environment (including 5,918 fruits) were used for testing the network. The test results showed that the recognition ratio of occluded fruit, overlapping fruit, adjacent fruit and separated fruit were 82.5%, 85.6%, 94.3% and 96.7%, respectively. Overall, the model reached a recognition rate of 92.3%. The technique took 0.274 s to process each image (for images with 2352×1568 pixels) and only 5.0 ms on average to detect a fruit. Comparing against the conventional methods, it suggested that the proposed method has higher recognition rate and faster speed. Especially, the proposed technique was able to simultaneously detect individual kiwifruit in clusters, which provides a promise for accurate yield mapping and multi-arm robotic harvesting.
---
paper_title: ImageNet: A large-scale hierarchical image database

paper_content:

The explosion of image data on the Internet has the potential to foster more sophisticated and robust models and algorithms to index, retrieve, organize and interact with images and multimedia data. But exactly how such data can be harnessed and organized remains a critical problem. We introduce here a new database called “ImageNet”, a large-scale ontology of images built upon the backbone of the WordNet structure. ImageNet aims to populate the majority of the 80,000 synsets of WordNet with an average of 500-1000 clean and full resolution images. This will result in tens of millions of annotated images organized by the semantic hierarchy of WordNet. This paper offers a detailed analysis of ImageNet in its current state: 12 subtrees with 5247 synsets and 3.2 million images in total. We show that ImageNet is much larger in scale and diversity and much more accurate than the current image datasets. Constructing such a large-scale database is a challenging task. We describe the data collection scheme with Amazon Mechanical Turk. Lastly, we illustrate the usefulness of ImageNet through three simple applications in object recognition, image classification and automatic object clustering. We hope that the scale, accuracy, diversity and hierarchical structure of ImageNet can offer unparalleled opportunities to researchers in the computer vision community and beyond.
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: Application of sensory body schemas to path planning for micro air vehicles (MAVs)

paper_content:

To date, most autonomous micro air vehicles (MAV-s) operate in a controlled environment, where the location ::: ::: of and attitude of the aircraft are measured be dedicated high-power computers with IR tracking capability. If ::: ::: MAV-s are to ever exit the lab and carry out autonomous missions, their flight control systems needs to utilize ::: ::: on-board sensors and high-efficiency attitude determination algorithms. To address this need, we investigate ::: ::: the feasibility of using body schemas to carry out path planning in the vision space of the MAV. Body schemas ::: ::: are a biologically-inspired approach, emulating the plasticity of the animal brains, allowing efficient representation ::: ::: of non-linear mapping between the body configuration space, i.e. its generalized coordinates and the ::: ::: resulting sensory outputs. This paper presents a numerical experiment of generating landing trajectories of ::: ::: a miniature rotor-craft using the notion of body and image schemas. More specifically, we demonstrate how ::: ::: a trajectory planning can be executed in the image space using a pseudo-potential functions and a gradient-based ::: ::: maximum seeking algorithm. It is demonstrated that a neural-gas type neural network, trained through ::: ::: Hebbian-type learning algorithm can learn a mapping between the rotor-craft position/attitude and the output ::: ::: of its vision sensors. Numerical simulations of the landing performance of a physical model is also presented, ::: ::: The resulting trajectory tracking errors are less than 8 %.
---
paper_title: Evolving Head Tracking Routines With Brain Programming

paper_content:

Visual tracking has long been studied in computer vision, and it has many practical applications such as surveillance and security, traffic control, human–computer interaction, and activity or behavior recognition to mention but a few. Head tracking attempts to follow a person’s head within a video sequence. This paper presents a methodology that automatically designs an artificial dorsal stream for the problem of head tracking. Multiple visual operators are synthesized to obtain several visual and conspicuity maps that are fused into a saliency map, which is converted to a binary image, thus defining the proto-object. The automatic design of visual attention programs for the problem of head tracking is achieved through an optimization process following the Darwinian paradigm of artificial evolution. Artificial brains are synthesized using multiple visual operators embedded within a complex hierarchical procedure consisting of several key processes such as center-surround mechanisms, normalization, and pyramid-scale processes. The proposed strategy robustly handles many difficulties including occlusion, distraction, and illumination, and the resulting programs are real-time systems that are able to track a person’s head with enough accuracy to automatically control the camera. Extensive experimentation shows that the proposed method outperforms several state-of-the-art methods in the challenging problem of head tracking.
---
paper_title: Deep learning based smart radar vision system for object recognition

paper_content:

The capability of manipulating objects based on vision system are essential to robotic applications. For a vision system, high accuracy and robustness is vital. However, due to external environment and interference influences, as well as to lighting conditions influences, the most existing image-based vision system are limited in applications. In this context, a novel and robust data-driven vision system for recognizing different types of objects in various external situations has been presented in this paper. The system consists of three parts, namely, integrated module based on X2 Ultra-wideband (UWB) radar, deep learning (DL) module and robot controller. Specifically, the introduction of X2 UWB radar aims to (1) capture and process radar echo signals of different objects in various environments; (2) extract Range-Doppler power spectrograms from the processed signals accurately. The second part combines the DL algorithm to take the spectrograms as the input from the former to realize object classification. And the last part is used to control the behavior of the robot based on the classification results. The experimental results show that the proposed vision system achieves very reliable performance in recognizing six types of objects in four kinds of external environment and lighting conditions with three deep learning network models. Besides, by experimental data comparison, the performance of the proposed system achieves higher than that of the image-based system in the same poor conditions and ResNet is the optimal model for our proposed system in all conditions.
---
paper_title: Active Vision via Extremum Seeking for Robots in Unstructured Environments: Applications in Object Recognition and Manipulation

paper_content:

In this paper, a novel active vision strategy is proposed for optimizing the viewpoint of a robot’s vision sensor for a given success criterion. The strategy is based on extremum seeking control (ESC), which introduces two main advantages: 1) Our approach is model free: It does not require an explicit objective function or any other task model to calculate the gradient direction for viewpoint optimization. This brings new possibilities for the use of active vision in unstructured environments, since a priori knowledge of the surroundings and the target objects is not required. 2) ESC conducts continuous optimization backed up with mechanisms to escape from local maxima. This enables an efficient execution of an active vision task. We demonstrate our approach with two applications in the object recognition and manipulation fields, where the model-free approach brings various benefits: for object recognition, our framework removes the dependence on offline training data for viewpoint optimization, and provides robustness of the system to occlusions and changing lighting conditions. In object manipulation, the model-free approach allows us to increase the success rate of a grasp synthesis algorithm without the need of an object model; the algorithm only uses continuous measurements of the objective value, i.e., the grasp quality. Our experiments show that continuous viewpoint optimization can efficiently increase the data quality for the underlying algorithm, while maintaining the robustness. Note to Practitioners —Vision sensors provide robots flexibility and robustness both in industrial and domestic settings by supplying required data to analyze the surroundings and the state of the task. However, the quality of these data can be very high or poor depending on the viewing angle of the vision sensor. For example, if the robot aims to recognize an object, images taken from certain angles (e.g., feature rich surfaces) can be more descriptive than the others, or if the robot’s goal is to manipulate an object, observing it from a viewpoint that reveals easy-to-grasp “handles” makes the task simpler to execute. The algorithm presented in this paper aims to provide the robot high quality visual data relative to the task at hand by changing vision sensors’ viewpoint. Different from other methods in the literature, our method does not require any task models (therefore, it is model free), and only utilizes a quality value that can be measured from the current viewpoint (e.g., object recognition success rate for the current image). The viewpoint of the sensor is changed continuously for increasing the quality value until the robot is confident enough about the success of the execution. We demonstrate the application of the algorithm in the object recognition and manipulation domains. Nevertheless, it can be applied to many other robotics tasks, where viewing angle of the scene affects the robot’s performance.
---
paper_title: Flame detection using deep learning

paper_content:

Flame detection is an increasingly important issue in intelligent surveillance. In fire flame detection, we need to extract visual features from video frames for training and test. Based on them, a group of shallow learning models have been developed to detect flames, such as color-based model, fuzzy-based model, motion and shape-based model, etc. Deep learning is a novel method which could be much efficient and accurate in flame detection. In this paper, we use YOLO model to implement flame detection and compare it with those shallow learning methods so as to determine the most efficient one for flame detection. Our contribution of this paper is to make use of the optimized YOLO model for flame detection from video frames. We collected the dataset and trained them using Google platform TensorFlow, the obtained accuracy of our proposed flame detection is up to 76%.
---
paper_title: Auto-Sorting System Toward Smart Factory Based on Deep Learning for Image Segmentation

paper_content:

Machine part sorting is important and monotonous in smart factory. In this paper, an auto-sorting system is proposed based on the deep learning method. In the proposed system, an industrial objection detection network combined with a robotic arm controlling system is designed to automatically and efficiently complete machine part sorting. Region-based full convolutional network (R-FCN) is applied for locating and recognizing different types of images of industrial object models. After comparison and simulation analysis, it illustrated that the R-FCN model trained with enough labeled data can efficiently and accurately recognize the object from the images captured by visual sensors. Furthermore, with enough data, the network can be robust to view angle rotation both vertically and horizontally, and a small part of overlapping of object will not mislead the judgment of the network in most situations. The case study results illustrate that the position and type of objects can be successfully detected. The code will be available publicly at https://github.com/tianwangbuaa/ .
---

Title: Machine Vision in the Context of Robotics: A Systematic Literature Review

Section 1: INTRODUCTION
Description 1: Introduce the growing importance of robotics and the role of machine vision in enabling intelligent robots, outlining the objectives and advantages of conducting a systematic literature review.

Section 2: RESEARCH METHODOLOGY
Description 2: Describe the methodology used for performing the systematic literature review, including the review protocol, research questions, search process, inclusion and exclusion criteria, quality assessment, data collection, and data analysis.

Section 3: RESULTS
Description 3: Present the analyzed data from the reviewed papers, including metadata such as publication year and form, measured quality, fields of research, practicability, and detailed observations about common development structures in the field.

Section 4: CONCLUSION
Description 4: Summarize the findings of the literature review, discuss potential future research directions, and address challenges such as balance between efficiency and accuracy, robustness, computation time, and environmental factors like occlusion and lighting variance.