Survey of Computer Vision and Machine Learning in Gastrointestinal Endoscopy
---
paper_title: Computer-assisted Diagnosis for Precancerous Lesions in the Esophagus

paper_content:

Objectives: The interpretation of endoscopic findings by gastroenterologists is still a difficult and highly subjective task. Despite important developments such as chromo-endoscopy, pit pattern analysis, fluorescence imaging as well as narrow band imaging it still requires lots of experience and training with a certain tentativeness until the final biopsy. By the development of computer-assisted diagnosis (CAD) systems this process can be supported. Methods: This paper presents a new approach to CAD for precancerous lesions in the esophagus based on color-texture analysis in a content-based image retrieval (CBIR) framework. The novelty of our approach lies in the combination of newly developed color-texture features with the interactive feedback loop provided by a relevance feedback algorithm. This allows the expert to steer the query and is still robust against accidental false decisions. Results: We reached an inter-rater reliability of k = 0.71 on a database of 390 endoscopic images. The retrieval accuracy didn’t change significantly until a wrong decision rate of 20%. Conclusions: Thus, the system could be able to support practitioners with less experience or in private practice. In combination with a connected case database it can also support case-based reasoning for the diagnostic decision process.
---
paper_title: An automatic bleeding detection scheme in wireless capsule endoscopy based on statistical features in hue space

paper_content:

Wireless capsule endoscopy (WCE) is a recently developed video technology to detect small intestine diseases, such as bleeding. For analyzing WCE video frames, instead of using the most common RGB (red, green, blue) color scheme, in this paper, HSV (hue, saturation, intensity value) color scheme is used, which corresponds better to human perception system. The HSV color scheme exhibits less sensitivity to illumination changes, which helps in handling the problem of illumination variation in WCE videos due to the weakening of battery. Different statistical features computed from H, S, and V spaces of WCE images are investigated and it is found that hue provides a useful feature as it captures intrinsic information about the color of objects or surfaces in a scene. Hence in this paper, an automatic bleeding detection scheme from WCE video is proposed utilizing the hue space. Among different statistical measures, mean, standard deviation, variance and moment exhibit significantly distinguishable characteristics for bleeding and non-bleeding images. For the purpose of classification, K-nearest neighbor (KNN) classifier is employed. From extensive experimentation on several WCE videos collected from a publicly available database, it is observed that the bleeding detection performance of the proposed method in terms of accuracy, sensitivity and specificity is quite satisfactory in comparison to that obtained by some of the existing methods.
---
paper_title: SVM based colon polyps classifier in a wireless active stereo endoscope

paper_content:

This work focuses on the recognition of three-dimensional colon polyps captured by an active stereo vision sensor. The detection algorithm consists of SVM classifier trained on robust feature descriptors. The study is related to Cyclope, this prototype sensor allows real time 3D object reconstruction and continues to be optimized technically to improve its classification task by differentiation between hyperplastic and adenomatous polyps. Experimental results were encouraging and show correct classification rate of approximately 97%. The work contains detailed statistics about the detection rate and the computing complexity. Inspired by intensity histogram, the work shows a new approach that extracts a set of features based on depth histogram and combines stereo measurement with SVM classifiers to correctly classify benign and malignant polyps.
---
paper_title: Lacunarity-Based Inherent Texture Correlation Approach for Wireless Capsule Endoscopy Image Analysis

paper_content:

Wireless capsule endoscopy (WCE) is a novel technology that offers non-invasive visual inspection of the digestive tract (DT) and especially small bowel. However, the revision of the large amount of images produced is highly time-consuming and prone to human error. This weakness was the rationale to propose a novel strategy for automatic detection of WCE images related to ulcer, one of the most common findings of DT. This paper introduces a new texture extraction method based on the inherent texture correlation measure and lacunarity index. WCE data are pre-processed with Bidimensional Ensemble Empirical Mode Decomposition to reveal their congenital structure primitives, and also to adaptively reconstruct new refined images. Classification results demonstrated promising classification accuracy (89.1%) exhibiting high potential towards further research in this field.
---
paper_title: Computer-aided Diagnosis for Internal Hemorrhoids by Measuring the Congestive Extent in Endoscopic Images

paper_content:

This paper presents a system of computer-aided diagnosis for internal hemorrhoids based on the congestive extent using endoscopic images. This system could be effective for young or even general practitioners as a second opinion. In the proposed method, a pre-processing is conducted to the images for enhancing saturation and contrast of congestive regions and blood vessels. Next, considering characteristics of internal hemorrhoids, the proposed method measures degree of the congestion in the images, and extracts the congestive region and abnormalities on the congestive extent. Experimental results of the discrimination using the proposed abnormalities between normal and abnormal cases for 204 images including 108 abnormal cases have shown that the abnormalities are well effective to diagnose the congestion in the hemorrhoids.
---
paper_title: A machine learning framework using SOMs: Applications in the intestinal motility assessment

paper_content:

Small Bowel Motility Assessment by means of Wireless Capsule Video Endoscopy constitutes a novel clinical methodology in which a capsule with a micro-camera attached to it is swallowed by the patient, emitting a RF signal which is recorded as a video of its trip throughout the gut. In order to overcome the main drawbacks associated with this technique -mainly related to the large amount of visualization time required-, our efforts have been focused on the development of a machine learning system, built up in sequential stages, which provides the specialists with the useful part of the video, rejecting those parts not valid for analysis. We successfully used Self Organized Maps in a general semi-supervised framework with the aim of tackling the different learning stages of our system. The analysis of the diverse types of images and the automatic detection of intestinal contractions is performed under the perspective of intestinal motility assessment in a clinical environment.
---
paper_title: Color and Position versus Texture Features for Endoscopic Polyp Detection

paper_content:

This paper presents a comparison of texture based and color and position based methods for polyp detection in endoscopic video images. Two methods for texture feature extraction that presented good results in previous studies were implemented and their performance is compared against a simple combination of color and position features. Although this more simple approach produces a much higher number of features than the other approaches, a SVM with a KBF kernel is able to deal with this high dimensional input space and it turns out that it outperforms the previous approaches on the experiments performed in a database of 4620 images from endoscopic video.
---
paper_title: Capsule endoscopy image analysis using texture information from various colour models

paper_content:

Wireless capsule endoscopy (WCE) is a novel imaging technique that is gradually gaining ground as it enables the non-invasive and efficacious visualization of the digestive track, and especially the entire small bowel including its middle part. However, the task of reviewing the vast amount of images produced by a WCE examination is a burden for the physicians. To tackle this major drawback, an innovative scheme for discriminating endoscopic images related to one of the most common intestinal diseases, ulceration, is presented here. This new approach focuses on colour-texture features in order to investigate how the structure information of healthy and abnormal tissue is distributed on RGB, HSV and CIE Lab colour spaces. The WCE images are pre-processed using bidimensional ensemble empirical mode decomposition so as to facilitate differential lacunarity analysis to extract the texture patterns of normal and ulcerous regions. Experimental results demonstrated promising classification performance (mean accuracy>95%), exhibiting a high potential towards automatic WCE image analysis.
---
paper_title: Eigenmotion-Based Detection of Intestinal Contractions

paper_content:

Intestinal contractions are one of the main features for analyzing intestinal motility and detecting different gastrointestinal pathologies. In this paper we propose Eigenmotion-based Contraction Detection (ECD), a novel approach for automatic annotation of intestinal contractions of video capsule endoscopy. Our approach extracts the main motion information of a set of contraction sequences in form of eigenmotions using Principal Component Analysis. Then, it uses a selection of them to represent the high dimension motion data. Finally, this contraction characterization is used to classify the contraction sequences by means of machine learning techniques. The experimental results show that motion information is useful in the contraction detection. Moreover, the proposed automatic method is essential to speed up the costly examination of the video capsule endoscopy.
---
paper_title: BP neural network classification for bleeding detection in wireless capsule endoscopy

paper_content:

Bleeding in the digestive tract is one of the most common gastrointestinal tract (GI) diseases, as well as the complication of some fatal diseases. Wireless capsule endoscopy (WCE) allows physicians to noninvasively examine the entire GI tract. However it is very laborious and time-consuming to inspect large numbers of WCE images, which limits the wider application of WCE. It is therefore important to develop an automatic and intelligent computer-aided bleeding detection technique. In this paper, a new method aimed at bleeding detection in WCE images is proposed. Colour texture features distinguishing the bleeding regions from non-bleeding regions are extracted in RGB and HSI colour spaces; then a neural network using the colour texture features as the feature vector inputs is designed to recognize the bleeding regions. The experiments demonstrate that the bleeding regions can be correctly recognized and clearly marked out. The sensitivity of the algorithm is 93% and the specificity is 96%.
---
paper_title: WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians

paper_content:

Abstract We introduce in this paper a novel polyp localization method for colonoscopy videos. Our method is based on a model of appearance for polyps which defines polyp boundaries in terms of valley information. We propose the integration of valley information in a robust way fostering complete, concave and continuous boundaries typically associated to polyps. This integration is done by using a window of radial sectors which accumulate valley information to create WM-DOVA (Window Median Depth of Valleys Accumulation) energy maps related with the likelihood of polyp presence. We perform a double validation of our maps, which include the introduction of two new databases, including the first, up to our knowledge, fully annotated database with clinical metadata associated. First we assess that the highest value corresponds with the location of the polyp in the image. Second, we show that WM-DOVA energy maps can be comparable with saliency maps obtained from physicians’ fixations obtained via an eye-tracker. Finally, we prove that our method outperforms state-of-the-art computational saliency results. Our method shows good performance, particularly for small polyps which are reported to be the main sources of polyp miss-rate, which indicates the potential applicability of our method in clinical practice.
---
paper_title: Computer-Aided Bleeding Detection in WCE Video

paper_content:

Wireless capsule endoscopy (WCE) can directly take digital images in the gastrointestinal tract of a patient. It has opened a new chapter in small intestine examination. However, a major problem associated with this technology is that too many images need to be manually examined by clinicians. Currently, there is no standard for capsule endoscopy image interpretation and classification. Most state-of-the-art CAD methods often suffer from poor performance, high computational cost, or multiple empirical thresholds. In this paper, a new method for rapid bleeding detection in the WCE video is proposed. We group pixels through superpixel segmentation to reduce the computational complexity while maintaining high diagnostic accuracy. Feature of each superpixel is extracted using the red ratio in RGB space and fed into support vector machine for classification. Also, the influence of edge pixels has been removed in this paper. Comparative experiments show that our algorithm is superior to the existing methods in terms of sensitivity, specificity, and accuracy.
---
paper_title: radial patterns characterization for automatic detection of tonic intestinal contractions

paper_content:

This work tackles the categorization of general linear radial patterns by means of the valleys and ridges detection and the use of descriptors of directional information, which are provided by steerable filters in different regions of the image. We successfully apply our proposal in the specific case of automatic detection of tonic contractions in video capsule endoscopy, which represent a paradigmatic example of linear radial patterns.
---
paper_title: Color eigen-subband features for endoscopy image classification

paper_content:

This paper presents a new image feature extraction approach in the wavelet domain. We incorporate color-channel information of the LAB color space into the feature extraction process by computing variances from decorrelated detail subbands of the stationary wavelet transform. We evaluate our approach on a medical image classification problem using a k-nearest neighbor classifier and sequential forward feature selection. our experimental results, which include a comparative study to the popular color wavelet energy correlation signatures show that we can produce highly discriminative feature sets in terms of leave-one-out classification accuracy.
---
paper_title: Polyp detection and radius measurement in small intestine using video capsule endoscopy

paper_content:

Video Capsule Endoscopy (VCE) was invented in the year 2000 and rapidly became one of the most popular noninvasive non-surgical inspection techniques in the diagnosis of gastrointestinal (GI) tract, especially in small intestine. A critical issue of capsule endoscopic examination is to determine the location and size of polyps. A critical problem associated with capsule endoscopic examination is to localize and correctly estimate the size of polyps for proper clinical treatment. In this paper we present a global statistical method which could automatically detect the polyps in VCE frames and determine their radii. The proposed method gathers the statistical information from available RGB channels. The statistical information is then fed to a support vector machine (SVM) to determine the existence and radii of polyps. The experimental result of this approach shows its improvement of accuracy when compared with other methods in the literature.
---
paper_title: Automatic hookworm image detection for wireless capsule endoscopy using hybrid color gradient and contourlet transform

paper_content:

Wireless Capsule Endoscopy (WCE) is a relative novel technology, which can view entire gastrointestinal (GI) tract without invasiveness and sedation. The main disadvantage associated with WCE is that the huge number of recorded images must be examined by clinicians. It is a tedious and time consuming task. Developing an automatic computer-aided detection system to alleviate the burden of clinicians is required. In this paper, we proposed a new hookworm image detection algorithm. A new gradient space, named Hybrid Color Gradient (HCG) for hookworm detection is developed by analyzing the characteristics of hookworm infection images. Contourlet transformation is introduced to construct the final features. Real experiments using SVM show that reasonable classification results can be obtained. Moreover, according to our literature survey, this is the first work on automatic hookworm detection of WCE images.
---
paper_title: Automated Diagnosis of Barrett’s Esophagus with Endoscopic Images

paper_content:

In this paper, we describe current progress on the development of a Computer Assisted Diagnosis System (CAD) for the classification of Barrett’s esophagus and associated neoplasia. Barrett’s esophagus is a condition in which normal squamous mucosa is replaced by columnar epithelium, which is similar to the lining of the intestine. Barrett’s esophagus as a known precancerous condition leading to esophageal cancer. Diagnosis is performed via histological analysis of tissue located during endoscopic examination. We compare four different automated classification tools (SVM, KNN, and Boosting) operating on three different imaging modalities (white light, narrow-band, and acetic acid chromoendoscopy) for lesion classification. Preliminary results suggest that narrow band imaging is more effective than either of the other two modalities for disease assessment.
---
paper_title: Identification of Intestinal Motility Events of Capsule Endoscopy Video Analysis

paper_content:

In this paper we introduce a system for assisting the analysis of capsule-endoscopy (CE) data, and identifying sequences of frames related to small intestine motility. The imbalanced recognition task of intestinal contractions was addressed by employing an efficient two-level video analysis system. At the first level, each video was processed resulting in a number of possible sequences of contractions. In the second level, the recognition of contractions was carried out by means of a SVM classifier. To encode patterns of intestinal motility a panel of textural and morphological features of the intestine lumen were extracted. The system exhibited an overall sensitivity of 73.53% in detecting contractions. The false alarm ratio was of the order of 59.92%. These results serve as a first step for developing assisting tools for computer based CE video analysis, reducing drastically the physician’s time spent in image evaluation and enhancing the diagnostic potential of CE examination.
---
paper_title: Towards automatic polyp detection with a polyp appearance model

paper_content:

This work aims at automatic polyp detection by using a model of polyp appearance in the context of the analysis of colonoscopy videos. Our method consists of three stages: region segmentation, region description and region classification. The performance of our region segmentation method guarantees that if a polyp is present in the image, it will be exclusively and totally contained in a single region. The output of the algorithm also defines which regions can be considered as non-informative. We define as our region descriptor the novel Sector Accumulation-Depth of Valleys Accumulation (SA-DOVA), which provides a necessary but not sufficient condition for the polyp presence. Finally, we classify our segmented regions according to the maximal values of the SA-DOVA descriptor. Our preliminary classification results are promising, especially when classifying those parts of the image that do not contain a polyp inside.
---
paper_title: Aligning endoluminal scene sequences in wireless capsule endoscopy

paper_content:

Intestinal motility analysis is an important examination in detection of various intestinal malfunctions. One of the big challenges of automatic motility analysis is how to compare sequence of images and extract dynamic paterns taking into account the high deformability of the intestine wall as well as the capsule motion. From clinical point of view the ability to align endoluminal scene sequences will help to find regions of similar intestinal activity and in this way will provide a valuable information on intestinal motility problems. This work, for first time, addresses the problem of aligning endoluminal sequences taking into account motion and structure of the intestine. To describe motility in the sequence, we propose different descriptors based on the Sift Flow algorithm, namely: (1) Histograms of Sift Flow Directions to describe the flow course, (2) Sift Descriptors to represent image intestine structure and (3) Sift Flow Magnitude to quantify intestine deformation. We show that the merge of all three descriptors provides robust information on sequence description in terms of motility. Moreover, we develop a novel methodology to rank the intestinal sequences based on the expert feedback about relevance of the results. The experimental results show that the selected descriptors are useful in the alignment and similarity description and the proposed method allows the analysis of the WCE.
---
paper_title: Intestinal Motility Assessment With Video Capsule Endoscopy: Automatic Annotation of Phasic Intestinal Contractions

paper_content:

Intestinal motility assessment with video capsule endoscopy arises as a novel and challenging clinical fieldwork. This technique is based on the analysis of the patterns of intestinal contractions shown in a video provided by an ingestible capsule with a wireless micro-camera. The manual labeling of all the motility events requires large amount of time for offline screening in search of findings with low prevalence, which turns this procedure currently unpractical. In this paper, we propose a machine learning system to automatically detect the phasic intestinal contractions in video capsule endoscopy, driving a useful but not feasible clinical routine into a feasible clinical procedure. Our proposal is based on a sequential design which involves the analysis of textural, color, and blob features together with SVM classifiers. Our approach tackles the reduction of the imbalance rate of data and allows the inclusion of domain knowledge as new stages in the cascade. We present a detailed analysis, both in a quantitative and a qualitative way, by providing several measures of performance and the assessment study of interobserver variability. Our system performs at 70% of sensitivity for individual detection, whilst obtaining equivalent patterns to those of the experts for density of contractions.
---
paper_title: Bleeding detection from wireless capsule endoscopy images using improved euler distance in CIELab

paper_content:

Bleeding in the digestive tract is one of the most common gastrointestinal (GI) tract diseases, as well as the complication of some fatal diseases. Wireless capsule endoscopy (WCE), which is widely applied in the clinical field, allows physicians to noninvasively examine the entire GI tract. However, it is very laborious and time-consuming to detect the huge amount of WCE images, and limits its wider application. It is urgent and necessary to develop the automatic and intelligent computer aided bleeding detection technique. This paper improves the Euler distance with the covariance matrix of image to measure the colour similarity in CIELab colorimetric system, and proposes a novel method of bleeding detection in WCE images. The experiments demonstrate that the bleeding region in WCE images can be correctly recognized and marked out, and the sensitivity of this method is 92%, the specificity is 95%.
---
paper_title: A new approach to detecting ulcer and bleeding in Wireless capsule endoscopy images

paper_content:

In recent years, Wireless capsule endoscopy (WCE) has been widely utilized in diagnosis of gastrointestinal (GI) tract disease. This new technology is painless and can see small intestine that traditional endoscopies cannot reach. However, Analysis of massive images for each WCE detection is tedious and time consuming to physicians. In this paper we present a computer-aid approach to help clinicians to discriminate amongst regions of normal or abnormal tissue. We use covariance of second-order statistical features which called as color wavelet covariance (CWC), based on discrete wavelet transform (DWT) and then optimize them by a selected algorithm. Accurate image segmentation and classification is achieved by a joint classifier, which is obtained by Texton Boost classifier. The whole approach has been validated on various WCE data and achieves a good result.
---
paper_title: Capsule endoscopy images classification by color texture and support vector machine

paper_content:

Capsule endoscopy (CE) is considered to be a state-of-the-art imaging modality for digest tract diseases detection, especially for small intestine, which is unreachable by traditional endoscopy techniques. However, the large number of images produced by the procedure, about 60,000 images for each examination, cause a time consuming and attention intensive task for physicians, necessitating the development of computer aided detection system. In this paper, we propose a computerized scheme to discriminate among normal CE images and CE images with three common diseases in GI tract, i.e., bleeding, ulcer and tumor. To achieve this goal, features related to color texture characteristics of CE images are extracted. Based on the features, multiclass support vector machine (SVM) built from binary SVM is further applied to separate different CE images. Preliminary experiments on our present image data demonstrate that it is promising to employ the proposed scheme built upon one-against-one binary SVM to differentiate different CE images.
---
paper_title: Polyp Detection in Endoscopic Video Using SVMs

paper_content:

Colon cancer is one of the most common cancers in developed countries. Most of these cancers start with a polyp. Polyps are easily detected by physicians. Our goal is to mimic this detection ability so that endoscopic videos can be pre-scanned with our algorithm before the physician analyses them. The method will indicate which part of the video needs attention (polyps were detected there) and hence can speedup the procedures. In this paper we present a method for polyp detection in endoscopic images that uses SVM for classification. Our experiments yielded a result of 93.16 ± 0.09% of area under the Receiver Operating Characteristic (ROC) curve on a database of 4620 images indicating that the approach proposed is well suited to the detection of polyps in endoscopic video.
---
paper_title: Color wavelet cross co-occurrence matrices for endoscopy image classification

paper_content:

In this work we propose a set of new wavelet-domain based color-texture features for the classification of zoom-endoscopy images in the field of medical imaging. We extend the concept of classic co-occurrence matrices to capture information between detail-subband pairs of different color channels. Our results show, that the proposed features outperform other wavelet-domain based color-texture features in terms of leave- one-out cross-validation accuracy.
---
paper_title: Gastroesophageal Reflux Disease Diagnosis Using Hierarchical Heterogeneous Descriptor Fusion Support Vector Machine

paper_content:

A new computer-aided diagnosis method is proposed to diagnose the gastroesophageal reflux disease (GERD) from endoscopic images of the esophageal-gastric junction. To avoid the interferences of different endoscope devices and automatic camera white balance adjustment, heterogeneous descriptors computed from heterogeneous color models are used to represent endoscopic images. Instead of concatenating these descriptors to a super vector, a hierarchical heterogeneous descriptor fusion support vector machine (HHDF-SVM) framework is proposed to simultaneously apply heterogeneous descriptors for GERD diagnosis and overcome the curse of dimensionality problem. During validation, heterogeneous descriptors are extracted from test endoscopic images at first. The classification result is obtained by using HHDF-SVM with heterogeneous descriptors. As shown in the experiments, our method can automatically diagnose GERD without any manual selection of region of interest and achieve better accuracy compared to states-of-the-art methods.
---
paper_title: Smart Videocapsule for Early Diagnosis of Colorectal Cancer: Toward Embedded Image Analysis

paper_content:

Wireless capsule endoscopy (WCE) enables screening of the gastrointestinal tract by a swallowable imaging system. However, contemporary WCE systems have several limitations—battery, low processing capabilities, among others—which often result in low diagnostic yield. In this chapter, after a technical presentation of the components of a standard WCE, the authors discuss the related limitations and introduce a new concept of smart capsule with embedded image processing capabilities based on a boosting approach using textural features. We discuss the feasibility of the hardware integration of the detection–recognition method, also with respect to the most recent FPGA technologies.
---
paper_title: Anisotropic Feature Extraction from Endoluminal Images for Detection of Intestinal Contractions

paper_content:

Wireless endoscopy is a very recent and at the same time unique technique allowing to visualize and study the occurrence of contractions and to analyze the intestine motility. Feature extraction is essential for getting efficient patterns to detect contractions in wireless video endoscopy of small intestine. We propose a novel method based on anisotropic image filtering and efficient statistical classification of contraction features. In particular, we apply the image gradient tensor for mining informative skeletons from the original image and a sequence of descriptors for capturing the characteristic pattern of contractions. Features extracted from the endoluminal images were evaluated in terms of their discriminatory ability in correct classifying images as either belonging to contractions or not. Classification was performed by means of a support vector machine classifier with a radial basis function kernel. Our classification rates gave sensitivity of the order of 90.84% and specificity of the order of 94.43% respectively. These preliminary results highlight the high efficiency of the selected descriptors and support the feasibility of the proposed method in assisting the automatic detection and analysis of contractions.
---
paper_title: Bleeding detection in wireless capsule endoscopy images based on binary feature vector

paper_content:

Wireless Capsule Endoscopy (WCE) is a non-invasive way which is getting its popularity in many hospitals for gastrointestinal examination. However, it produces too many images that need physicians to check manually. This is a huge burden for physicians. To solve this problem, an automatic method based on Support Vector Machine is proposed in this paper. A binary feature vector is used to overcome the drawbacks of conventional color histogram, which compares similarity between histograms rather than checks out the existence of a specified pattern. Considering the property of WCE images, a clipped illumination invariant color space is introduced. Experiments demonstrate that the binary feature vector is more effective than histograms in detecting bleeding patterns of WCE images.
---
paper_title: MPEG-7 Visual Descriptors&#8212;Contributions for Automated Feature Extraction in Capsule Endoscopy

paper_content:

Recent advances in miniaturization led to the development of what is now called the endoscopic capsule. This small device is swallowed by a patient and films the whole gastrointestinal tract, allowing the detection of abnormalities. Currently, a doctor typically needs up to two hours to analyze a full exam, so automation is desirable. This paper presents a methodology for measuring the potential of selected visual MPEG-7 descriptors for the task of specific medical event detection such as blood, ulcers. Experiments show that the best results are obtained by the Scalable Color and Homogenous Texture descriptors, especially if only relevant coefficients are used.
---
paper_title: Barrett's esophagus and esophageal cancer: an overview.

paper_content:

Although esophageal cancer (EC) is the eighth most common cancer in several European countries, it is one of deadliest worldwide. The most frequent predisposing factor implicated in its development is Barrett's esophagus (BE), an acquired metaplastic transformation of the esophageal lining cells from normal squamous epithelium into specialised or intestinal-like columnar epithelium. The major risk factor for BE is gastroesophageal reflux disease. Although BE is in itself a benign and often asymptomatic disorder, its clinical importance stems from the recognition that it represents the main precursor lesion for the development of esophageal adenocarcinoma (AC), a tumor that is rapidly increasing especially in developed countries and is associated with a low survival rate. This paper provides an overview of the epidemiology and natural history of BE as well as of the possible pathogenetic mechanisms underlying the development of BE and its progressive transition to AC. New diagnostic tests are described, recommendations for screening and surveillance are provided and surgical and ablative procedures to treat dysplastic lesions and early neoplasia are discussed. Claimed chemopreventive agents and biomarkers that in the near future may help identify people with a higher risk of EC are also considered.
---
paper_title: Detection of Wrinkle Frames in Endoluminal Videos Using Betweenness Centrality Measures for Images

paper_content:

Intestinal contractions are one of the most important events to diagnose motility pathologies of the small intestine. When visualized by wireless capsule endoscopy (WCE), the sequence of frames that represents a contraction is characterized by a clear wrinkle structure in the central frames that corresponds to the folding of the intestinal wall. In this paper, we present a new method to robustly detect wrinkle frames in full WCE videos by using a new mid-level image descriptor that is based on a centrality measure proposed for graphs. We present an extended validation, carried out in a very large database, that shows that the proposed method achieves state-of-the-art performance for this task.
---
paper_title: Sequential image analysis for computer-aided wireless endoscopy

paper_content:

Wireless Capsule Endoscopy (WCE) is a technique for inner-visualization of the entire small intestine and, thus, offers an interesting perspective on intestinal motility. The two major drawbacks of this technique are: 1) huge amount of data acquired by WCE makes the motility analysis tedious and 2) since the capsule is the first tool that offers complete inner-visualization of the small intestine, the exact importance of the observed events is still an open issue. Therefore, in this thesis, a novel computer-aided system for intestinal motility analysis is presented. The goal of the system is to provide an easily-comprehensible visual description of motility-related intestinal events to a physician. In order to do it, several tools based either on computer vision concepts or on machine learning techniques are presented. A method for transforming 3D video signal to a holistic image of intestinal motility, called motility bar, is proposed. The method calculates the optimal mapping from video into image from the intestinal motility point of view. To characterize intestinal motility, methods for automatic extraction of motility information from WCE are presented. Two of them are based on the motility bar and two of them are based on frame-per-frame analysis. In particular, four algorithms dealing with the problems of intestinal contraction detection, lumen size estimation, intestinal content characterization and wrinkle frame detection are proposed and validated. The results of the algorithms are converted into sequential features using an online statistical test. This test is designed to work with multivariate data streams. To this end, we propose a novel formulation of concentration inequality that is introduced into a robust adaptive windowing algorithm for multivariate data streams. The algorithm is used to obtain robust representation of segments with constant intestinal motility activity. The obtained sequential features are shown to be discriminative in the problem of abnormal motility characterization. Finally, we tackle the problem of efficient labeling. To this end, we incorporate active learning concepts to the problems present in WCE data and propose two approaches. The first one is based the concepts of sequential learning and the second one adapts the partition-based active learning to an error-free labeling scheme. All these steps are sufficient to provide an extensive visual description of intestinal motility that can be used by an expert as decision support system.
---
paper_title: Automated classification of colon polyps in endoscopic image data

paper_content:

Colon cancer is the third most commonly diagnosed type of cancer in the US. In recent years, however, early ::: diagnosis and treatment have caused a significant rise in the five year survival rate. Preventive screening is ::: often performed by colonoscopy (endoscopic inspection of the colon mucosa). Narrow Band Imaging (NBI) is ::: a novel diagnostic approach highlighting blood vessel structures on polyps which are an indicator for future ::: cancer risk. ::: In this paper, we review our automated inter- and intra-observer independent system for the automated ::: classification of polyps into hyperplasias and adenomas based on vessel structures to further improve the ::: classification performance. To surpass the performance limitations we derive a novel vessel segmentation approach, ::: extract 22 features to describe complex vessel topologies, and apply three feature selection strategies. ::: Tests are conducted on 286 NBI images with diagnostically important and challenging polyps (10mm ::: or smaller) taken from our representative polyp database. Evaluations are based on ground truth data ::: determined by histopathological analysis. Feature selection by Simulated Annealing yields the best result ::: with a prediction accuracy of 96.2% (sensitivity: 97.6%, specificity: 94.2%) using eight features. ::: Future development aims at implementing a demonstrator platform to begin clinical trials at University ::: Hospital Aachen.
---
paper_title: Computer-based detection of bleeding and ulcer in wireless capsule endoscopy images by chromaticity moments

paper_content:

The wireless capsule endoscopy (WCE) invented by Given Imaging has been gradually used in hospitals due to its great breakthrough that it can view the entire small bowel for gastrointestinal diseases. However, a tough problem associated with this new technology is that too many images to be examined by eyes cause a huge burden to physicians, so it is significant if we can help physicians do diagnosis using computerized methods. In this paper, a new method aimed for bleeding and ulcer detection in WCE images is proposed. This new approach mainly focuses on color feature, also a very powerful clue used by physicians for diagnosis, to judge the status of gastrointestinal tract. We propose a new idea of chromaticity moment as the features to discriminate normal regions and abnormal regions, which make full use of the Tchebichef polynomials and the illumination invariant of HSI color space, and we verify performances of the proposed features by employing neural network classifier. Experimental results on our present image data of bleeding and ulcer show that it is feasible to exploit the proposed chromaticity moments to detect bleeding and ulcer for WCE images.
---
paper_title: Multi-Directional Multi-Resolution Transforms for Zoom-Endoscopy Image Classification

paper_content:

In this paper, we evaluate the discriminative power of image features, extracted from subbands of the Gabor Wavelet Transform and the Dual-Tree Complex Wavelet Transform for the classification of zoom-endoscopy images. Further, we incorporate color channel information into the classification process and show, that this leads to superior classification results, compared to luminance-channel based image processing.
---
paper_title: Tumor detecting in colonoscopic narrow-band imaging data

paper_content:

In recent years, colorectal cancer is the most common type of cancer worldwide. However, this disease can be prevented by detection and removal of precursor adenomatous polyps during an optical colonoscopy course. In an optical colonoscopy course, the endoscopist looks for colon polyps. Hyperplastic polyp is benign lesion; adenomatous polyp is likely to become cancerous. Hence, it is common practice to remove all of the identified polyps and send them to subsequent histological analysis. But, removing hyperplastic polyps will pose unnecessary risk and incur unnecessary costs for histological analysis. In this paper, we pose a novel optical biopsy application based on narrow band imaging (NBI) in the first part. A common barrier to an automatic system is that require manual segmentations of the polyps in NBI data, therefore, we automatically segment polyps in NBI data. We propose an algorithm, Classification of Regional Feature (CoRF), that is an extension of the sparse matrix and vector quantization algorithms, a state of the art algorithm for feature detection and segmentation. CoRF solves the intrinsic block selection problem of vector quantization by including training codebook about the shape of the regional feature. CoRF outperforms previous methods with a better polyp region segmentation — traditional clustering algorithm, LBG or Kmean clustering algorithms will disperse the energy to other similar block regions, reducing accuracy of analysis in polyp detection.
---
paper_title: Supportive automatic annotation of early esophageal cancer using local gabor and color features

paper_content:

Over the past years High Definition (HD) endoscopy has become a crucial tool for the early detection of esophageal cancer. The high resolution offers specialist physicians high-quality visual information, enabling them to identify dysplastic tissue leading to Early Adenocarcinoma (EAC). The detection and removal of these early types of cancer drastically increases the survival chances of the patient. However, even for an experienced specialist it remains an arduous task to identify the patterns associated with early cancer. Therefore, a computer-aided detection system that supports the physician seems highly attractive. We present a novel algorithm for automatic detection of early cancerous tissue in HD endoscopic images. The algorithm computes local color- and texture features based on the original and on the Gabor-filtered image. We explore the spectral characteristics of the image regions that contain early cancer and we design appropriate filters based on this analysis. The features are classified by a trained Support Vector Machine (SVM) after which additional post-processing techniques are applied in order to annotate the image region containing early cancer. For 7 patients, we compare 32 annotations made by the algorithm with the corresponding delineations made by an expert gastroenterologist. Of 38 lesions indicated independently by the gastroenterologist, the system detects 36 of those lesions with a recall of 0.95 and a precision of 0.75.
---
paper_title: A new method for ulcer detection in endoscopic images

paper_content:

The entire visualization of GastroIntestinal (GI) tract is not possible with conventional endoscopic exams. Wireless Capsule Endoscopy (WCE) is a low risk, painless, noninvasive procedure for diagnosing diseases such as bleeding, polyps, ulcers, and Crohns disease within the human digestive tract, especially the small intestine that was unreachable using the traditional endoscopic methods. However, analysis of massive images of WCE detection is tedious and time consuming to physicians. Hence, researchers have developed software methods to detect these diseases automatically. Thus, the effectiveness of WCE can be improved. In this paper, a novel textural feature extraction method is proposed based on Contourlet transform and Log Gabor filter to distinguish ulcer regions from normal regions. The results show that the proposed method performs well with a high accuracy rate of 94.16% using Support Vector Machine (SVM) classifier in HSV colour space.
---
paper_title: Ulcer Detection in Wireless Capsule Endoscopy Images Using Bidimensional Nonlinear Analysis

paper_content:

Wireless Capsule Endoscopy (WCE) constitutes a recent technological breakthrough that enables the observation of the gastrointestinal tract (GT) and especially the entire small bowel in a non-invasive way compared to the traditional imaging techniques. WCE allows a detailed inspection of the intestine and identification of its clinical lesions. However, the main drawback of this method is the time consuming task of reviewing the vast amount of images produced. To address this, a novel technique for discriminating abnormal endoscopic images related to ulcer, the most common disease of GT, is presented here. Towards this direction, the Bidimensional Ensemble Empirical Mode Decomposition (BEEMD) was applied to images of the small bowel acquired by a WCE system in order to extract their Intrinsic Mode Functions (IMFs). The IMFs reveal differences in structure from their finest to their coarsest scale providing a new analysis domain. Additionally, lacunarity analysis was employed as a method to quantify and extract the texture patterns of the images and differentiate the ulcer from the healthy regions. Experimental results demonstrated promising classification accuracy (>90%), exhibiting a high potential towards WCE analysis.
---
paper_title: An automatic bleeding detection technique in wireless capsule endoscopy from region of interest

paper_content:

Wireless capsule endoscopy (WCE) is a painless operative video technology to detect small intestine diseases, such as bleeding. Instead of using the most common RGB (red, green, blue) color scheme, in this paper, YIQ (luminance-Y, chrominance-IQ: in phase-I and quadrature-Q) color scheme is used for analyzing WCE video frames, which corresponds better to human color response characteristics. Analyzing the behavior of each of the four YIQ spaces, first, a region of interest is determined depending on the Q value of the pixels and some morphological operations. Next, instead of considering three spaces of YIQ color model separately, a new composite space Y.I/Q is proposed to capture intrinsic information about the luminance and chrominance of images. Four statistical measures, namely mean, median, skewness and minima of the pixel values in composite space within the ROI are computed as features. It is exhibited that use of composite space lower computational complexity as well as offers noticeably better discrimination between bleeding and non-bleeding pixels. For the purpose of classification, support vector machine (SVM) classifier is employed. Satisfactory bleeding detection performance result is achieved in terms of accuracy, sensitivity and specificity from severe experimentation on several WCE videos which is collected from a publicly available database. Also it is observed that proposed method over performs with comparing some of the existing methods.
---
paper_title: Automatic Discrimination of Duodenum in Wireless Capsule Video Endoscopy

paper_content:

Wireless Capsule Video Endoscopy is a recent acquisition method providing an internal view of the gastrointestinal tract which is currently applied in a large quantity of methods for detecting different intestinal diseases. In some of these applications, the automatic identification of some regions of the small intestine is essential. However, the high amount of time needed for video visualization makes this task unfeasible. In this paper, we present a novel system for automatical labelling the transition from proximal to distal parts of the small bowel in the capsule endoscopy video based on textural descriptors. Results show an accuracy of the proximal-distal boundary detection of more than 70%.
---
paper_title: Hybrid magnetic and vision localization technique of capsule endoscope for 3D recovery of pathological tissues

paper_content:

Wireless capsule endoscope (WCE) is a miniature medical instrument for inspection of gastrointestinal tract non-invasively. Commercial available WCE only can provide monocular view, so it is different from what the physician see from the actual sites of the body, where clinically valuable depth, size and shape information can be perceived. The dimension of the lesion area is usually estimated by physician's experience with more or less inaccuracy. In order to precisely measure the dimension of the pathological tissues, 3D imaging techniques can be applied to recover the structure of the pathological tissues. In this paper, a hybrid tracking method of WCE motion is proposed, integrating magnetic sensing localization and image-based localization, which provide robust and accurate estimation of capsular camera motion for 3D reconstruction. And then a classical 3D reconstruction method based on stereo triangulation is performed using full 6D position and orientation information. The experiments on real data are presented to show the performance of our proposed method.
---
paper_title: Impact of Keypoint Detection on Graph-Based Characterization of Blood Vessels in Colonoscopy Videos

paper_content:

We explore the potential of the use of blood vessels as anatomical landmarks for developing image registration methods in colonoscopy images. An unequivocal representation of blood vessels could be used to guide follow-up methods to track lesions over different interventions. We propose a graph-based representation to characterize network structures, such as blood vessels, based on the use of intersections and endpoints. We present a study consisting of the assessment of the minimal performance a keypoint detector should achieve so that the structure can still be recognized. Experimental results prove that, even by achieving a loss of \(25\,\%\) of the keypoints, the descriptive power of the associated graphs to the vessel pattern is still high enough to recognize blood vessels.
---
paper_title: Learning discriminative local features from image-level labelled data for colonoscopy image classification

paper_content:

In this paper we propose a novel weakly-supervised feature learning approach, learning discriminative local features from image-level labelled data for image classification. Unlike existing feature learning approaches which assume that a set of additional data in the form of matching/non-matching pairs of local patches are given for learning the features, our approach only uses the image-level labels which are much easier to obtain. Experiments on a colonoscopy image dataset with 2100 images shows that the learned local features outperforms other hand-crafted features and gives a state-or-the-art classification accuracy of 93.5%.
---
paper_title: Image partitioning and illumination in image-based pose detection for teleoperated flexible endoscopes

paper_content:

Objective: Colorectal cancer is one of the leading causes of cancer-related deaths in the world, although it can be effectively treated if detected early. Teleoperated flexible endoscopes are an emerging technology to ease patient apprehension about the procedure, and subsequently increase compliance. Essential to teleoperation is robust feedback reflecting the change in pose (i.e., position and orientation) of the tip of the endoscope. The goal of this study is to first describe a novel image-based tracking system for teleoperated flexible endoscopes, and subsequently determine its viability in a clinical setting. The proposed approach leverages artificial neural networks (ANNs) to learn the mapping that links the optical flow between two sequential images to the change in the pose of the camera. Secondly, the study investigates for the first time how narrow band illumination (NBI) - today available in commercial gastrointestinal endoscopes - can be applied to enhance feature extraction, and quantify the effect of NBI and white light illumination (WLI), as well as their color information, on the strength of features extracted from the endoscopic camera stream. Methods and materials: In order to provide the best features for the neural networks to learn the change in pose based on the image stream, we investigated two different imaging modalities - WLI and NBI - and we applied two different spatial partitions - lumen-centered and grid-based - to create descriptors used as input to the ANNs. An experiment was performed to compare the error of these four variations, measured in root mean square error (RMSE) from ground truth given by a robotic arm, to that of a commercial state-of-the-art magnetic tracker. The viability of this technique for a clinical setting was then tested using the four ANN variations, a magnetic tracker, and a commercial colonoscope. The trial was performed by an expert endoscopist (>2000 lifetime procedures) on a colonoscopy training model with porcine blood, and the RMSE of the ANN output was calculated with respect to the magnetic tracker readings. Using the image stream obtained from the commercial endoscope, the strength of features extracted was evaluated. Results: In the first experiment, the best ANNs resulted from grid-based partitioning under WLI (2.42mm RMSE) for position, and from lumen-centered partitioning under NBI (1.69^o RMSE) for rotation. By comparison, the performance of the tracker was 2.49mm RMSE in position and 0.89^o RMSE in rotation. The trial with the commercial endoscope indicated that lumen-centered partitioning was the best overall, while NBI outperformed WLI in terms of illumination modality. The performance of lumen-centered partitioning with NBI was 1.03+/-0.8mm RMSE in positional degrees of freedom (DOF), and 1.26+/-0.98^o RMSE in rotational DOF, while with WLI, the performance was 1.56+/-1.15mm RMSE in positional DOF and 2.45+/-1.90^o RMSE in rotational DOF. Finally, the features extracted under NBI were found to be twice as strong as those extracted under WLI, but no significance in feature strengths was observed between a grayscale version of the image, and the red, blue, and green color channels. Conclusions: This work demonstrates that both WLI and NBI, combined with feature partitioning based on the anatomy of the colon, provide valid mechanisms for endoscopic camera pose estimation via image stream. Illumination provided by WLI and NBI produce ANNs with similar performance which are comparable to that of a state-of-the-art magnetic tracker. However, NBI produces features that are stronger than WLI, which enables more robust feature tracking, and better performance of the ANN in terms of accuracy. Thus, NBI with lumen-centered partitioning resulted the best approach among the different variations tested for vision-based pose estimation. The proposed approach takes advantage of components already available in commercial gastrointestinal endoscopes to provide accurate feedback about the motion of the tip of the endoscope. This solution may serve as an enabling technology for closed-loop control of teleoperated flexible endoscopes.
---
paper_title: A video-based speed estimation technique for localizing the wireless capsule endoscope inside gastrointestinal tract

paper_content:

Wireless Capsule Endoscopy (WCE) is progressively emerging as one of the most popular non-invasive imaging tools for gastrointestinal (GI) tract inspection. As a critical component of capsule endoscopic examination, physicians need to know the precise position of the endoscopic capsule in order to identify the position of intestinal disease. For the WCE, the position of the capsule is defined as the linear distance it is away from certain fixed anatomical landmarks. In order to measure the distance the capsule has traveled, a precise knowledge of how fast the capsule moves is urgently needed. In this paper, we present a novel computer vision based speed estimation technique that is able to extract the speed of the endoscopic capsule by analyzing the displacements between consecutive frames. The proposed approach is validated using a virtual testbed as well as the real endoscopic images. Results show that the proposed method is able to precisely estimate the speed of the endoscopic capsule with 93% accuracy on average, which enhances the localization accuracy of the WCE to less than 2.49 cm.
---
paper_title: Neural network-based approach for the classification of wireless-capsule endoscopic images

paper_content:

The importance of computer-assisted diagnosis in endoscopy is to assist the physician in detecting the status of tissues by characterising the features from the endoscopic image. In this paper schemes have been developed to extract new texture features from the texture spectra in the chromatic and achromatic domains for a selected region of interest from each colour component histogram of images acquired by the new M2A swallowable imaging capsule. The concept of fusion of multiple classifiers dedicated to specific feature parameters and the implementation of an advanced intelligent scheme have been also adopted in this study. The high detection accuracy of the proposed systems provides thus an indication that such intelligent schemes could be used as a supplementary diagnostic tool in capsule endoscopy.
---
paper_title: Automatic detection of small bowel tumors in capsule endoscopy based on color curvelet covariance statistical texture descriptors

paper_content:

Traditional endoscopic methods do not allow the visualization of the entire Gastrointestinal (GI) tract. Wireless Capsule Endoscopy (CE) is a diagnostic procedure that overcomes this limitation of the traditional endoscopic methods. The CE video frames possess rich information about the condition of the stomach and intestine mucosa, encoded as color and texture patterns. It is known for a long time that human perception of texture is based in a multi-scale analysis of patterns, which can be modeled by multi-resolution approaches. Furthermore, modeling the covariance of textural descriptors has been successfully used in classification of colonoscopy videos. Therefore, in the present paper it is proposed a frame classification scheme based on statistical textural descriptors taken from the Discrete Curvelet Transform (DCT) domain, a recent multi-resolution mathematical tool. The DCT is based on an anisotropic notion of scale and high directional sensitivity in multiple directions, being therefore suited to characterization of complex patterns as texture. The covariance of texture descriptors taken at a given detail level, in different angles, is used as classification feature, in a scheme designated as Color Curvelet Covariance. The classification step is performed by a multilayer perceptron neural network. The proposed method has been applied in real data taken from several capsule endoscopic exams and reaches 97.2% of sensitivity and 97.4% specificity. These promising results support the feasibility of the proposed method.
---
paper_title: Data Processing Tasks in Wireless GI Endoscopy: Image-Based Capsule Localization & Navigation and Video Compression

paper_content:

The paper addresses data processing support that is required in capsule gastrointestinal endoscopy. First, capsule position estimation method using standard MPEG-7 image features (descriptors) is discussed. The proposed approach makes use of vector quantization, principal component analysis and neural networks. Next, new algorithms dedicated for virtual colonoscopy (VC) human body inspection are described. The VC images can be registered with endoscopic ones and help in capsule localization and navigation. Finally, an original, low- complexity, efficient image compression method, based on integer-to-integer 4x4 DCT transform, is presented and experimentally verified.
---
paper_title: Efficient homography-based video visualization for wireless capsule endoscopy

paper_content:

Wireless capsule endoscopy (WCE) is performed by a swallowable pill capsule equipped with a camera wirelessly transmitting color video frames to an external receiver. The resulting video consists usually of several thousands of frames and its visual examination requires hours of endoscopists' undivided attention. In this paper we propose a novel visualization approach for WCE which enables faster examination of the endoscopic video, while providing a broader field of view. This is achieved by an algorithm that iteratively samples clusters of consecutive frames from the original video. The frames of each cluster are geometrically transformed, so as to generate a seamless collage subsequently projected into a new frame without any information loss. The new frames compose a new WCE video with a smaller number of frames. The video frame collage is based on homography matrix estimation from frame correspondences. The experiments show that the length of the WCE video, and therefore the reading times required can be significantly reduced.
---
paper_title: Capsule endoscope localization based on visual features

paper_content:

Computational analysis of wireless capsule endoscopy (WCE) videos has already proved its potentials in the discovery or characterization of lesions and in the reduction of the time required by the endoscopists to perform the examination. An open problem that has only partially been addressed is the localization of the capsule endoscope in the gastrointestinal (GI) tract. Previous works have been based mainly on external, wearable, sensors. In this paper we propose a novel approach based solely on visual information extracted from WCE videos. This approach is based on a feature tracking method for visual odometry, which enables the estimation of both the rotation and the displacement of a capsule endoscope from reference anatomical landmarks. Its implementation is based on a novel, open access Java Video Analysis (JVA) framework, which enables quick and standardized development of intelligent video analysis applications. The experimental evaluation presented in this paper, indicates the feasibility of the proposed methodological approach and the efficiency of its implementation.
---
paper_title: Geometric estimation of intestinal contraction for motion tracking of video capsule endoscope

paper_content:

Wireless video capsule endoscope (VCE) provides a noninvasive method to examine the entire gastrointestinal ::: (GI) tract, especially small intestine, where other endoscopic instruments can barely reach. VCE is able to ::: continuously provide clear pictures in short fixed intervals, and as such researchers have attempted to use image ::: processing methods to track the video capsule in order to locate the abnormalities inside the GI tract. To ::: correctly estimate the speed of the motion of the endoscope capsule, the radius of the intestinal track must be ::: known a priori. Physiological factors such as intestinal contraction, however, dynamically change the radius of ::: the small intestine, which could bring large errors in speed estimation. In this paper, we are aiming to estimate ::: the radius of the contracted intestinal track. First a geometric model is presented for estimating the radius of ::: small intestine based on the black hole on endoscopic images. To validate our proposed model, a 3-dimentional ::: virtual testbed that emulates the intestinal contraction is then introduced in details. After measuring the size ::: of the black holes on the test images, we used our model to esimate the radius of the contracted intestinal track. ::: Comparision between analytical results and the emulation model parameters has verified that our proposed ::: method could preciously estimate the radius of the contracted small intestine based on endoscopic images.
---
paper_title: A multi-scale comparison of texture descriptors extracted from the Wavelet and Curvelet domains for small bowel tumor detection in Capsule Endoscopy exams

paper_content:

Traditional endoscopic methods do not reach the entire GI tract. Wireless Capsule Endoscopy is a diagnostic procedure that allows the visualization of the whole GI tract, acquiring video frames, at a rate of two frames per second, while travels through the GI tract. These frames possess rich information about the condition of the stomach and intestine mucosa. In the present paper it is compared the classification performance for small bowel tumor detection of different combinations of texture descriptors taken at different scales of the Discrete Wavelet Transform and Discrete Curvelet Transform domains. The classification step is performed by a multilayer perceptron neural network. The proposed method has been applied in real data taken from several capsule endoscopic exams and reaches 91.7% of sensitivity and 89.4% specificity for features extracted from the DWT domain and 94.1% of sensitivity and 92.4% specificity for features extracted from the DCT domain. These promising results support the feasibility of the proposed method.
---
paper_title: Capsule endoscopy images classification by color texture and support vector machine

paper_content:

Capsule endoscopy (CE) is considered to be a state-of-the-art imaging modality for digest tract diseases detection, especially for small intestine, which is unreachable by traditional endoscopy techniques. However, the large number of images produced by the procedure, about 60,000 images for each examination, cause a time consuming and attention intensive task for physicians, necessitating the development of computer aided detection system. In this paper, we propose a computerized scheme to discriminate among normal CE images and CE images with three common diseases in GI tract, i.e., bleeding, ulcer and tumor. To achieve this goal, features related to color texture characteristics of CE images are extracted. Based on the features, multiclass support vector machine (SVM) built from binary SVM is further applied to separate different CE images. Preliminary experiments on our present image data demonstrate that it is promising to employ the proposed scheme built upon one-against-one binary SVM to differentiate different CE images.
---
paper_title: Inter-Cluster Features for Medical Image Classification

paper_content:

Feature encoding plays an important role for medical image classification. Intra-cluster features such as bag of visual words have been widely used for feature encoding, which are based on the statistical information within each clusters of local features and therefore fail to capture the inter-cluster statistics, such as how the visual words co-occur in images. This paper proposes a new method to choose a subset of cluster pairs based on the idea of Latent Semantic Analysis (LSA) and proposes a new inter-cluster statistics which capture richer information than the traditional co-occurrence information. Since the cluster pairs are selected based on image patches rather than the whole images, the final representation also captures the local structures present in images. Experiments on medical datasets show that explicitly encoding inter-cluster statistics in addition to intra-cluster statistics significantly improves the classification performance, and adding the rich inter-cluster statistics performs better than the frequency based inter-cluster statistics.
---
paper_title: Localization of Endoscopic Capsule in the GI Tract Based on MPEG-7 Visual Descriptors

paper_content:

The paper addresses the problem of localization of video endoscopic capsule in the gastrointestinal (GI) tract on the base of appropriate classification of images received from it. In this context usefulness of MPEG-7 image descriptors as classification features has been verified. For classification purpose various state of the art tools were used including Neural Networks and Vector Quantization. The dimension of the problem was also reduced by the Principal Component Analysis. Novelty of the presented approach consists in joint application of mentioned above techniques for recognition of the GI region inspected by the capsule by means of classification of MPEG-7 features to different parts of GI tract. In this research recognition of the upper part organs of the GI tract has been performed.
---
paper_title: Measurement of motion detection of Wireless Capsule Endoscope inside large intestine

paper_content:

Wireless Capsule Endoscope (WCE) provides a noninvasive way to inspect the entire Gastrointestinal (GI) tract, including large intestine, where intestinal diseases most likely occur. As a critical component of capsule endoscopic examination, physicians need to know the precise position of the endoscopic capsule in order to identify the position of detected intestinal diseases. Knowing how the capsule moves inside the large intestine would greatly complement the existing wireless localization systems by providing the motion information. Since the most recently released WCE can take up to 6 frames per second, it’s possible to estimate the movement of the capsule by processing the successive image sequence. In this paper, a computer vision based approach without utilizing any external device is proposed to estimate the motion of WCE inside the large intestine. The proposed approach estimate the displacement and rotation of the capsule by calculating entropy and mutual information between frames using Fibonacci method. The obtained results of this approach show its stability and better performance over other existing approaches of motion measurements. Meanwhile, findings of this paper lay a foundation for motion pattern of WCEs inside the large intestine, which will benefit other medical applications.
---
paper_title: Visibility Map: A New Method in Evaluation Quality of Optical Colonoscopy

paper_content:

Optical colonoscopy is performed by insertion of a long flexible endoscope into the colon. Inspecting the whole colonic surface for abnormalities has been a main concern in estimating quality of a colonoscopy procedure. In this paper we aim to estimate areas that have not been inspected thoroughly as a quality metric by generating a visibility map of the colon surface. The colon was modeled as a cylinder. By estimating the camera motion parameters between each consecutive frame, circumferential bands from the cylinder of the colon surface were extracted. Registering these extracted band images from adjacent video frames provide a visibility map, which could reveal uncovered areas by clinicians from colonoscopy videos. The method was validated using a set of realistic videos generated using a colonoscopy simulator for which the ground truth was known, and by analyzing results from processing actual colonoscopy videos by a clinical expert. Our method was able to identify 100% of uncovered areas on simulated data and achieved with sensitivity of 96% and precision of 74% on real videos. The results suggest that visibility map can increase clinicians' awareness of uncovered areas, and would reduce the chance of missed polyps.
---
paper_title: An Elastic Image Registration Approach for Wireless Capsule Endoscope Localization

paper_content:

Wireless Capsule Endoscope (WCE) is an innovative imaging device that permits physicians to examine all the areas of the Gastrointestinal (GI) tract. It is especially important for the small intestine, where traditional invasive endoscopies cannot reach. Although WCE represents an extremely important advance in medical imaging, a major drawback that remains unsolved is the WCE precise location in the human body during its operating time. This is mainly due to the complex physiological environment and the inherent capsule effects during its movement. When an abnormality is detected, in the WCE images, medical doctors do not know precisely where this abnormality is located relative to the intestine and therefore they can not proceed efficiently with the appropriate therapy. The primary objective of the present paper is to give a contribution to WCE localization, using image-based methods. The main focus of this work is on the description of a multiscale elastic image registration approach, its experimental application on WCE videos, and comparison with a multiscale affine registration. The proposed approach includes registrations that capture both rigid-like and non-rigid deformations, due respectively to the rigid-like WCE movement and the elastic deformation of the small intestine originated by the GI peristaltic movement. Under this approach a qualitative information about the WCE speed can be obtained, as well as the WCE location and orientation via projective geometry. The results of the experimental tests with real WCE video frames show the good performance of the proposed approach, when elastic deformations of the small intestine are involved in successive frames, and its superiority with respect to a multiscale affine image registration, which accounts for rigid-like deformations only and discards elastic deformations.
---
paper_title: Incorporation of 3D Model and Panoramic View for Gastroscopic Lesion Surveillance

paper_content:

Natural Orifice Transluminal Endoscopic Surgery (NOTES) is widely used for clinical diagnoses. However, NOTES has two main problems: difficulties brought by endoscope’s flexibility and narrow view of endoscope. Image-guided system is helpful to deal with these problems. In our previous work, a computer aided endoscopic navigation system (CAEN) was developed for gastroscopic lesion surveillance. In this paper, 3D model and panoramic view are incorporated into CAEN with three improvements: selection of reference and tracking features; perspective projection for constructing local and global panoramic view; 3D surface modeling using structure from motion. The system is evaluated from three clinic applications: broadening the view, non-invasive retargeting, and overall lesion locations. The evaluation results show that the mean accuracy of broadening the view is 0.43 mm, the mean accuracy of non-invasive retargeting is 7.5 mm, and the mean accuracy for overall lesion diagnosis is 3.71 ± 0.35 mm.
---
paper_title: Automatic small bowel tumor diagnosis by using multi-scale wavelet-based analysis in wireless capsule endoscopy images

paper_content:

BackgroundWireless capsule endoscopy has been introduced as an innovative, non-invasive diagnostic technique for evaluation of the gastrointestinal tract, reaching places where conventional endoscopy is unable to. However, the output of this technique is an 8 hours video, whose analysis by the expert physician is very time consuming. Thus, a computer assisted diagnosis tool to help the physicians to evaluate CE exams faster and more accurately is an important technical challenge and an excellent economical opportunity.MethodThe set of features proposed in this paper to code textural information is based on statistical modeling of second order textural measures extracted from co-occurrence matrices. To cope with both joint and marginal non-Gaussianity of second order textural measures, higher order moments are used. These statistical moments are taken from the two-dimensional color-scale feature space, where two different scales are considered. Second and higher order moments of textural measures are computed from the co-occurrence matrices computed from images synthesized by the inverse wavelet transform of the wavelet transform containing only the selected scales for the three color channels. The dimensionality of the data is reduced by using Principal Component Analysis.ResultsThe proposed textural features are then used as the input of a classifier based on artificial neural networks. Classification performances of 93.1% specificity and 93.9% sensitivity are achieved on real data. These promising results open the path towards a deeper study regarding the applicability of this algorithm in computer aided diagnosis systems to assist physicians in their clinical practice.
---
paper_title: Automatic normal-abnormal video frame classification for colonoscopy

paper_content:

Two novel schemes are proposed to represent intermediate-scale features for normal-abnormal classification of colonoscopy images. The first scheme works on the full-resolution image, the second on a multi-scale pyramid space. Both schemes support any feature descriptor; here we use multi-resolution local binary patterns which outperformed other features reported in the literature in our comparative experiments. We also compared experimentally two types of features not previously used in colonoscopy image classification, bag of features and sparse coding, each with and without spatial pyramid matching (SPM). We find that SPM improves performance, therefore supporting the importance of intermediate-scale features as in the proposed schemes for classification. Within normal-abnormal frame classification, we show that our representational schemes outperforms other features reported in the literature in leave-N-out tests with a database of 2100 colonoscopy images.
---
paper_title: Extended Gaussian-Filtered Local Binary Patterns for Colonoscopy Image Classification

paper_content:

Local Binary Patterns (LBP) and its variants are widely used for texture classification. In this paper we propose a new variant of LBP descriptor called the extended Gaussian filtered Local Binary Patterns (GF-LBP) which is robust to illumination changes, noise and captures more informative edge-like features for classification. Experiments on a colonoscopy image dataset with 2100 images for binary (`normal' or `abnormal') classification show that the proposed xGF-LBP descriptor significantly outperforms the standard LBP descriptor and its considered variants.
---
paper_title: Video-Specific SVMs for Colonoscopy Image Classification

paper_content:

We propose a novel classification framework called the video-specific SVM (V-SVM) for normal-vs-abnormal white-light colonoscopy image classification. V-SVM is an ensemble of linear SVMs, with each trained to separate the abnormal images in a particular video from all the normal images in all the videos. Since V-SVM is designed to capture lesion-specific properties as well as intra-class variations it is expected to perform better than SVM. Experiments on a colonoscopy image dataset with about \(10{,}000\) images show that V-SVM significantly improves the performance over SVM and other baseline classifiers.
---
paper_title: Towards Intelligent Systems for Colonoscopy

paper_content:

Colorectal cancer is one of the leading causes of cancer related deaths. Colorectal cancer’s survival rate depends on the stage in which it is detected, decreasing from rates higher than 95% in the first stages to rates lower than 35% in stages IV and V (Tresca, A. (2010)); hence the importance of detecting it on its early stages by using screening techniques, such as colonoscopy (Hassinger, J.P., Holubar, S.D. et al. (2010)), which is still considered the gold standard for the screening of patients for colon cancers and lesions. Classical focal colonoscopy has been proved to be a successful tool for colon screening, although other tools are being also used for this purpose, such as Virtual Colonoscopy, Computed Tomography Colonoscopy, Chromoendoscopy or Wireless Capsule Video Endoscopy, among others. In this chapter we present tools that can be used to build intelligent systems for colonoscopy. The idea is, by using methods based on computer vision and artificial intelligence, add significant value to the colonoscopy procedure. Intelligent systems are being used to assist in other medical interventions. For instance, we can build systems that can be used to develop the knowledge bases used by expert systems, such as KARDIO (Bratko et al. (1990), which was developed to interpret electrocardiograms. Another example can consist of developing a system that, in the context of anesthesia, provides a robust/reliable control system that could determine the optimal infusion rate of the drugs (muscle relaxant, anesthetic, and analgesic) simultaneously, and titrate each drug in accordance to its effects and interactions. Such a system would be a valuable assistant to the anesthetist in the operating theater. An example of such a system can be found in the work of Nunes et al. (2005). More close to our topic of interest, colonoscopy, we can find many examples of intelligent systems build to assist in cancer detection. Such is the case of breast cancer detection (Wei et al. (2011)) or prostate cancer detection (Viswanath et al. (2011)). The question that arises now is: how can intelligent systems help in colonoscopy? What kind of applications these systems can be built for? In Figure 1 we depict some of the potential areas related to colonoscopy where an intelligent system can play a key role. As shown in Figure 1, we foresee four different areas where an intelligent system can be introduced and add significant value to the colonoscopy procedure:
---
paper_title: Detection of Lesions in Colonoscopic Images: A Review

paper_content:

Colonoscopy is one of the best methods for screening colon cancer and still is the “gold standard” despite advancements in the field of virtual endoscopy based on computer-tomographic imaging. As the automatic detection of polyps in endoscopic images is a challenging task for image processing, a variety of research groups have proposed methods that try to fulfill this task to develop a system which supports the doctors during examination. However, the problem can still not be assumed to be solved. This paper provides a review of the state of the art in detection methods published within the last decade. We found out that the major drawback of many approaches is the lack of representative video data, which hinders comparison and evaluation of the published methods.
---
paper_title: Detection of small bowel tumors in capsule endoscopy frames using texture analysis based on the discrete wavelet transform

paper_content:

Capsule endoscopy is an important tool to diagnose tumor lesions in the small bowel. The capsule endoscopic images possess vital information expressed by color and texture. This paper presents an approach based in the textural analysis of the different color channels, using the wavelet transform to select the bands with the most significant texture information. A new image is then synthesized from the selected wavelet bands, trough the inverse wavelet transform. The features of each image are based on second-order textural information, and they are used in a classification scheme using a multilayer perceptron neural network. The proposed methodology has been applied in real data taken from capsule endoscopic exams and reached 98.7% sensibility and 96.6% specificity. These results support the feasibility of the proposed algorithm.
---
paper_title: Capsule endoscope localization based on computer vision technique

paper_content:

To build a new type of wireless capsule endoscope with interactive gastrointestinal tract examination, a localization and orientation system is needed for tracking 3D location and 3D orientation of the capsule movement. The magnetic localization and orientation method produces only 5 DOF, but misses the information of rotation angle along capsule’s main axis. In this paper, we presented a complementary orientation approach for the capsule endoscope, and the 3D rotation can be determined by applying computer vision technique on the captured endoscopic images. The experimental results show that the complementary orientation method has good accuracy and high feasibility.
---
paper_title: Circular generalized cylinder fitting for 3D reconstruction in endoscopic imaging based on MRF

paper_content:

Endoscopy has become an established procedure for the diagnosis and therapy of various gastrointestinal (GI) ailments, and has also emerged as a commonly-used technique for minimally-invasive surgery. Most existing endoscopes are monocular, with stereo-endoscopy facing practical difficulties, preventing the physicians/surgeons from having a desired, realistic 3D view. Traditional monocular 3D reconstruction approaches (e.g., structure from motion) face extraordinary challenges for this application due to issues including noisy data, lack of textures supporting robust feature matching, nonrigidity of the objects, and glare artifacts from the imaging process, etc. In this paper, we propose a method to automatically reconstruct 3D structure from a monocular endoscopic video. Our approach attempts to address the above challenges by incorporating a circular generalized cylinder (CGC) model in 3D reconstruction. The CGC model is decomposed as a series of 3D circles. To reconstruct this model, we formulate the problem as one of maximum a posteriori estimation within a Markov random field framework, so as to ensure the smoothness constraints of the CGC model and to support robust search for the optimal solution, which is achieved by a two-stage heuristic search scheme. Both simulated and real data experiments demonstrate the effectiveness of the proposed approach.
---
paper_title: Automatic detection and inpainting of specular reflections for colposcopic images

paper_content:

Abstract Specular reflections are not wanted in images because they can really reduce the performance of image processing techniques. This is particularly true for medical images and especially for colposcopic images. There are several methods in the literature allowing to extract specular reflections, but only a few methods can perform an automatic extraction. In this paper, we propose a new method to extract and to restore specularities automatically. This method is based on Dichromatic Reflection Model (DRM) and multi-resolution inpainting technique (MIT). The DRM approach will retrieve specularities while the MIT technique re-establish colors in bright zones using local information. The proposed method achieves good results and does not need any a priori knowledge. The efficiency of this method for colposcopic images has been demonstrated through a collaboration with the oncology center of Marrakech University Hospital.
---
paper_title: Three-dimensional image reconstruction in capsule endoscopy

paper_content:

To date, limited research has been carried out in developing methods and materials that offer three-dimensional (3-D) representation of the digestive tract. In the field of capsule endoscopy (CE), hardware approaches have been developed that provide real time both 3-D information and texture using an infrared projector and a complementary metal oxide semiconductor camera. The major drawbacks of this system are its size, power consumption and packaging issues. A software approach to approximate a 3-D representation of digestive tract surface utilising current CE technology has been proposed. The algorithm utilizes the Shape from Shading technique and seem to provide promising results for polypoid structures and angioectasias. Further clinical evaluation is currently under way.
---
paper_title: Affine-invariant anisotropic detector for soft tissue tracking in minimally invasive surgery

paper_content:

Reliable feature tracking is important for accurate tissue deformation recovery, 3D anatomical registration and navigation in computer assisted minimally invasive surgical procedures. Despite a wide range of feature detectors developed in the computer vision community, direct application of these approaches to surgical navigation has shown significant difficulties due to the paucity of reliable feature landmarks coupled with free-form tissue deformation and contrastingly different visual appearances of changing surgical scenes. The purpose of this paper is to introduce an affine-invariant feature detector based on anisotropic features to ensure reliable and persistent feature tracking. A novel scale-space representation is proposed for scale adaptation based on the strength of the anisotropic pattern whereas affine adaptation relies on its intrinsic Fourier properties with an efficient spatial implementation based on the second moment matrix. The proposed detector is compared against the current state-of-the-art feature detectors and their respective performance is evaluated with in vivo video sequences recorded from robotic assisted minimally invasive surgical procedures.
---
paper_title: Intra-operative monocular 3D reconstruction for image-guided navigation in active locomotion capsule endoscopy

paper_content:

Wireless capsule endoscopy (WCE) can be considered an example of ‘disruptive technology’ since it represents a bright alternative to traditional diagnostic methodologies. However, currently available clinical products are passive devices whose locomotion is driven by natural peristalsis, with the drawback of failing to capture the images of potential pathological gastrointestinal tract regions, due to the impossibility to control the camera's motion and orientation. As a consequence, many research groups are working to develop active locomotion devices and endoscopic platforms that allow WCE to be performed in a totally controlled manner. In this paper we propose a system to reliably calibrate a Shape-from-Shading system to recover the unknown scale factor immediately prior to a WCE procedure. The system would allow to reconstruct in a metrically accurate fashion the surfaces obtained intraoperatively from the WCE capsule, which can be exploited for accurate trajectory planning and ultimately automatic WCE active capsule navigation. To the best of our knowledge, this is the first practical system for Shape-from-Shading calibration aimed at guidance applications that will finally be applied to an endoscopic active locomotion platform developed by the authors.
---
paper_title: 3D reconstruction of wireless capsule endoscopy images

paper_content:

Wireless capsule endoscopy (WCE) has been gradually applied for inspecting the gastrointestinal (GI) tract. However, WCE can only provide monocular view. Moreover, only a small part of GI wall is visible frame by frame due to the limited illumination and irregular motion of the capsule endoscope. The perception of entire GI structure could be hard even for the experienced endoscopists. A realistic friendly three dimension view is needed to help the physicians to get a better perception of the GI tract. In this paper, we present a method to reconstruct the three dimension surface of the intestinal wall by applying the SIFT feature detector and descriptor to a sequence of WCE images. Epipolar geometry is employed to further constrain the matching feature points in order to obtain a more accurate 3D view. The experiments on real data are presented to show the performance of our proposed method.
---
paper_title: Discarding Non Informative Regions for Efficient Colonoscopy Image Analysis

paper_content:

The diagnostic yield of colon cancer screening using colonoscopy could improve using intelligent systems. The large amount of data provided by high definition equipments contains frames with large non-informative regions. Non-informative regions have such a low visual quality that even physicians can not properly identify structures. Thus, identification of such regions is an important step for an efficient and accurate processing. We present a strategy for discarding non-informative regions in colonoscopy frames based on a model of appearance of such regions. Three different methods are proposed to characterize accurately the boundary between informative and non-informative regions. Preliminary results shows that there is a statistically significant difference between each of the methods as some of them are more strict when deciding which part of the image is informative and others regarding which is the non-informative region.
---
paper_title: Indistinct Frame Detection in Colonoscopy Videos

paper_content:

An automated system for analysis of colonoscopy videos is expected to complement the expertise and the experience of a medical professional in: (a) detecting lesions and (b) assessing the quality of a given procedure. Colonoscopy videos contain a significant number of frames which do not carry any clinical information. The presence of such frames would slow down or cause the failure of the processing steps of such an automated system. Furthermore, many existing metrics to measure the quality of the colonoscopy procedures directly involve the number of such indistinct frames present in the videos. We propose a novel algorithm to detect indistinct frames based on the wavelet analysis. The L2 norm of the detail coefficients of the wavelet decomposition of a colonoscopy image is considered as the feature vector of the proposed classification system. The algorithm was tested on a manually labeled, balanced data set. It achieved an accuracy of 99.59% in a leave-two-out cross validation procedure based on Bayesian classification. Furthermore, when applied to full colonoscopy videos, the presented algorithm detected 26.2% of the frames as indistinct, of which 92.3% were correctly classified. The proposed method outperforms the current best performing algorithm both in terms of accuracy and computation time.
---
paper_title: Online Scene Association for Endoscopic Navigation

paper_content:

Endoscopic surveillance is a widely used method for monitoring abnormal changes in the gastrointestinal tract such as Barrett’s esophagus. Direct visual assessment, however, is both time consuming and error prone, as it involves manual labelling of abnormalities on a large set of images. To assist surveillance, this paper proposes an online scene association scheme to summarise an endoscopic video into scenes, on-the-fly. This provides scene clustering based on visual contents, and also facilitates topological localisation during navigation. The proposed method is based on tracking and detection of visual landmarks on the tissue surface. A generative model is proposed for online learning of pairwise geometrical relationships between landmarks. This enables robust detection of landmarks and scene association under tissue deformation. Detailed experimental comparison and validation have been conducted on in vivo endoscopic videos to demonstrate the practical value of our approach.
---
paper_title: Image partitioning and illumination in image-based pose detection for teleoperated flexible endoscopes

paper_content:

Objective: Colorectal cancer is one of the leading causes of cancer-related deaths in the world, although it can be effectively treated if detected early. Teleoperated flexible endoscopes are an emerging technology to ease patient apprehension about the procedure, and subsequently increase compliance. Essential to teleoperation is robust feedback reflecting the change in pose (i.e., position and orientation) of the tip of the endoscope. The goal of this study is to first describe a novel image-based tracking system for teleoperated flexible endoscopes, and subsequently determine its viability in a clinical setting. The proposed approach leverages artificial neural networks (ANNs) to learn the mapping that links the optical flow between two sequential images to the change in the pose of the camera. Secondly, the study investigates for the first time how narrow band illumination (NBI) - today available in commercial gastrointestinal endoscopes - can be applied to enhance feature extraction, and quantify the effect of NBI and white light illumination (WLI), as well as their color information, on the strength of features extracted from the endoscopic camera stream. Methods and materials: In order to provide the best features for the neural networks to learn the change in pose based on the image stream, we investigated two different imaging modalities - WLI and NBI - and we applied two different spatial partitions - lumen-centered and grid-based - to create descriptors used as input to the ANNs. An experiment was performed to compare the error of these four variations, measured in root mean square error (RMSE) from ground truth given by a robotic arm, to that of a commercial state-of-the-art magnetic tracker. The viability of this technique for a clinical setting was then tested using the four ANN variations, a magnetic tracker, and a commercial colonoscope. The trial was performed by an expert endoscopist (>2000 lifetime procedures) on a colonoscopy training model with porcine blood, and the RMSE of the ANN output was calculated with respect to the magnetic tracker readings. Using the image stream obtained from the commercial endoscope, the strength of features extracted was evaluated. Results: In the first experiment, the best ANNs resulted from grid-based partitioning under WLI (2.42mm RMSE) for position, and from lumen-centered partitioning under NBI (1.69^o RMSE) for rotation. By comparison, the performance of the tracker was 2.49mm RMSE in position and 0.89^o RMSE in rotation. The trial with the commercial endoscope indicated that lumen-centered partitioning was the best overall, while NBI outperformed WLI in terms of illumination modality. The performance of lumen-centered partitioning with NBI was 1.03+/-0.8mm RMSE in positional degrees of freedom (DOF), and 1.26+/-0.98^o RMSE in rotational DOF, while with WLI, the performance was 1.56+/-1.15mm RMSE in positional DOF and 2.45+/-1.90^o RMSE in rotational DOF. Finally, the features extracted under NBI were found to be twice as strong as those extracted under WLI, but no significance in feature strengths was observed between a grayscale version of the image, and the red, blue, and green color channels. Conclusions: This work demonstrates that both WLI and NBI, combined with feature partitioning based on the anatomy of the colon, provide valid mechanisms for endoscopic camera pose estimation via image stream. Illumination provided by WLI and NBI produce ANNs with similar performance which are comparable to that of a state-of-the-art magnetic tracker. However, NBI produces features that are stronger than WLI, which enables more robust feature tracking, and better performance of the ANN in terms of accuracy. Thus, NBI with lumen-centered partitioning resulted the best approach among the different variations tested for vision-based pose estimation. The proposed approach takes advantage of components already available in commercial gastrointestinal endoscopes to provide accurate feedback about the motion of the tip of the endoscope. This solution may serve as an enabling technology for closed-loop control of teleoperated flexible endoscopes.
---
paper_title: 3D Colon Segment and Endoscope Motion Reconstruction from Colonoscopy Video

paper_content:

This dissertation addresses the problem of reconstruction of the 3D structure of colon segments and the 3D motion of an endoscope from colonoscopy images. Colonoscopy is currently the preferred screening modality for early colorectal cancer (CRC) detection, claiming about 50,000 annual deaths in the US. During colonoscopy, the endoscopist performs diagnosis and/or treatment based on images of the interior of the human colon taken by the camera at the tip of the endoscope. ::: The reconstruction of the virtual colon and the endoscope motion from colonoscopy images has the potential to (1) improve endoscopist's understanding of the colon structure of the patient and the endoscopist's navigation technique at the time of colonoscopy; (2) determine the estimated area and location of the colon mucosa uninspected by the endoscopist; domain experts estimated that as much as 95% of the colon mucosa should be inspected; and (3) bridge the gap between virtual colonoscopy and optical colonoscopy. Besides our previous work, we found no other techniques that address the same problem in the literature. We propose a set of new algorithms and evaluate them using images inside an Olympus synthetic colon as well as colonoscopy images from real colonoscopy procedures. Edges of colon fold contours are first detected and processed to generate the wire frame of the reconstructed virtual colon. We propose a colon fold contour estimation algorithm using a single colonoscopy image. We introduce Depth-from-Intensity—the depth and shape estimation of colon folds using brightness intensity of pixels. We introduce an algorithm to estimate the amount of protrusion and width of colon folds and interpolate the folds and the colon wall to connect the folds in 3D space. We present our new method for reconstructing the virtual colon from sequential colonoscopy images and deriving corresponding camera motions. We discuss two quality metrics derived using the information from the reconstruction: spiral score and percentage of areas uninspected. Spiral score is found strongly correlated with the ground truth visualization quality on 159 colonoscopy videos. The percentage of areas uninspected metric is of great interest to domain experts. ::: Broader impact: The proposed work has potential to improve the amount of the colon mucosa inspected in clinical practice; thereby, potentially increasing the protective effect of colonoscopy against CRC. The technology may change the way colonoscopy is performed in routine practice and may be used for training new endoscopists. Finally, the algorithm design will add to existing knowledge in endoscopy video processing and may be beneficial for other endoscopy procedures.
---
paper_title: Automatic detection of informative frames from wireless capsule endoscopy images

paper_content:

Wireless capsule endoscopy (WCE) is a new clinical technology permitting visualization of the small bowel, the most difficult segment of the digestive tract. The major drawback of this technology is the excessive amount of time required for video diagnosis. We therefore propose a method for generating smaller videos by detecting informative frames from original WCE videos. This method isolates useless frames that are highly contaminated by turbid fluids, faecal materials and/or residual foods. These materials and fluids are presented in a wide range of colors, from brown to yellow, and/or have bubble-like texture patterns. The detection scheme therefore consists of two steps: isolating (Step-1) highly contaminated non-bubbled (HCN) frames and (Step-2) significantly bubbled (SB) frames. Two color representations, viz., local color moments in Ohta space and the HSV color histogram, are attempted to characterize HCN frames, which are isolated by a support vector machine (SVM) classifier in Step-1. The rest of the frames go to Step-2, where a Gauss Laguerre transform (GLT) based multiresolution texture feature is used to characterize the bubble structures in WCE frames. GLT uses Laguerre Gauss circular harmonic functions (LG-CHFs) to decompose WCE images into multiresolution components. An automatic method of segmentation was designed to extract bubbled regions from grayscale versions of the color images based on the local absolute energies of their CHF responses. The final informative frames were detected by using a threshold on the segmented regions. An automatic procedure for selecting features based on analyzing the consistency of the energy-contrast map is also proposed. Three experiments, two of which use 14,841 and 37,100 frames from three videos and the rest uses 66,582 frames from six videos, were conducted for justifying the proposed method. The two combinations of the proposed color and texture features showed excellent average detection accuracies (86.42% and 84.45%) with the final experiment, when compared with the same color features followed by conventional Gabor-based (78.18% and 76.29%) and discrete wavelet-based (65.43% and 63.83%) texture features. Although intra-video training–testing cases are typical choices for supervised classification in Step-1, combining a suitable number of training sets using a subset of the input videos was shown to be possible. This mixing not only reduced computation costs but also produced better detection accuracies by minimizing visual-selection errors, especially when processing large numbers of WCE videos.
---
paper_title: Re-localisation of microscopic lesions in their macroscopic context for surgical instrument guidance

paper_content:

Optical biopsies interrogate microscopic structure in vivo with a 2mm diameter miniprobe ::: placed in contact with the tissue for detection of lesions and assessment of disease ::: progression. After detection, instruments are guided to the lesion location for a new optical ::: interrogation, or for treatment, or for tissue excision during the same or a future examination. ::: As the optical measurement can be considered as a point source of information at the surface ::: of the tissue of interest, accurate guidance can be difficult. A method for re-localisation of the ::: sampling point is, therefore, needed. ::: The method presented in this thesis has been developed for biopsy site re-localisation ::: during a surveillance examination of Barrett’s Oesophagus. The biopsy site, invisible ::: macroscopically during conventional endoscopy, is re-localised in the target endoscopic ::: image using epipolar lines derived from its locations given by the tip of the miniprobe visible ::: in a series of reference endoscopic images. A confidence region can be drawn around the relocalised ::: biopsy site from its uncertainty that is derived analytically. This thesis also presents ::: a method to improve the accuracy of the epipolar lines derived for the biopsy site relocalisation ::: using an electromagnetic tracking system. ::: Simulations and tests on patient data identified the cases when the analytical ::: uncertainty is a good approximation of the confidence region and showed that biopsy sites ::: can be re-localised with accuracies better than 1mm. Studies on phantom and on porcine ::: excised tissue demonstrated that an electromagnetic tracking system contributes to more ::: accurate epipolar lines and re-localised biopsy sites for an endoscope displacement greater ::: than 5mm. The re-localisation method can be applied to images acquired during different ::: endoscopic examinations. It may also be useful for pulmonary applications. Finally, it can be ::: combined with a Magnetic Resonance scanner which can steer cells to the biopsy site for ::: tissue treatment.
---
paper_title: Optical Biopsy Mapping for Minimally Invasive Cancer Screening

paper_content:

The quest for providing tissue characterization and functional mapping during minimally invasive surgery (MIS) has motivated the development of new surgical tools that extend the current functional capabilities of MIS. Miniaturized optical probes can be inserted into the instrument channel of standard endoscopes to reveal tissue cellular and subcellular microstructures, allowing excision-free optical biopsy. One of the limitations of such a point based imaging and tissue characterization technique is the difficulty of tracking probed sites in vivo. This prohibits large area surveillance and integrated functional mapping. The purpose of this paper is to present an image-based tracking framework by combining a semi model-based instrument tracking method with vision-based simultaneous localization and mapping. This allows the mapping of all spatio-temporally tracked biopsy sites, which can then be re-projected back onto the endoscopic video to provide a live augmented view in vivo, thus facilitating re-targeting and serial examination of potential lesions. The proposed method has been validated on phantom data with known ground truth and the accuracy derived demonstrates the strength and clinical value of the technique. The method facilitates a move from the current point based optical biopsy towards large area multi-scale image integration in a routine clinical environment.
---
paper_title: Vision and inertial-based image mapping for capsule endoscopy

paper_content:

Capsule endoscopy is a non-invasive procedure for gastrointestinal diagnosis. It does not require sedation and it is comfortable and well tolerated by patient. However, the problem with such procedure is that a huge number of images is collected, which require time to investigate and diagnose; furthermore, the capsule movement is not controlled leading, in some cases, to inaccurate diagnosis. In this context, a mapping of the lumen is required to guarantee a higher reliability of the inspection, enabling the medical doctor to evaluate all the parts of the lumen for a better diagnosis. In this paper, we propose a method for mapping images from a capsule-based endoscope: the technique uses visual and inertial-based data fusion to obtain a 3D map of the lumen from 2D capsule images, also paving the way for the implementation of a path planning and autonomous locomotion and inspection.
---
paper_title: A novel method for informative frame selection in wireless capsule endoscopy video

paper_content:

Wireless capsule endoscopy (WCE) has been validated to be an important tool in the evaluation of gastrointestinal (GI) tract. Compared with traditional endoscope technologies, its non-invasiveness property meets with great favor of patients. However, from physician's point of view, WCE video suffers from low resolution, limited illumination, irregular movement, more importantly, imbalanced rate of abnormality which brings a lot of challenges for diagnosis. These challenges motivate us to devise an approach to guide the physicians to focus on the informative frames which could be convenient for review of the content of the GI tract. This paper presents a novel approach for automatic selection of the WCE video frames with lumen and coherent motility. We adopt lumen detection based on mean shift to provide robust and reliable selection of lumen. Together with the evaluation of coherent motility, we can provide a full and fast approach to select the informative frames for diagnosis. The experiments on real date are presented to show the performance of our proposed method.
---
paper_title: Simultaneous Stereoscope Localization and Soft-Tissue Mapping for Minimal Invasive Surgery

paper_content:

Minimally Invasive Surgery (MIS) has recognized benefits of reduced patient trauma and recovery time. In practice, MIS procedures present a number of challenges due to the loss of 3D vision and the narrow field-of-view provided by the camera. The restricted vision can make navigation and localization within the human body a challenging task. This paper presents a robust technique for building a repeatable long term 3D map of the scene whilst recovering the camera movement based on Simultaneous Localization and Mapping (SLAM). A sequential vision only approach is adopted which provides 6 DOF camera movement that exploits the available textured surfaces and reduces reliance on strong planar structures required for range finders. The method has been validated with a simulated data set using real MIS textures, as well as in vivo MIS video sequences. The results indicate the strength of the proposed algorithm under the complex reflectance properties of the scene, and the potential for real-time application for integrating with the existing MIS hardware.
---
paper_title: A non-invasive navigation system for retargeting gastroscopic lesions

paper_content:

Biopsy is a traditional endoscopic surveillance of premalignant gastric lesions, and endoscopic tattooing is used for marking the biopsy's location. However, the tattooing has several disadvantages. For example, the procedure is an invasive operation and may not be durable due to the diffusion. Moreover, it is procedurally cumbersome with an associated risk of technical failure. In this study, a computer aided endoscopic navigation system (CAEN system) was developed for a non-invasive biopsy procedure. The CAEN system consists of a new, designed six degree of freedom (6-DOF) tracking endoscope device and a computer simulated work station. During the procedure, the endoscopist uses the tip of the tracking endoscope to touch the lesion. Then, the lesion's location is recorded in the work station, which then guides the endoscopist in retargeting the lesions in the follow-ups. The clinical experimental results demonstrate that the accuracy at the angularis is 5.22.8 mm, at the antral lesser curvature is 7.22.0 mm, at the antral greater curvature is 6.33.1 mm, at the antral posterior wall is 8.21.6 mm, and at the antral anterior wall is 7.91.3 mm. The mean accuracy is 7.5 mm, and the P-value is 0.023, which is likely suitable for clinical practice. Furthermore, the proposed CAEN system requires less procedural time than the tattooing.
---
paper_title: A probabilistic framework for tracking deformable soft tissue in minimally invasive surgery

paper_content:

The use of vision based algorithms in minimally invasive surgery has attracted significant attention in recent years due to its potential in providing in situ 3D tissue deformation recovery for intra-operative surgical guidance and robotic navigation. Thus far, a large number of feature descriptors have been proposed in computer vision but direct application of these techniques to minimally invasive surgery has shown significant problems due to free-form tissue deformation and varying visual appearances of surgical scenes. This paper evaluates the current state-of-the-art feature descriptors in computer vision and outlines their respective performance issues when used for deformation tracking. A novel probabilistic framework for selecting the most discriminative descriptors is presented and a Bayesian fusion method is used to boost the accuracy and temporal persistency of soft-tissue deformation tracking. The performance of the proposed method is evaluated with both simulated data with known ground truth, as well as in vivo video sequences recorded from robotic assisted MIS procedures.
---
paper_title: Informative-frame filtering in endoscopy videos

paper_content:

Advances in video technology are being incorporated into today’s healthcare practice. For example, colonoscopy is an important screening tool for colorectal cancer. Colonoscopy allows for the inspection of the entire colon and provides the ability to perform a number of therapeutic operations during a single procedure. During a colonoscopic procedure, a tiny video camera at the tip of the endoscope generates a video signal of the internal mucosa of the colon. The video data are displayed on a monitor for real-time analysis by the endoscopist. Other endoscopic procedures include upper gastrointestinal endoscopy, enteroscopy, bronchoscopy, cystoscopy, and laparoscopy. However, a significant number of out-of-focus frames are included in this type of videos since current endoscopes are equipped with a single, wide-angle lens that cannot be focused. The out-of-focus frames do not hold any useful information. To reduce the burdens of the further processes such as computer-aided image processing or human expert’s examinations, these frames need to be removed. We call an out-of-focus frame as non-informative frame and an in-focus frame as informative frame. We propose a new technique to classify the video frames into two classes, informative and non-informative frames using a combination of Discrete Fourier Transform (DFT), Texture Analysis, and K-Means Clustering. The proposed technique can evaluate the frames without any reference image, and does not need any predefined threshold value. Our experimental studies indicate that it achieves over 96% of four different performance metrics (i.e. precision, sensitivity, specificity, and accuracy).
---
paper_title: Endoscopic Video Manifolds for Targeted Optical Biopsy

paper_content:

Gastro-intestinal (GI) endoscopy is a widely used clinical procedure for screening and surveillance of digestive tract diseases ranging from Barrett's Oesophagus to oesophageal cancer. Current surveillance protocol consists of periodic endoscopic examinations performed in 3-4 month intervals including expert's visual assessment and biopsies taken from suspicious tissue regions. Recent development of a new imaging technology, called probe-based confocal laser endomicroscopy (pCLE), enabled the acquisition of in vivo optical biopsies without removing any tissue sample. Besides its several advantages, i.e., noninvasiveness, real-time and in vivo feedback, optical biopsies involve a new challenge for the endoscopic expert. Due to their noninvasive nature, optical biopsies do not leave any scar on the tissue and therefore recognition of the previous optical biopsy sites in surveillance endoscopy becomes very challenging. In this work, we introduce a clustering and classification framework to facilitate retargeting previous optical biopsy sites in surveillance upper GI-endoscopies. A new representation of endoscopic videos based on manifold learning, "endoscopic video manifolds" (EVMs), is proposed. The low dimensional EVM representation is adapted to facilitate two different clustering tasks; i.e., clustering of informative frames and patient specific endoscopic segments, only by changing the similarity measure. Each step of the proposed framework is validated on three in vivo patient datasets containing 1834, 3445, and 1546 frames, corresponding to endoscopic videos of 73.36, 137.80, and 61.84 s, respectively. Improvements achieved by the introduced EVM representation are demonstrated by quantitative analysis in comparison to the original image representation and principal component analysis. Final experiments evaluating the complete framework demonstrate the feasibility of the proposed method as a promising step for assisting the endoscopic expert in retargeting the optical biopsy sites.
---
paper_title: Global and Local Panoramic Views for Gastroscopy: An Assisted Method of Gastroscopic Lesion Surveillance

paper_content:

Gastroscopy plays an important role in the diagnosis of gastric disease. In this paper, we develop an image panoramic system to assist endoscopists in improving lesion surveillance and reducing many of the tedious operations associated with gastroscopy. The constructed panoramic view has two categories: 1) the local view broadens the endoscopist's field of view in real time. Combining with the original gastroscopic video, this mosaicking view enables the endoscopist to diagnose the lesion comprehensively; 2) the global view constructs a large-area panoramic scene of the internal gastric surface, which can be used for intraoperative surgical navigation and postoperative scene review. Due to the irregular texture and inconsistent reflection of the gastric internal surface, common registration methods cannot accurately stitch this surface. Thereby, a six degree of freedom position tracking endoscope is devised to accommodate for the accumulated mosaicking error and provide efficient mosaicking results. For the global view, a dual-cube constraint model and a Bundle Adjustment algorithm are incorporated to deal with the mosaicking error caused by the irregular inflation and nonrigid deformation of the stomach. Moreover, texture blending and frame selection schemes are developed to make the mosaicking results feasible in real-clinical applications. The experimental results demonstrate that our system performs with a speed of 7.12 frames/s in a standard computer environment, and the mosaicking mean error is 0.43 mm for local panoramic view and 3.71 mm for global panoramic view.
---
paper_title: 3D Reconstruction of virtual colon structures from colonoscopy images

paper_content:

Abstract This paper presents the first fully automated reconstruction technique of 3D virtual colon segments from individual colonoscopy images. It is the basis of new software applications that may offer great benefits for improving quality of care for colonoscopy patients. For example, a 3D map of the areas inspected and uninspected during colonoscopy can be shown on request of the endoscopist during the procedure. The endoscopist may revisit the suggested uninspected areas to reduce the chance of missing polyps that reside in these areas. The percentage of the colon surface seen by the endoscopist can be used as a coarse objective indicator of the quality of the procedure. The derived virtual colon models can be stored for post-procedure training of new endoscopists to teach navigation techniques that result in a higher level of procedure quality. Our technique does not require a prior CT scan of the colon or any global positioning device. Our experiments on endoscopy images of an Olympus synthetic colon model reveal encouraging results with small average reconstruction errors (4.1 mm for the fold depths and 12.1 mm for the fold circumferences).
---
paper_title: A fast and accurate segmentation technique for the extraction of gastrointestinal lumen from endoscopic images

paper_content:

A novel region-growing algorithm for the segmentation of endoscopic images is proposed in this paper. The objective of the research work is fast and accurate segmentation of gastrointestinal lumen from the endoscopic images for real-time applications. The proposed technique consists of a dual-step methodology in which a quasi Region of Interest (qROI) is segmented first using a global thresholding technique and then the actual lumen is extracted using differential region growing. An adaptive progressive thresholding technique is used to obtain qROI for a given endoscopic image. The centre of mass of qROI acts as the seed for the region growing in the next step. A differential region growing technique is suggested which grows the region on the basis of a similarity criterion. A dynamic hill-clustering method is utilised to ensure the effectiveness of the terminating condition during the growth process. The proposed scheme is faster than the conventional gradient based region-growing technique. The accuracy and high speed response of the proposed technique is validated with several endoscopic images and the results are presented.
---
paper_title: Real-time automatic extraction of lumen region and boundary from endoscopic images

paper_content:

A new approach to the automatic extraction of the lumen region and its boundary for gastrointestinal (GI) endoscopic images is presented. First, a quasi region of interest, the darker regions of the image, is segmented using a region splitting scheme termed progressive thresholding. The centre of mass of this segmented region acts as a seed for further processing. Then the lumen region is obtained using a region growing technique called the integrated neighbourhood search (INS). A new quad structure based technique is introduced to enhance the INS speed significantly. A back projection algorithm is suggested to optimise the search for pixels belonging to the lumen region and boundary. A boundary-thinning algorithm is also proposed to remove the redundant pixels from the lumen boundary and to generate a connected single pixel width boundary. The proposed approach does not need a priori knowledge about the image characteristics. The experimental results indicate that the proposed technique enhances the speed of conventional INS by 45.5% to 28.6% based on the lumen size varying from 22,709 pixels to 4947 pixels. The main advantage of the proposed technique is its high-speed response that facilitates real-time analysis of endoscopic images.
---
paper_title: 3D reconstruction of the WCE images by affine SIFT method

paper_content:

Wireless capsule endoscopy (WCE) has been gradually applied for examining the gastrointestinal (GI) tract, particular the small intestine. One WCE video comprised over 50,000 images requires more than 1.5 hours in an average for the physician to give a full assessment of the patient's GI tract. However, for the power consumption reason and the miniature-design, the captured images are suffering from the resolution and illumination invariance. Moreover, the motion of the WCE camera is uncontrollable increases the complexity of the WCE images inspiration and the time needed for analyzing the image sequences even for experienced endoscopiest. Due to these limitations, a tool is required to help physicians to make fast and reliable diagnosis. In this paper, we propose a method to reconstruct the three dimension surface of the intestinal wall by the affine scale invariant feature transform (SIFT). This algorithm can help to reconstruct the GI tract's tube-like surface within selected consecutive frames. It estimates the relative movement of the WCE by applying the affine SIFT feature detector and descriptor to a sequence of WCE images. Epipolar geometry is employed to further constrain the matching feature points in order to obtain accurate 3D view. The experiments on real WCE images are presented to show the performance of our proposed method. As a conclusion, our system can provide a more informative and friendly virtualization of the WCE images to reduce the total time spent for abnormalities detections for physicians.
---
paper_title: A novel endoscopic image analysis approach using deformable region model to aid in clinical diagnosis

paper_content:

A novel endoscopic image analysis approach using deformable region model has been proposed for clinical diagnosis. The deformable region model approach determines the lumen from the endoscopic image by giving an approximate boundary plan of the lumen by using minimum cross-entropy algorithm, and then deforming this plan to the real boundary automatically. A growing and shrinking method is employed for deforming the boundary. Connected components labeling method is used to eliminate small regions outside the lumen. The algorithm uses the grassfire concept implemented recursively. Applying connected components labeling algorithm on the inverted image eliminates small regions inside the lumen. Subsequently, deriving relevant features from the lumen information, abnormalities in the endoscopic image can be detected. The proposed scheme has been tested using a number of endoscopic images. The results obtained support the feasibility of the proposed approach and are of great interest and potential diagnostic value to the clinicians.
---
paper_title: A stable optic-flow based method for tracking colonoscopy images

paper_content:

In this paper, we focus on the robustness and stability of our algorithm to plot the position of an endoscopic camera (during a colonoscopy procedure) on the corresponding pre-operative CT scan of the patient. The colon has few topological landmarks, in contrast to bronchoscopy images, where a number of registration algorithms have taken advantage of features such as anatomical marks or bifurcations. Our method estimates the camera motion from the optic-flow computed from the information contained in the video stream. Optic-flow computation is notoriously susceptible to errors in estimating the motion field. Our method relies on the following features to counter this, (1) we use a small but reliable set of feature points (sparse optic-flow field) to determine the spatio-temporal scale at which to perform optic-flow computation in each frame of the sequence, (2) the chosen scales are used to compute a more accurate dense optic flow field, which is used to compute qualitative parameters relating to the main motion direction, and (3) the sparse optic-flow field and the main motion parameters are then combined to estimate the camera parameters. A mathematical analysis of our algorithm is presented to illustrate the stability of our method, as well as comparison to existing motion estimation algorithms. We present preliminary results of using this algorithm on both a virtual colonoscopy image sequence, as well as a colon phantom image sequence.
---
paper_title: Pathological Site Retargeting under Tissue Deformation Using Geometrical Association and Tracking

paper_content:

Recent advances in microscopic detection techniques include fluorescence spectroscopy, fibred confocal microscopy and optical coherence tomography. These methods can be integrated with miniaturised probes to assist endoscopy, thus enabling diseases to be detected at an early and pre-invasive stage, forgoing the need for histopathological samples and off-line analysis. Since optical-based biopsy does not leave visible marks after sampling, it is important to track the biopsy sites to enable accurate retargeting and subsequent serial examination. In this paper, a novel approach is proposed for pathological site retargeting in gastroscopic examinations. The proposed method is based on affine deformation modelling with geometrical association combined with cascaded online learning and tracking. It provides online in vivo retargeting, and is able to track pathological sites in the presence of tissue deformation. It is also robust to partial occlusions and can be applied to a range of imaging probes including confocal laser endomicroscopy.
---
paper_title: A System for Biopsy Site Re-targeting with Uncertainty in Gastroenterology and Oropharyngeal Examinations

paper_content:

Endoscopy guided probe-based optical biopsy is a new method for detecting sites for tissue biopsy. These sites need to be re-targeted precisely and accurately in subsequent images of the same endoscopy session for treatment or for visual guidance of surgical instruments. A new system for re-targeting biopsy sites and for characterising analytically their uncertainty is presented. It makes use of epipolar lines derived from multiple endoscopic views of the same site. It was tested on real patient data acquired during colonoscopy and gastroscopy. Gold standards of the biopsy site were obtained by Argon Plasma Coagulation tattooing. Re-targeting precision and accuracy were better than 0.8mm which is sufficient for most clinical endoscopic applications.
---
paper_title: A Comparative Study of Ego-Motion Estimation Algorithms for Teleoperated Robotic Endoscopes

paper_content:

Colorectal cancer is one of the leading causes of cancer-related mortality in the world, although it can be efficiently treated if detected early. Colonoscopy is the most-commonly adopted visual screening procedure of the colon by means of a flexible tiny endoscopic camera. In an effort to promote early screening and to facilitate mastering the endoscope motion by the physician, teleoperable robotic endoscopes and wireless capsules are being developed. In order to enable precise and fast closed-loop control for these devices, the accurate 3-D position and orientation of the camera must be known. Estimating the camera ego-motion by processing the endoscopic video provides a viable solution since it does not require the adoption of external magnetic trackers during the screening procedure that can occupy the scope’s operation channel. Furthermore, and compared to SLAM or registration approaches, ego-motion estimation algorithms do not require to deal with the highly deformable (and thus highly-uncertain) global 3D map of the colon.
---
paper_title: Specular Reflectance Suppression in Endoscopic Imagery via Stochastic Bayesian Estimation

paper_content:

A novel stochastic Bayesian estimation method is introduced for the purpose of suppressing specular reflectance in endoscopic imagery, benefiting both computer aided and manual analysis of endoscopic data. The maximum diffuse chromaticity, which is necessary for the calculation of the specular reflectance, is estimated via Bayesian least-squares minimization, with the posterior probability of maximum diffuse chromaticity given maximum chromaticity constructed via an adaptive Monte Carlo sampling approach. Experimental results using a set of clinical endoscopic imagery showed that the proposed method resulted in lower coefficient of variation values when compared to existing methods in homogeneous regions contaminated by strong specular highlights, which is indicative of improved specular reflectance suppression. These findings are further reinforced by visual assessment of the specular suppressed endoscopic imagery produced by the proposed method.
---
paper_title: A feature point extraction and matching algorithm for Wireless Capsule Endoscope image

paper_content:

For the Wireless Capsule Endoscopy (WCE), one of the most important medical actions is to find the size of the target disease tissue, on which the proper diagnosis can be made to determine the further medical operation. Since 2D estimation results in large error for the tissue size, 3D calculation is preferred. To obtain 3D information for the objective tissue, we can apply 3D reconstruction for the GI track and the target tissue. It can be realized by the localization and orientation of the endoscope by the calculation on the corresponding points on the successive images; and to do so one important step is to select the feature points and determine their corresponding positions. In this paper, we present an algorithm for image feature extraction and corresponding matching on the two successive endoscopic images. It is evaluated on the different area and different noises. The experimental results show that the algorithm has high feasibility and good accuracy.
---
paper_title: Incorporation of 3D Model and Panoramic View for Gastroscopic Lesion Surveillance

paper_content:

Natural Orifice Transluminal Endoscopic Surgery (NOTES) is widely used for clinical diagnoses. However, NOTES has two main problems: difficulties brought by endoscope’s flexibility and narrow view of endoscope. Image-guided system is helpful to deal with these problems. In our previous work, a computer aided endoscopic navigation system (CAEN) was developed for gastroscopic lesion surveillance. In this paper, 3D model and panoramic view are incorporated into CAEN with three improvements: selection of reference and tracking features; perspective projection for constructing local and global panoramic view; 3D surface modeling using structure from motion. The system is evaluated from three clinic applications: broadening the view, non-invasive retargeting, and overall lesion locations. The evaluation results show that the mean accuracy of broadening the view is 0.43 mm, the mean accuracy of non-invasive retargeting is 7.5 mm, and the mean accuracy for overall lesion diagnosis is 3.71 ± 0.35 mm.
---
paper_title: A recursive Otsu-Iris filter technique for high-speed detection of lumen region from endoscopic images

paper_content:

In this paper, a hardware efficient technique to segment the lumen region from endoscopic images is presented. It is based on applying the combined Otsu-Iris filter operations recursively. The proposed technique applies the Otsu's procedure recursively to obtain a coarse region of interest (ROI), which is then subjected to an Iris filter operation so that a smaller enhanced region can be identified. This enhanced region is subjected to the Otsu's procedure recursively and the process of performing Iris filter operation is repeated as before. It has been shown that by repeating this Otsu-Iris filter combination in an iterative manner facilitates in the rapid identification of the lumen region accurately. It has been shown that the proposed method substantially reduces the number of computations of Otsu's procedure when compared with the APT-Iris filter method. Finally, unlike the APT-Iris filter method, it does not require the precomputation of the cumulative limiting factor, which is highly dependent on the complex endoscopic images.
---
paper_title: Quality Improvement of Endoscopy Videos

paper_content:

We address two quality issues in endoscopy videos originating from: (a) Colour Channel Misalignment: In current endoscope systems, the colour channel images are most commonly acquired sequentially at different time instances. Whenever the camera moves significantly between acquisition instances, the colour channels get misaligned. This misalignment degrades the quality of the video due to the appearance of stroboscopic artefacts. (b) Specular Highlights: The surfaces of human organs, visualised in endoscopy videos, usually have a glossy appearance which causes specular highlights in the images. For many image analysis algorithms, these distinct and bright artefacts can become a significant source of error. In this paper, we propose two novel algorithms to remove the aforementioned artefacts. The colour channel misalignment artefacts are removed by estimating the camera motion in the time interval between the acquisition of two colour channel images. The issue of specular highlights is addressed using: (i) a segmentation method based on nonlinear filtering and colour image thresholding and (ii) a fast inpainting method. Both algorithms are evaluated using image sets extracted from colonoscopy videos. The colour channel realignment algorithm achieved a success rate of 86% and 78% in image sets with artificial and real misalignment, respectively. When specular highlights are a priori removed by the proposed algorithm, these success rates increase to 92% and 84%, respectively. The specular highlight removal algorithm achieved an accuracy of 91.99%.
---
paper_title: 3D reconstruction of GI tract texture surface using Capsule Endoscopy Images

paper_content:

Wireless Capsule Endoscopy (CE) is a novel technique to screen the entire gastrointestinal (GI) tract, especially the small bowel. It was invented by Given Imaging in 2000 to solve the problem that the conventional tools cannot explore the entire small bowel. As it is non-invasive, reliable, and safe, it has been widely applied around the world. Although the clinical findings of CE are promising, there still exist some limitations. The main problem is that it produces over 50,000 images during one examination. It is a highly time-consuming and labor-intensive task to inspect such long video for clinicians. Moreover, CE images are lack of prominent characteristics, which make it difficult to diagnose diseases even for an experienced clinician. So it is desirable and necessary to develop a tool to assist clinicians during diagnosis. In this paper, we propose a method to reconstruct the 3D texture surface of GI tract using single CE image. We applied Shape from Shading (SfS) technique to represent the surface of GI tract using 2D CE images. It is a novel way to view the abnormalities, especially for polyp, which can lead to a more efficient examination. Preliminary experiments on real CE images have been conducted. The results show that our method achieved promising performance.
---
paper_title: Biopsy Site Re-localisation Based on the Computation of Epipolar Lines from Two Previous Endoscopic Images

paper_content:

Tracking biopsy sites in endoscopic images can be useful to provide a visual aid for the guidance of surgical tools, for example when endoscopic guided biopsy is required. A new method for re-localisation of these sites is presented in this paper. It makes use of epipolar geometry properties between three images of the same site observed from different viewpoints with an endoscope. Two epipolar lines are derived from the two first images in the third image where the site needs to be re-localised. Their intersection corresponds to the location of the biopsy site. This method was tested with gastroscopic data from 2 patients with 9 series of three images of the oesophagus. The re-localisation error was estimated at less than 1.5 millimetres by a clinical endoscopist, which is sufficient for most clinical endoscopic applications.
---
paper_title: Gastroscopic Image Graph: Application to Noninvasive Multitarget Tracking under Gastroscopy

paper_content:

Gastroscopic examination is one of the most common methods for gastric disease diagnosis. In this paper, a multitarget tracking approach is proposed to assist endoscopists in identifying lesions under gastroscopy. This approach analyzes numerous preobserved gastroscopic images and constructs a gastroscopic image graph. In this way, the deformation registration between gastroscopic images is regarded as a graph search problem. During the procedure, the endoscopist marks suspicious lesions on the screen and the graph is utilized to locate and display the lesions in the appropriate frames based on the calculated registration model. Compared to traditional gastroscopic lesion surveillance methods (e.g., tattooing or probe-based optical biopsy), this approach is noninvasive and does not require additional instruments. In order to assess and quantify the performance, this approach was applied to stomach phantom data and in vivo data. The clinical experimental results demonstrated that the accuracy at angularis, antral, and stomach body was 6.3 ± 2.4 mm, 7.6 ± 3.1 mm, and 7.9 ± 1.6 mm, respectively. The mean accuracy was 7.31 mm, average targeting time was 56 ms, and the value was 0.032, which makes it an attractive candidate for clinical practice. Furthermore, this approach provides a significant reference for endoscopic target tracking of other soft tissue organs.
---
paper_title: Computer-assisted bleeding detection in wireless capsule endoscopy images

paper_content:

Wireless capsule endoscopy (WCE) has revolutionised the diagnosis and treatment of gastrointestinal tract, especially the small intestine where traditional endoscopies cannot reach. However, this new technology leads to the inspection of a large number of images, which is a time-consuming process and also too hard by naked eyes for doctors. In this paper, we propose a new computerised method for bleeding detection in WCE images. We use the second component of CIE Lab colour space together with appropriate segmentation and enhancement techniques, involving an adaptive anisotropic diffusion (alike Perona–Malik diffusion). As a result of this procedure, it is possible to devise four functions to discriminate between bleeding and normal regions in WCE images. These four bleeding detectors rely on the eigenvalues of the Hessian and on the Laplacian of the modified enhanced image. Multiscale image analysis approach is also involved in the definition of these detectors for handling the maximum and minimum sizes ...
---
paper_title: Image-Enhanced capsule endoscopy preserving the original color tones

paper_content:

This paper describes a technique to enhance a region-of- interest (ROI) in a capsule endoscopy (CE) image. Our aim is to improve distinguishing suspicious regions from normal ones without over-enhancing the image. Given a ROI by physicians, the proposed technique enhances the ROI so that the enhanced region remains as natural as possible. To achieve this, we utilize image features relating to a specific color space dedicated to gastrointestinal (GI) wall regions. The proposed approach starts by utilizing a self-organizing map to handle GI wall color components. The goal of preserving the original color tones is obtainable by using a histogram equalization technique in the proposed color space. As a result, the enhanced images only contain color components inferred from the color gamut of the human small bowel. The results of the proposed method are judged in terms of visual appearance by comparison with images obtained by the conventional enhancement technique.
---
paper_title: Wireless capsule endoscopy images enhancement via adaptive contrast diffusion

paper_content:

Wireless capsule endoscopy (WCE) has been widely applied to diagnose diseases in human digestive tract due to its advantage that it can directly view the entire small intestine for the first time. However, many WCE images are rather dark, which challenge to analysis and diagnosis exerted by a clinician. To overcome this shortcoming so as to assist physicians, especially computer aided detection, we propose an adaptive contrast diffusion to enhance WCE images. Based on local analysis of WCE images, we put forward a new idea of contrast diffusion. Then, we employ contrast diffusion to enhance WCE images with an adaptive choice of the conductance parameter, which plays an important role in diffusion. Extensive experiments demonstrate that this new method exhibits promising performance of enhancement for WCE images, leading into a better visualization as well as an improved classification performance of WCE images using computerized methods.
---
paper_title: Computer-aided capsule endoscopy images evaluation based on color rotation and texture features: An educational tool to physicians

paper_content:

Wireless capsule endoscopy (WCE) is a revolutionary, patient-friendly imaging technique that enables non-invasive visual inspection of the patient's digestive tract and, especially, small intestine. However, reviewing the endoscopic data is time consuming and requires intense labor of highly experienced physicians. These limitations were the motive to propose a novel strategy for automatic discrimination of WCE images related to ulcer, the most common finding of digestive tract. Towards this direction, WCE data are color-rotated in order to boost the chromatic attributes of ulcer regions. Then, texture information is extracted by utilizing the local binary pattern operator that analyses the spatial structure of the images at a very local level. Experimental results demonstrated promising classification accuracy (91.1%) exhibiting high potential towards a complete computer-aided diagnosis system that will not only reduce WCE data reviewing time, but also serve as an assisting tool for the training of inexperienced physicians.
---
paper_title: Cable Footprint History: Spatio-Temporal Technique for Instrument Detection in Gastrointestinal Endoscopic Procedures

paper_content:

We propose a new fast spatio-temporal technique that detects an operation scene---a video segment corresponding to a single purpose diagnosis action or a single purpose therapeutic action. The technique utilizes (1) color contrast of the cable region and the background, (2) the new area-based coordinate system to compute spatial features, and (3) the history of locations of detected cables of the instrument in a video to discard false regions. The proposed technique and software are useful for (1) automatic documentation of diagnostic or therapeutic operations at the end of the procedure, (2) a second review for causes of complications due to these operations, and (3) a building block for an effective content-based retrieval system to facilitate endoscopic research and education. On 38 fulllength colonoscopy and upper endoscopy video files with different cable colors and four different insertion directions of the instruments, the average percentage of false positive duration is small at 2.23%. The average percentage of true positive duration is high at 92.9%. The average analysis time per image is 13 milliseconds on an inexpensive off-the-shelf PC.
---
paper_title: WCE video segmentation using textons

paper_content:

Wireless Capsule Endoscopy (WCE) integrates wireless transmission with image and video technology. It has ::: been used to examine the small intestine non invasively. Medical specialists look for signicative events in the ::: WCE video by direct visual inspection manually labelling, in tiring and up to one hour long sessions, clinical ::: relevant frames. This limits the WCE usage. To automatically discriminate digestive organs such as esophagus, ::: stomach, small intestine and colon is of great advantage. In this paper we propose to use textons for the ::: automatic discrimination of abrupt changes within a video. In particular, we consider, as features, for each ::: frame hue, saturation, value, high-frequency energy content and the responses to a bank of Gabor filters. The ::: experiments have been conducted on ten video segments extracted from WCE videos, in which the signicative ::: events have been previously labelled by experts. Results have shown that the proposed method may eliminate up ::: to 70% of the frames from further investigations. The direct analysis of the doctors may hence be concentrated ::: only on eventful frames. A graphical tool showing sudden changes in the textons frequencies for each frame is also proposed as a visual aid to find clinically relevant segments of the video.
---
paper_title: Capsule endoscopy video segmentation by spectral clustering

paper_content:

Capsule endoscopy is a new imaging technology for small intestine due to its breakthrough for direct visualization of small intestine for the first time. However, the video data produced for each patient costs a physician much time to inspect. Aiming for reducing the burden of a physician, video scene analysis is indispensable. In this paper, we propose a new video segmentation method to analyze a CE video data since video segmentation is the first step in video scene analysis. A novel color textural feature is utilized to describe the content of the frame in a CE video, then spectral clustering method is applied to segment a CE video into meaningful parts via shot boundary detection. Preliminary experiments on ten short CE videos demonstrate a promising performance of the proposed scheme.
---
paper_title: Wireless Capsule Endoscopy Video Automatic Segmentation

paper_content:

Wireless capsule endoscopy (WCE) is an advanced technology that allows diagnosis inside human’s digestive tract without invasiveness, however, it is a time-consuming task for clinicians to diagnose due to the large number of frames in video. A novel and efficient algorithm is proposed in this paper to help clinicians segment the WCE video automatically according to tomach, small intestine, and large intestine regions. Firstly, since there are many impurities and bubbles in WCE video frames which add the difficulty of segmentation, a pre-procedure is presented to denote the valid regions in the ::: frames based on color and wavelet texture features. Secondly, the boundaries between adjacent organs of WCE video are estimated in two levels which consist of a rough and a fine level. In the rough level, color feature is utilized to draw a dissimilarity curve between frames and the aim is to find the peak of the curve,which represents the approximate boundary we want to locate. In the fine level, Hue-Saturation histogram color feature in HSI color space and uniform LBP texture feature from grayscale images are extracted. And support vector machine (SVM) classifier is utilized to segment the WCE video into different regions. The experiments demonstrate a promising efficiency of the proposed algorithm and the average precision and recall achieve as high as 94.33% and 89.50% respectively.
---
paper_title: Impact of Visual Features on the Segmentation of Gastroenterology Images Using Normalized Cuts

paper_content:

Gastroenterology imaging is an essential tool to detect gastrointestinal cancer in patients. Computer-assisted diagnosis is desirable to help us improve the reliability of this detection. However, traditional computer vision methodologies, mainly segmentation, do not translate well to the specific visual characteristics of a gastroenterology imaging scenario. In this paper, we propose a novel method for the segmentation of gastroenterology images from two distinct imaging modalities and organs: chromoendoscopy (CH) and narrow-band imaging (NBI) from stomach and esophagus, respectively. We have used various visual features individually and their combinations (edgemaps, creaseness, and color) in normalized cuts image segmentation framework to segment ground truth datasets of 142 CH and 224 NBI images. Experiments show that an integration of edgemaps and creaseness in normalized cuts gives the best segmentation performance resulting in high-quality segmentations of the gastroenterology images.
---
paper_title: Intestinal event segmentation for endoluminal video analysis

paper_content:

In this paper, we tackle the problem of unsupervised segmentation of intestinal events using various motility descriptors extracted from the endoluminal video. The segments of constant intestinal activity are detected with a robust statistical test that is based on Hoeffding's inequality. The qualitative analysis of the results shows that the segments adjust well to the motility information and thus, this segmentation can constitute basic units for higher-level motility description.
---
paper_title: Automated Segmentation of Endoscopic Images Based on Local Shape-Adaptive Filtering and Color Descriptors

paper_content:

This paper presents a novel technique for automatic segmentation of wireless capsule endoscopic images. The main contribution resides in the integration of three computational blocks: 1) local polynomial approximation algorithm which finds locally-adapted neighborhood of each pixel; 2) color texture analysis which describes each pixel by a vector of numerical attributes that reflect this pixel local neighborhood characteristcs; and 3) cluster analysis (k-means) for grouping pixels into homgeneous regions based on their color information. The proposed approach leads to a robust segmentation procedure which produces fine segments well matched to the image contents.
---
paper_title: Stomach, intestine, and colon tissue discriminators for wireless capsule endoscopy images

paper_content:

Wireless Capsule Endoscopy (WCE) is a new colour imaging technology that enables close examination of the interior of the entire small intestine. Typically, the WCE operates for ~8 hours and captures ~40,000 useful images. The images are viewed as a video sequence, which generally takes a doctor over an hour to analyse. In order to activate certain key features of the software provided with the capsule, it is necessary to locate and annotate the boundaries between certain gastrointestinal (GI) tract regions (stomach, intestine and colon) in the footage. In this paper we propose a method of automatically discriminating stomach, intestine and colon tissue in order to significantly reduce the video assessment time. We use hue saturation chromaticity histograms which are compressed using a hybrid transform, incorporating the Discrete Cosine Transform (DCT) and Principal Component Analysis (PCA). The performance of two classifiers is compared: k-nearest neighbour ( kNN ) and Support Vector Classifier (SVC). After training the classifier, we applied a narrowing step algorithm to converge to the points in the video where the capsule firstly passes through the pylorus (the valve between the stomach and the intestine) and later the ileocaecal valve (IV, the valve between the intestine and colon). We present experimental results that demonstrate the effectiveness of this method.
---
paper_title: Colour Image Analysis of Wireless Capsule Endoscopy Video: A Review

paper_content:

Wireless capsule endoscopy (CE) has been available since 2001 and is now established as one of the principal approaches used to examine the small bowel, with a range of devices available from four manufacturers. But although its use is widespread the reading of CE videos remains an arduous and time consuming exercise for gastroenterologists because relevant frames which are of interest to the physician constitute only about 1 % of the video. CE exam viewing times vary from 40–90 minutes, depending on the clinician’s experience, the complexity of the case and the small bowel transit time. Colour image analysis has been applied by manufacturers to speed up this process, for example, Given Imaging’s Rapid Reader Software includes a suspected blood indicator (SBI) designed to detect bleeding in the video. However, some evaluations of this tool reported concerns with regard to its specificity and sensitivity and so it does not free the specialist from reviewing the entire footage and can only be used as a fast screening aid. Over the past decade a number of papers have proposed a range of colour image processing and computer vision applications to assist the gastroenterologist in the analysis of CE video. These techniques can be divided into three categories, the first considers the topographic segmentation of CE video into meaningful parts such as mouth, oesophagus, stomach, small intestine, and colon. The second involves the detection of clinically significant video events (both abnormal and normal) and the third attempts to adaptively adjust the video viewing speed. This chapter reviews this research focusing particularly on the role of colour and texture descriptors and concludes by suggesting possible future directions for CE analysis.
---
paper_title: Homography-based orientation estimation for capsule endoscope tracking

paper_content:

Wireless capsule endoscopy (WCE) is a developing imaging technology for screening the gastrointestinal system. Tracking of the swallowable capsule endoscope within the human body is typically performed by external sensor arrays enabling approximate localization and detection of its orientation. In this paper we propose a method for capsule orientation estimation based only on image features, without the requirement of any external equipment. The proposed method involves two steps: a) salient point detection and extraction of image features from these points, and b) detection of feature correspondences between consecutive frames and homography estimation. The capsule orientation is estimated by decomposition of the homography matrix into rotation and translation components. The results obtained by the application of this method on WCE video frames indicate its effectiveness and improved performance over the state of the art.
---
paper_title: A Method of Bubble Removal for Computer-Assisted Diagnosis of Capsule Endoscopic Images

paper_content:

Capsule endoscopy can take tens of thousands of pictures at one examination, and it is impossible for physicians to diagnose such huge number of pictures. Thus, studies of computer-assisted diagnosis of capsule endoscopic images have been conducted. Computer-assisted diagnosis includes detection of lesion areas. Besides lesion areas, capsule endoscopic images contain areas of residue, intestinal juice, and bubbles, which can be an obstacle to diagnosis of the intestinal wall. To improve the performance of diagnosis, we propose a method of removing bubbles in capsule endoscopic images. Experimental results show that the proposed method works well.
---
paper_title: Hierarchical key frame extraction for wireless capsule endoscopy video based on the saliency map

paper_content:

Wireless capsule endoscopy (WCE) is a non-invasive imaging technique to examine the entire small intestine. This paper presents a novel hierarchical key frame extraction algorithm based on the saliency map to automatically select a small number of key informative frames in the WCE video to assist the diagnostic process. We start from calculating the saliency map and extracting the inside colour, texture and shape features. We then compute the image information entropies with adapted weights on three features; calculate the differences of entropies in the successive frames and extract the first level key frames to divide the video into multi-segments. Finally, the mean shift clustering method is applied on each segment to obtain the final key frames. The proposed technique achieves the fidelity of 95.07% and the compression ratio of 87.9% on average, validating that the proposed scheme is highly promising for the key frame extraction in the WCE video.
---
paper_title: Panoramic Visual Summaries for Efficient Reading of Capsule Endoscopy Videos

paper_content:

Wireless capsule endoscopy (WCE) is a minimally invasive procedure enabling visual examination of the entire gastrointestinal tract. It is performed by a swallow able camera-equipped capsule wirelessly transmitting color video frames. Reading the resulting video is a challenging process for the endoscopists. It requires a considerable amount of time, usually hours of undivided attention and experience to detect and correctly interpret findings. In this paper we propose a novel method for automatic construction of panoramic visual summaries of WCE video frames enabling efficient reading of the video, while providing a broader field of view for the endoscopists. It involves extraction of visual features from each frame, and frame registration. Consecutive frames are geometrically transformed and stitched together to form new sequences of panoramic frames, each of which summarizes the information contained in multiple original ones. The proposed method has been integrated with Java Video Analysis (JVA), a novel software development framework for video applications. Its experimental evaluation on real, publicly available WCE video clips show that its application can result in a reduction of 84.5%of the video frames to be examined by the endoscopes.
---
paper_title: Automatic frame reduction of Wireless Capsule Endoscopy video

paper_content:

Wireless Capsule Endoscopy (WCE) is a non-invasive colour imaging technique that has been introduced for the screening of the gastrointestinal tract and especially the small intestine. WCE is performed by a wireless swallowable endoscopic capsule that transmits more than 50,000 video frames per examination. The visual inspection of the resulting video is a highly time-consuming task even for the experienced gastroenterologist. In this paper we propose a novel WCE video summarization approach which is subsequently evaluated using real world patient data. The proposed approach aims to the reduction of the number of the video frames to be visually inspected so as to enable significant reduction in the video assessment time. It is based on clustering using symmetric non-negative matrix factorization initialized by the fuzzy c-means algorithm and supported by non-negative Lagrangian relaxation to extract a subset of video scenes containing the most representative frames from an entire examination. Real world patient data that display abnormal findings at several sites in the small intestine were annotated by expert gastroenterologists in order to experimentally evaluate the proposed approach. The results demonstrate that the suggested approach leads to significant reduction of the total number of frames in the input video without losing critical information related to the abnormal regions of the small intestine.
---
paper_title: Computer-assisted Diagnosis for Precancerous Lesions in the Esophagus

paper_content:

Objectives: The interpretation of endoscopic findings by gastroenterologists is still a difficult and highly subjective task. Despite important developments such as chromo-endoscopy, pit pattern analysis, fluorescence imaging as well as narrow band imaging it still requires lots of experience and training with a certain tentativeness until the final biopsy. By the development of computer-assisted diagnosis (CAD) systems this process can be supported. Methods: This paper presents a new approach to CAD for precancerous lesions in the esophagus based on color-texture analysis in a content-based image retrieval (CBIR) framework. The novelty of our approach lies in the combination of newly developed color-texture features with the interactive feedback loop provided by a relevance feedback algorithm. This allows the expert to steer the query and is still robust against accidental false decisions. Results: We reached an inter-rater reliability of k = 0.71 on a database of 390 endoscopic images. The retrieval accuracy didn’t change significantly until a wrong decision rate of 20%. Conclusions: Thus, the system could be able to support practitioners with less experience or in private practice. In combination with a connected case database it can also support case-based reasoning for the diagnostic decision process.
---
paper_title: Comparison of k-NN, SVM, and NN in Pit Pattern Classification of Zoom-Endoscopic Colon Images using Co-Occurrence Histograms

paper_content:

Co-occurrence histograms are used as features to classify magnifying endoscope imagery with k-NN, SVM, and NN classifiers. In the k-NN classification case these histograms may improve the classification accuracy of simple ID color histograms up to 10% in the 2 classes case and up to 5% in the 6 classes case. The classification results of SVM and NN classifiers have turned out to be noncompetitive and do not improve the classification result of ID color histograms.
---
paper_title: Feature selection and classification for Wireless Capsule Endoscopic frames

paper_content:

Wireless Capsule Endoscopy (WCE) is an important device to detect abnormalities in small intestine. Despite emerging technologies, reviewing capsule endoscopic video is a labor intensive task and very time consuming. Computational tools which automatically detect informative frames and tag abnormal conditions such as bleeding, ulcer or tumor will dramatically reduce the clinician's effort. In this paper, we explored various machine-learning methodologies based on different feature extraction and selection criteria, and developed an optimized classification method. The experiment results shows that, comparing to texture feature, using color feature for classification achieved better accuracy, regardless of machine-learning method chosen. The proposed method has been applied in real data taken from capsule endoscopic exams. For informative frames detection, classification method using color feature gives an accuracy of 94.10% and 93.44% for support vector machines (SVM) and neural network (NN) classifiers respectively. For the bleeding detection using color feature, the accuracy achieved 99.41% and 98.97% for SVM and NN respectively. In addition, we also investigated the computational time required for feature extraction and classification. In our experiments, color feature significantly outperformed texture feature in WCE image classification. The overall computational time (per frame) using color feature is 0.7125s (informative frame with SVM), 1.0329s (informative frame with NN), 0.51s (bleeding frame with SVM) and 1.2163s (bleeding frame with NN). Classifiers for more gastro-intestinal (GI) diseases detection will be developed based on this work subsequently.
---
paper_title: A fusion-based clinical decision support for disease diagnosis from endoscopic images

paper_content:

Abstract This paper presents an intelligent decision support system designed on a decision fusion framework coupled with a priori knowledge base for abnormality detection from endoscopic images. Sub-decisions are made based on associated component feature sets derived from the endoscopic images and predefined algorithms, and subsequently fused to classify the patient state. Bayesian probability computations are employed to evaluate the accuracies of sub-decisions, which are utilized in estimating the probability of the fused decision. The overall detectability of abnormalities by using the proposed fusion approach is improved in terms of detection of true positive and true negative conditions when compared with corresponding results from individual methods.
---
paper_title: Predicting the histology of colorectal lesions in a probabilistic framework

paper_content:

In this paper, we present a novel approach to predict the histological diagnosis of colorectal lesions from high-magnification colonoscopy images by means of Pit Pattern analysis. Motivated by the shortcomings of discriminant classifier approaches, we present a generative model based strategy which is closely related to content-based image retrieval (CBIR) systems. The ingredients of the approach are the Dual-Tree Complex Wavelet Transform (DTCWT) and the mathematical construct of copulas. Our experimental study on a set of 627 images confirms, that the joint statistical model leads to impressive prediction results compared to previous work.
---
paper_title: Complex Wavelet Transform variants and Discrete Cosine Transform for scale invariance in magnification-endoscopy image classification

paper_content:

In this paper, scale invariant features are extracted from different variants of the Dual-Tree Complex Wavelet Transform (DT-CWT) in order to classify high-magnification colon endoscopy imagery with respect to the pit pattern scheme. To enhance the scale invariance, the Discrete Cosine Transform is applied to the feature vectors, that are achieved from a DT-CWT variant. The feature vectors either consist of the means and standard deviations of the subbands from a DTC-WT variant or of the Weibull parameter of these subbands. Superior results as compared to techniques described previously in literature are reported.
---
paper_title: An intelligent system for automatic detection of gastrointestinal adenomas in video endoscopy

paper_content:

Today 95% of all gastrointestinal carcinomas are believed to arise from adenomas. The early detection of adenomas could prevent their evolution to cancer. A novel system for the support of the detection of adenomas in gastrointestinal video endoscopy is presented. Unlike other systems, it accepts standard low-resolution video input thus requiring less computational resources and facilitating both portability and the potential to be used in telemedicine applications. It combines intelligent processing techniques of SVMs and color–texture analysis methodologies into a sound pattern recognition framework. Concerning the system’s accuracy this was measured using ROC analysis and found to exceed 94%. 2005 Elsevier Ltd. All rights reserved.
---
paper_title: Evaluation of cross-validation protocols for the classification of endoscopic images of colonic polyps

paper_content:

We evaluate different cross-validation (CV) protocols for an automated classification of colonic polyps. For this purpose we select six previously developed methods which achieved promising results already in the past. We then evaluate the methods using the cross-validation protocols leave-one-image-out (LOO-CV), leave-one-parent-image-out (LOPIO-CV), leave-one-lesion-out (LOLO-CV), and leave-one-patient-out (LOPO-CV). We show that, in general, the more restrictive cross-validation protocols lead to high results drops. While in case of LOO-CV the accuracies are rather high across all methods evaluated, the picture changes the more strictness a cross-validation mode imposes on the set of training images.
---
paper_title: Automated Real-time Objects Detection in Colonoscopy Videos for Quality Measurements

paper_content:

The effectiveness of colonoscopy depends on the quality of the inspection of the colon. There was no automated measurement method to evaluate the quality of the inspection. This thesis addresses this issue by investigating an automated post-procedure quality measurement technique and proposing a novel approach automatically deciding a percentage of stool areas in images of digitized colonoscopy video files. It involves the classification of image pixels based on their color features using a new method of planes on RGB (red, green and blue) color space. The limitation of post-procedure quality measurement is that quality measurements are available long after the procedure was done and the patient was released. A better approach is to inform any sub-optimal inspection immediately so that the endoscopist can improve the quality in real-time during the procedure. This thesis also proposes an extension to post-procedure method to detect stool, bite-block, and blood regions in real-time using color features in HSV color space. These three objects play a major role in quality measurements in colonoscopy. The proposed method partitions very large positive examples of each of these objects into a number of groups. These groups are formed by taking intersection of positive examples with a hyper plane. This hyper plane is named as 'positive plane'. 'Convex hulls' are used to model positive planes. Comparisons with traditional classifiers such as K-nearest neighbor (K-NN) and support vector machines (SVM) proves the soundness of the proposed method in terms of accuracy and speed that are critical in the targeted real-time quality measurement system.
---
paper_title: Endoscopic Image Classification Using Edge-Based Features

paper_content:

We present a system for an automated colon cancer detection based on the pit pattern classification. In contrast to previous work we exploit the visual nature of the underlying classification scheme by extracting features based on detected edges. To focus on the most discriminative subset of features we use a greedy forward feature subset selection. The classification is then carried out using the k-nearest neighbors (k-NN) classifier. The results obtained are very promising and show that an automated classification of the given imagery is feasible by using the proposed method.
---
paper_title: Degradation adaptive texture classification

paper_content:

Image degradations such as noise, blur and scale-variations are known to significantly affect the classification process of textured images. However, due to difficult visual according conditions, such degradation are often prevalent in digital real-world images. We show that these degradations not necessarily strongly affect the discriminative powers of features, in a scenario where similarly degraded images are classified. Contrarily, if the training and the evaluation set contain differently degraded images, the accuracies are decreasing extremely. In this paper, we exploit this knowledge and propose an approach which divides one large database into several smaller ones, each containing similarly degraded images. In order to get sensible database divisions, we use criteria adapted to the respective degradation. In experiments with several degradations, classifiers and feature extraction methods, we show that our method continuously and significantly enhances the classification accuracies.
---
paper_title: Feature extraction from multi-directional multi-resolution image transformations for the classification of zoom-endoscopy images

paper_content:

In this article, we discuss the discriminative power of a set of image features, extracted from detail subbands of the Gabor wavelet transform and the dual-tree complex wavelet transform for the purpose of computer-assisted zoom-endoscopy image classification. We incorporate color channel information into the classification process and show that this leads to superior classification results, compared to luminance-channel-only-based image analysis.
---
paper_title: Analysis of Wireless Capsule Endoscopy Images using Local Binary Patterns

paper_content:

Wireless capsule endoscopy, the gold standard in the screening and diagnosis of small bowel diseases, is one of the most recent investigations for gastrointestinal pathology. This examination has the advantages of being non-invasive, painless, with a large clinical yield, especially for small bowel diseases, but also some disadvantages. The long time necessary for reading and interpreting all frames acquired is one of these disadvantages. This inconvenient could be improved through different methods by using software applications. In this study we have used a software application for texture analysis based on local binary pattern (LBP) operator. This operator detects and removes non-informative frames in a first step, then identifies potential lesions. Our study group consisted of 33 patients from the Gastroenterology and Hepatology Centre Craiova and from the 1st Internal Medicine and Gastroenterology Clinic from the Emergency County Hospital of Craiova. The patients included in the study have corresponded to our inclusion criteria established. The exclusion criteria were represented by the contraindications of the capsule endoscopy. In the first phase of the study, we have removed the non-informative frames from the original videos obtained, and we have acquired an average reduction of 6.96% from the total number of images. In the second phase, using the same LBP operator, we have correctly identified 93.16% of telangiectasia lesions. Our study demonstrated that software applications based on LBP operator can lead to a shorter analysis time, by reducing the overall frames number, and can also provide support in diagnosis.
---
paper_title: Learning pit pattern concepts for gastroenterological training

paper_content:

In this article, we propose an approach to learn the characteristics of colonic mucosal surface structures, the so called pit patterns, commonly observed during high-magnification colonoscopy. Since the discrimination of the pit pattern types usually requires an experienced physician, an interesting question is whether we can automatically find a collection of images which most typically show a particular pit pattern characteristic. This is of considerable practical interest, since it is imperative for gastroenterological training to have a representative image set for the textbook descriptions of the pit patterns. Our approach exploits recent research on semantic image retrieval and annotation. This facilitates to learn a semantic space for the pit pattern concepts which eventually leads to a very natural formulation of our task.
---
paper_title: Endoscopic Image Retrieval System Using Multi-scale Image Features

paper_content:

We present a novel Content Based Medical Image Retrieval (CBMIR) scheme for color endoscopic images using Multi-scale Geometric Analysis (MGA) of Nonsubsampled Contourlet Transform (NSCT) and the statistical framework based on Generalized Gaussian Density (GGD) model and Kullback-Leibler Distance (KLD). The subband images obtained from the NSCT decomposition are divided into number of blocks and then the coefficients of each block of each subband is modeled with GGD parameters and computing the similarity using the KLD among the model parameters. The retrieval performance of the proposed system is further improved using Least Square-Support Vector Machine (LSSVM) classifier. Extensive experiments were carried out to evaluate the effectiveness of the proposed system on endoscopic image databases consisting of 276 images. Experimental results show that the proposed CBMIR system performs efficiently in image retrieval paradigm.
---
paper_title: Arthemis: Annotation software in an integrated capturing and analysis system for colonoscopy

paper_content:

Colonoscopy is an endoscopic technique that allows physicians to inspect the inside of the human colon. During a colonoscopic procedure, a tiny video camera at the tip of the endoscope generates a video signal of the internal mucosa of the colon. In current practice, the entire colonoscopic procedure is not routinely captured. Software tools providing easy access to important contents of videos that are digitally captured during colonoscopy are not available. Hence, it is very time consuming to review an entire video, locate important contents, annotate them, and extract the annotated contents for research, teaching, and training purposes. Arthemis, a software application, was developed to facilitate this process. For convenient data sharing, Arthemis allows annotation according to the European Gastrointestinal Society for Endoscopy (ESGE) Minimal Standard Terminology (MST), an internationally accepted standard for digestive endoscopy. Arthemis is part of our integrated capturing and content analysis system for colonoscopy called Endoscopic Multimedia Information System (EMIS). This paper presents Arthemis as a component of EMIS, the design and implementation of Arthemis, and key lessons learned from the development process.
---
paper_title: Biomedical Image Analysis on Wireless Capsule Endoscopy Images and Videos

paper_content:

Wireless capsule endoscopy (WCE) is a newly booming technology on gastrointestinal (GI) examinations. After the patient swallows the capsule, it starts taking video of the entire interior of the digestive tract, from the esophagus, stomach to small intestine, and colon. It transmits sequences of color images back to the computer, and hence allows the doctors to observe the patient’s GI tract vividly and hence diagnose precisely based on the visually accessed video instead of reconstructed images such as MRI. The entire process takes hours with around 50,000 images about the inside of the GI tract. It is very time-consuming to go through those images frame by frame. Therefore, many image/video processing techniques are adopted to develop software which is expected to annotate the images in different stage of the GI tract and detect unusual events, such as bleeding, polyps, and ulcers, automatically. In another words, the problems in WCE images/videos can be categorized in two classes, video segmentations and event detections. Various methods have been discussed in the literatures and generally can be seen as two-stage approaches, describing images and classifying images or regions. In this chapter, we will review various feature descriptors and classification methods that are often used on WCE images/videos.
---
paper_title: Deep sparse feature selection for computer aided endoscopy diagnosis

paper_content:

In this paper, we develop a computer aided diagnosis algorithm to detect and classify the abnormalities in vision-based endoscopic examination. We focus on analyzing the traditional gastroscope data and help the medical experts improve the accuracy of medical diagnosis with our analysis tool. To achieve this, we first segment the image into superpixels, then extract various color and texture features from them and combine the features into one feature vector to represent the images. This approach is more flexible and accurate than the traditional patch-based image representation. Then we design a novel feature selection model with group sparsity, Deep Sparse SVM (DSSVM) that not only can assign a suitable weight to the feature dimensions like the other traditional feature selection models, but also directly exclude useless features from the feature pool. Thus, our DSSVM model can maintain the accuracy while reducing the computation complexity. Moreover, the image quality is also pre-assessed. For the experiments, we build a new gastroscope dataset with a total of about 3800 images from 1284 volunteers, and conducted various experiments and comparisons with other algorithms to justify the effectiveness and efficiency of our algorithm. HighlightsWe design a new feature selection model via group sparsity, named Deep Sparse SVM.We design a general computer aided endoscopy diagnosis framework.We build a new endoscopy dataset including 10k+ images from 1284 volunteers.We annotate 3.8k+ images with pixel-level and frame-level groundtruth.
---
paper_title: Colour and Texture Based Gastrointestinal Tissue Discrimination

paper_content:

Wireless Capsule Endoscopy is a colour imaging technology that enables close examination of the interior of the entire small intestine. The Wireless Capsule Endoscope (WCE) operates for ∼ 8 hours and captures ∼ 40, 000 useful images. The images are viewed by a clinician as a video sequence, generally taking over an hour to analyse. In this paper we present a method of automatically discriminating stomach and intestine tissue which can significantly speed-up one key part of the video analysis time, namely the process of locating the Pylorus- the valve between the stomach and the intestine. We divide the WCE image into 28 sub-regions and process only those regions where tissue is clearly visible. We create a feature vector using colour and texture information. The colour features are derived from Hue Saturation chromaticity histograms of the useful regions, compressed using a hybrid transform, incorporating the Discrete Cosine Transform (DCT) and Principal Component Analysis (PCA). The texture features are derived by Singular Value Decomposition of the same tissue regions. After training the Support Vector classifier, we apply a discriminator algorithm, which scans the video with an increasing step and builds up a classification result sequence. By minimizing the number of misclassifications within this sequence, we predict the most probable position of the Pylorus. We present experimental results that demonstrate the effectiveness of this method.
---
paper_title: Making texture descriptors invariant to blur

paper_content:

Besides a high distinctiveness, robustness (or invariance) to image degradations is very desirable for texture feature extraction methods in real-world applications. In this paper, focus is on making arbitrary texture descriptors invariant to blur which is often prevalent in real image data. From previous work, we know that most state-of-the-art texture feature extraction methods are unable to cope even with minor blur degradations if the classifier’s training stage is based on idealistic data. However, if the training set suffers similarly from the degradations, the obtained accuracies are significantly higher. Exploiting that knowledge, in this approach the level of blur of each image is increased to a certain threshold, based on the estimation of a blur measure. Experiments with synthetically degraded data show that the method is able to generate a high degree of blur invariance without loosing too much distinctiveness. Finally, we show that our method is not limited to ideal Gaussian blur.
---
paper_title: Pit pattern classification using extended Local Binary Patterns

paper_content:

In this work we present a method for automated classification of endoscopic images according to the pit pattern classification scheme. Images taken during colonoscopy are transformed using a modified version of the Local Binary Patterns operator (LBP). Then, two-dimensional histograms based on the LBP data from different color channels are created. Finally, the classification is carried out by employing the nearest-neighbors (1-NN) classifier in conjunction with the Bhattacharyya distance metric. The experimental results show that the extended LBP operator delivers superior results and an automated classification of endoscopic images based on the pit pattern classification scheme is feasible.
---
paper_title: Towards integrating temporal information in capsule endoscopy image analysis

paper_content:

Analysis of Wireless Capsule Endoscopy (CE) images has become a very active area of research since this novel technology enabled access to previously inaccessible areas of the gastrointestinal tract, especially the small intestine. Art has investigated automatic segmentation of organ boundaries, detection of lesions and bleeding as well as other supervised and unsupervised analysis. However, all of this art has focused on treating the images as individual and independent observations that contribute towards a unique and separate decision. Given the overlap between the images, this is clearly not the case. A human, by contrast, performs assessment by combining the information seen in all neighboring views of the anatomy in a study. This article makes two significant contributions. Towards combining information from multiple images, we propose a supervised classification approach using an HMM framework. Secondly, we use a weak (k-NN) classifier to prototype and evaluate such a framework for regions of the GI tract containing polyps. The combined framework significantly improves the performance of the individual classifier and experiments show promising performance with accuracy > 0.9.
---
paper_title: Intelligent systems for computer-assisted clinical endoscopic image analysis

paper_content:

The importance of computer-assisted diagnosis in endoscopy is to assist the physician in detecting the status of tissues by characterising the features from the endoscopic image. Due to the complex nature of clinical manifestations, employing a single "feature" technique to detect different abnormalities may not necessarily yield accurate results. In this paper schemes have been developed to extract new texture features from the texture spectra in the chromatic and achromatic domains for a selected region of interest from each colour component histogram of endoscopic images. The implementation of an advanced neural network scheme and the concept of fusion of multiple classifiers dedicated to specific feature parameters have been also adopted in this paper. The endoscopic images used in this study have been obtained using the new M2ATM Swallowable Imaging Capsule - a patented, video colour-imaging disposable capsule. The detection accuracy of the proposed system has reached to 100%, providing thus an indication that such intelligent schemes could be used as a supplementary diagnostic tool in endoscopy.
---
paper_title: Pit Pattern Classification of Zoom-Endoscopical Colon Images Using DCT and FFT

paper_content:

This work presents a classification approach for images taken from magnifying colonoscopy. Classification is done according to the pit pattern scheme. Images are not classified directly in the proposed classifier. Instead, they are transformed to a frequency domain using discrete cosine or Fourier transformation. Feature selection is optimized using a genetic algorithm, the actual classification is done using standard methods from statistical pattern recognition (a Bayes normal classifier).
---

Title: Survey of Computer Vision and Machine Learning in Gastrointestinal Endoscopy

Section 1: Endoscopic Applications
Description 1: Discuss the various clinical applications of computer vision and machine learning in gastrointestinal endoscopy, classified into different categories (e.g., Polyp Detection and Classification, Ulcer Detection).

Section 2: Polyp Detection and Classification (PD)
Description 2: Examine methods and techniques used for detecting and classifying polyps in colonoscopy, including various feature descriptors and classifiers.

Section 3: Ulcer Detection (UD)
Description 3: Describe approaches for detecting ulcers caused by conditions like GORD and inflammatory bowel diseases, using image decomposition and texture features.

Section 4: Motility Detection (MD)
Description 4: Discuss methodologies for detecting motility in the GI tract, including feature extraction and classification based on contraction sequences and lumen analysis.

Section 5: Endoscopic Abnormality Detection and Classification (ABD)
Description 5: Cover techniques for detecting and classifying general abnormalities in endoscopic images that don't fit into other specific categories.

Section 6: Endoscopic Navigation (NAV) and 6-DOF Localization (LOC)
Description 6: Explore methods for endoscopic navigation and localization using image information to determine current pose or anatomical location within the GI tract.

Section 7: Intra and Inter-Operative Re-localization (IAO and IRO)
Description 7: Discuss approaches for tracking and localizing biopsy sites during and between interventions, including feature matching and SLAM.

Section 8: Lumen Detection (LD)
Description 8: Describe techniques aimed at identifying the lumen within endoscopic images, essential for navigation and motility detection.

Section 9: Uninformative Frame or Region Detection (UI)
Description 9: Look into methods used to detect frames or regions within endoscopic videos that lack informative content.

Section 10: Endoscopic Reconstruction (REC)
Description 10: Discuss the challenges and methods of 3D reconstruction in flexible endoscopic procedures, which can aid in diagnosis and surgical planning.

Section 11: Endoscopic Image Enhancement (IE)
Description 11: Cover preprocessing steps aimed at enhancing the quality of endoscopic images for better visibility and feature extraction.

Section 12: Endoscopic Video Summarization (ES)
Description 12: Explore techniques for summarizing lengthy endoscopic videos, particularly from wireless capsule endoscopy, to assist in faster diagnosis.

Section 13: Clinical Decision Support (CDS)
Description 13: Discuss systems designed to support clinicians in decision-making by detecting and categorizing lesions based on endoscopic images.