---
abstract: 'Automated detection of cervical cancer cells or cell clumps has the potential to significantly reduce error and increase productivity in cervical cancer screening. However, most traditional methods rely on the success of accurate cell segmentation and discriminative hand-crafted features extraction. Recently there are emerging deep learning-based methods which train convolutional neural networks (CNN) to classify image patches, but they are computationally expensive. In this paper we propose to exploit contemporary object detection methods for cervical cancer detection. To deal with the limited size of training samples, we develop the comparison classifier into the state-of-the-art two-stage object detection method based on the comparison with the reference images of each category. In addition, we propose to learn the reference images of the background from the data instead of manually choosing them by some heuristic rules. This architecture, called the Comparison detector, shows significant improvement for small size dataset, achieving a mean Average Precision (mAP) 26.3% and an Average Recall (AR) 35.7%, both improving about **20** points compared to baseline model. Moreover, Comparison detector achieves same mAP performance as the current state-of-the-art model when training on the medium size dataset, and improves AR by **4** points. Our method is promising for the development of automation-assisted cervical cancer screening systems.'
author:
- 'Yixiong Liang, Zhihong Tang, Meng Yan, Jialin Chen, and Yao Xiang [^1] [^2] [^3]'
bibliography:
- 'IEEEabrv.bib'
- 'paper.bib'
title: 'Comparison Detector: Convolutional Neural Networks for Cervical Cell Detection'
---

Cervical cancer screening, object detection, convolutional neural networks, prototype-based classification

Introduction
============

Cervical cytology is the most common and effective screening method for cervical cancer and premalignant cervical lesions [@davey2006effect], which is performed by a visual examination of cytopathological analysis under the microscope of the collected cells that have been smeared on a glass slide and stained and finally giving a diagnosis report according to the descriptive diagnosis method of the Bethesda system (TBS)[@nayar2015bethesda]. Currently in developed countries, it has been widely used and has significantly reduced the number of deaths caused by related diseases, but it is still unavailable for population-wide screening in the developing countries [@saslow2012american], partly due to the fact that it is labor-intensive, time-consuming and expensive [@bengtsson2014screening]. In addition, it is subjective and therefore has motivated lots of automated methods for the automation of cervical screening based on the image analysis techniques.

Over the past 30 years extensive research has attempted to develop automation-assisted screening methods [@birdsong1996automated; @chankong2014automatic; @zhang2014automation; @phoulady2016automatic]. Most of them try to classify a single cell into various stages of carcinoma, which often consists three steps: cell (cytoplasm and nuclei) segmentation, feature extraction and classification. The performance of these methods, however, heavily depends on the accuracy of the segmentation and the effectiveness of the hand-crafted features. With the overwhelming success in a broad range of applications such as image classification [@krizhevsky2012imagenet; @he2016deep], semantic segmentation [@long2015fully], object detection [@ren2015faster; @lin2017feature] and medical imaging analysis [@litjens2017survey; @gulshan2016development; @esteva2017dermatologist], CNN has also been applied to the segmentation and classification of cervical cell [@tareef2017optimizing; @song2017accurate; @song2015accurate; @zhang2017combining; @lu2017evaluation; @zhang2017deeppap; @jith2018deepcerv; @gautam2018considerations]. The majority of them [@tareef2017optimizing; @song2015accurate; @zhang2017combining] are trying to take advantage of CNN to improve the segmentation accuracy of cytoplasm and nuclei, but they do not provide the needed segmentation accuracy [@lu2017evaluation; @jith2018deepcerv], whereas once the segmentation error are taken into account, the classification accuracy would drop [@zhang2017deeppap]. To avoid the dependence on accurate segmentation, the patch-based methods try to use CNN to classify the image patches [@jith2018deepcerv; @gautam2018considerations]. However, the extraction of such patches still requires the segmentation of nuclei. The recent work [@zhang2017deeppap] also adopt the patch-based strategy but during the inference the random-view aggregation and multiple crop testing are needed to produce the final prediction results and thereby is time-consuming.

In this paper, we propose an efficient and effective strategy to apply CNN for cervical cancer screening, *without* any pre-segmentation step. Specifically, we exploit the contemporary CNN-based object detection methods [@ren2015faster; @lin2017feature] to detect the cervical cytological abnormalities directly. It is straightforward and has been successfully applied for other medical image analysis [@liang2018end; @liang2018object], but we are not aware of any works try to apply CNN-based object detection for automated cervical cytology. We attribute this to the lack of the right cervical cancer microscopic image dataset for the detection task. CNN-based object detection methods often need sufficient annotated data to obtain good generalization, but for cervical cytological abnormalities detection, collecting the large amounts of data with careful and accurate annotation is difficult partially due to the limitation by laws, the scarcity of positive samples and especially the unanimous agreement between cytopathologists [@stoler2001interobserver].

To alleviate the small training dataset size problem, we propose the named *Comparison Detector*, which migrate the idea of *comparison* in one/few-shot learning for image classification [@koch2015siamese; @vinyals2016matching; @snell2017prototypical; @yang2018learning] into CNN-based object detection, for cervical cancer detection. Specifically, we choose the state-of-the-art object detection method, Faster R-CNN [@ren2015faster] with FPN [@lin2017feature], as our baseline model and replace the original parameter classifier with a non-parametric one based on the idea of comparison with the reference images of each category. Furthermore, instead of manually choosing the reference images of the background category by some heuristic rules, we propose to learn them from the data. We also investigate several important factors including generating prototype representations of categories and the design of head model for cervical cell detection. Our algorithm directly operate on the whole image rather than the extracted patches based on the nuclei and hereby only need one forward propagation for each image, making the inference very efficient. In addition, the proposed method is *flexible* to be intergraded into other proposal-based methods.

We collect a small size dataset $D_s$ and a medium size dataset $D_f$ which are directly dedicated to cervical cell/clumps detection, on which we evaluate the performance of the proposed Comparison Detector. When the model is learned from the small size dataset, the performance of our method is significantly better than the one of baseline model, i.e. Comparison Detector has an mAP 26.3% and an AR 35.7% but the baseline model only gains an mAP 6.6% and an AR 12.9%. When the model is learn from the medium size dataset, our Comparison Detector achieves almost the same performance with a mAP of 45.3%, but improves nearly 4 points comparing to baseline model with AR. We summarize our contributions as follows: 1) To the best of our knowledge, this is the first application of the CNN-based object detection methods to cervical cancer detection; 2) We propose Comparison Detector method to deal with the small training sample size problem in cervical cell detection; 3) We propose a strategy to directly learn the background reference images and 4) Our method performs much better than the baseline on both small size and medium size dataset and has the potential applications to the real automation-assisted cervical cancer screening systems.

Related work
------------

**Cervical cell segmentation and classification.** Traditional cytological criteria for classifying cervical cell abnormalities are based on the changes in nucleus to cytoplasm ratio, nuclear size, irregularity of nuclear shape and membrane, therefore there are numerous works focusing on the segmentation of cell or cell components (nuclei, cytoplasm) [@chankong2014automatic; @zhang2014segmentation; @gencctav2012unsupervised; @chen2014semi; @song2015accurate; @song2017accurate; @zhang2017graph; @lu2015improved; @lee2016segmentation; @li2012cytoplasm; @tareef2018multi]. Although significant progress has been achieved recently, the segmentation of cell or cell components remains an open problem due to the large shape and appearance variation between cells, the poor contrast of cytoplasm boundaries and the overlap between cells [@zhang2017deeppap; @lee2016segmentation; @tareef2018multi; @lu2017evaluation].

On the other hand, cervical cell classification methods try to differentiate the dysplastic cells from the norm cells and classify them into various stages of carcinoma. According to TBS rules [@nayar2015bethesda], a large number of hand-crafted features are designed to describe the shape, texture and appearance characteristics of the nucleus and cytoplasm [@gencctav2012unsupervised; @chen2014semi; @marinakis2009pap; @chankong2014automatic; @phoulady2016automatic; @bora2017automated]. The resulting features are often further organized by feature selection or dimensionality reduction and then are fed into various classifiers (e.g. random forests, SVM, softmax regression, neural network, etc.) to perform the final classification. However, as mentioned above, the extraction of those engineered features depends on the accurate segmentation of cell or cell components. Furthermore, it is also limited by the current understanding of cervical cytology [@zhang2017deeppap]. To reduce the dependency on the accurate segmentation, the CNN are used to learn the features from data recently [@jith2018deepcerv; @gautam2018considerations], but an approximate segmentation or (region of interest) ROI detection is still necessary. Although the DeepPap [@zhang2017deeppap] is claimed totally segmentation-free, it still needs the nucleus centroid information for training and the random-view aggregation and multiple crop testing during the inference stage, which are very time-consuming. There are a handful public available microscopic image datasets dedicated to cervical cell segmentation such as ISBI-14[^4], ISBI-15 [^5], but to our best knowledge for cervical cell classification the only public available microscopic image dataset is the Herlev benchmark dataset [@marinakis2009pap], which consists of 917 single cell images corresponding to four categories of abnormal cell with different severity (namely light dysplastic, moderate dysplastic, severe dysplastic and carcinoma in situ) and three categories of normal cells (normal columnar, normal intermediate and normal superficial). The limited annotated data prevents the applications of traditional object detection methods such as Viola-Jones detector [@viola2004robust] or contemporary CNN-based detectors [@liu2018deep] to cervical cancer screening.

**CNN-based object detection.** The Overfeat [@sermanet2013overfeat] made the earliest efforts to apply CNN for object detection and has achieved a significant improvement of more than 50% mAP when compared to the best methods at that time which were based on the hand-crafted features. Since then, a lot of CNN-based methods [@ren2015faster; @liu2016ssd; @girshick2014rich; @girshick2015fast; @he2017mask; @li2017light; @lin2018focal; @singh2018r; @redmon2016you; @redmon2017yolo9000; @zhang2018single; @redmon2018yolov3] have been proposed for high-quality object detection, which can be roughly classified into two categories: object proposal-based and proposal-free. The road-map of proposal-based methods starts from the notable R-CNN [@girshick2014rich] and is improved by Fast-RCNN [@girshick2015fast] in an end-to-end manner and by Faster R-CNN [@ren2015faster] to quickly generate object regions, which has motivated a lot of follow-up improvements [@lin2017feature; @he2017mask; @li2017light; @lin2018focal; @singh2018r] in terms of accuracy and speed. The proposal-free methods [@liu2016ssd; @redmon2016you; @redmon2017yolo9000; @redmon2018yolov3] directly predict the bounding boxes without the proposal generation step. Generally, the proposal-free methods are conceptually simpler and much faster than the proposal-based methods, but the detection accuracy is usually behind that of the proposal-based methods [@zhang2018single]. Here we choose the Faster R-CNN [@ren2015faster] with FPN [@lin2017feature] as our baseline model but our method is compatible with other proposal-based methods.

![image](figure1.pdf)

**One/few-shot learning.** One/few-shot learning is a task of learning from just one or a few training samples per class and has been extensively discussed in the context of image recognition and classification [@koch2015siamese; @vinyals2016matching; @snell2017prototypical]. Recently significant progress has been made for one/few-shot learning tackled by meta-learning or learning-to-learn strategy, which can be roughly divided into three categories: metric-based, memory-based and optimization-based. The metric-based methods [@koch2015siamese; @vinyals2016matching; @snell2017prototypical; @yang2018learning] learn to compare the query image with support set images. The memory-based methods [@santoro2016one] exploit the memory-augmented neural network to quickly store and retrieve sufficient information for each classification task, while the optimization-based methods [@ravi2016optimization; @finn2017model] aim to learning a base-model which can be fine-tuned quickly for a new classification task. All these works only tackle image classification tasks.

**Object detection with limited-data.** Most prior works on object detection with limited labels use semi-/weakly-supervised methods or few-example learning [@dong2018few] to make use of abundant unlabeled data, whereas in limited-data regime there are few work focus on using few-shot learning to address this problem [@schwartz2018repmet; @kang2018few]. Kang et al. [@kang2018few] decompose the training into base-model learning and meta-model learning and train a meta-model to reweight the features extracted by the base-model to assist novel object detection. However, the training of base model still needs abundant annotated data for base classes. RepMet [@schwartz2018repmet] introduces a metric learning-based sub-network architecture to learn the embedding space and distribution of the training categories without using external data. However, RepMat involves an alternating optimization between the external class distribution module learning and net parameters updating, whereas our solution is a clean, single-step training framework.

Comparison Detector {#section3}
===================

Basic Architecture
------------------

Our proposed comparison detector is based on proposal-based detection framework consisting of a region proposal network (RPN) for proposal generating, a backbone network for feature extraction and a head for the proposal classification and bounding box regression. Here we choose the Faster R-CNN with FPN [@lin2017feature] as our baseline. Then we decouple the regression and classification in the head and replace the original parameter classifier with our comparison classifier. Our no-parameters classifier introduces a inductive bias, namely the within-class distance is less than the between-class in the embedding space, into the model and henceforth mitigates the small sample size issue to some extent [@battaglia2018relational].

The framework of the proposed Comparison detector is depicted in Fig.\[fig:1\], which is divided into three stages to describe. At the first stage, as shown in Fig. \[fig:1\](a), the features of both the reference and the object images are computed by backbone network with FPN [@lin2017feature], without using any extra models to encode the reference images. The only difference is there are no RPN operation on the reference images. Assuming that there are $n$ samples per category with $t$ levels pyramid feature in the reference images. Let $F_i^l$ be the $i$-th categories’ prototype representation of the $l$-level pyramid features, which can be computed by average operation as follows $$F_i^l = \frac{1}{n} \sum_jF^l (R_{ij}), \label{equ:1}$$ where $F^l(\cdot)$ and $R_{ij}$ denote the $l$-th level feature extraction function and the $j$-th reference image of class $i$, respectively. The second stage is to generate the prototype representations of each category from the reference images’ pyramid features. We need to find a map function $S(\cdot)$ which use all level pyramid features each category as input to compute the final prototype representation $F_i$ for class $i$ $$F_i = S (\{F_i^l\}). \label{equ:3}$$

The third stage is the design of the head model for classification and bounding box regression (Fig. \[fig:1\](b)), consisting of a few convolutional ($Conv$) and fully connected ($FC$) layers. Let $d(P_m, F_i)$ be a metric function to compute the distance between the feature of $m$-th proposal $P_m$ and prototype representation of the $i$-th category $F_i$. It is important to note that $P_m$ and $F_i$ have the same size. Each proposal’s classification $p_i$ and bounding box regression $b_i$ can be obtained by $$p_i = \frac{e^{-d(P_m, F_i)}}{\sum_ke^{-d(P_m, F_k)}}, \label{equ:4}$$ $$b_i = B(P_m, F_i), \label{equ:5}$$ where $B(\cdot,\cdot)$ denotes the box regression function. The rest of the model is the same as Faster R-CNN with FPN model [@lin2017feature].

![ The block for learning prototype representations of background class[]{data-label="fig:2"}](figure2.pdf)

Learning the reference background
---------------------------------

There are many negative proposals generated by RPN, so the R-CNN [@girshick2014rich] adds a background category to represent them. In our Comparison detector, we need to select a number of reference images for each category and therefore we also need to choose reference images for the background category. Due to the overwhelming diversity, selecting background reference is very difficult. Notice that a region is considered to a the proposal indicating that it has certain similarity with categories. Therefore, it can be inferred that its features are a combination of different categories in the most case. So we propose to learn it by combining the prototype representations of all the categories in the reference samples, which can be implemented by a simple $1\times1$ convolution operation, as shown in Fig. \[fig:2\].

Generating prototype representations of categories
--------------------------------------------------

As shown in Eq. \[equ:4\], the Comparison detector uses metric function to measure distance or the dissimilarity between the prototype representation of categories and the features of the proposal, then obtains the label of the proposals based on the dissimilarity. Features of proposal may come from any of the four level pyramids, and the prototype representation of the categories is obtained according to Eq. \[equ:3\]. For simplicity, we directly resize the each feature pyramid which is generated by reference images to a fixed size, and then calculate prototype representation by averaging operation, i.e. $$\label{pr_avg}
S(\{F_i^l\}) = \frac{1}{t}\sum_{l}r(F_i^l, s),$$ where $t$ is the total number of level feature pyramids, $r(\cdot, \cdot)$ is resize function and $s$ is the size of final features. Different levels pyramid features of the category are resized into fixed size, and then getting the prototype representation by simply averaging them.

The head for classification and regression {#section:3.3}
------------------------------------------

As shown in Fig. \[fig:3\](a), the structure of the baseline model’s head is to transform the proposal feature firstly and then one branch is used for classification, and another is used to predict the offset of the bounding box. For our Comparison detector, due to the introduction of the reference images, we need to re-organise the head. The are two choices according to whether the reference images are involved in the box regression branch. One is that the reference prototypes are only used for classification, as shown in Fig. \[fig:1\](b). Unlike the baseline model, the comparison classifier and bounding box regressor in the head of Comparison detector are independent. And the bounding box regressor only uses the features of ROI to predict the offset of the bounding box. It is equivalent to $$\begin{aligned}
   d(P_m, F_i) & = FC(F(P_m, F_i)), \\
   B(P_m, F_i) &  = FC(FC(FC(P_m))),\end{aligned}$$ where $F(P_m, F_i) = Conv_3(Conv_1(|F_i - P_m|^2))$. Another choice is to use the reference prototypes for both classification and regression, as shown in Fig. \[fig:3\](b), which means $$B(P_m, F_i) = FC(F(P_m, F_i)).$$ We call this method as shared module. They all achieve good performance in our experiments but the independent module performs slightly better (see Table \[tab:1\]).

Reference images sampling
-------------------------

In our Comparison detector, we also need to choose the reference images for each category. A intuitive way is to select them according to the Bethesda atlas [@nayar2015bethesda]. However, there are very significant difference between the given atlas and our data due to the variations of the preparation and digitization of slide. Hence we resort to other feasible data-driven alternatives. We randomly select about 150 instances of each category from the training sets. The shortest side of these instances is greater than 16 pixels. Therefore we get a total of 1560 instances and from them, we can select suitable instances in these objects as our reference images.

There are two possible way. The first is to randomly choose several instances of each category as the reference images. The second is to first map all 1560 objects into the feature space through the pre-trained model and get the features of each object and then use t-SNE [@maaten2008visualizing] for feature dimension reduction (Fig. \[fig:4\]). Based on the results of t-SNE, we select the most representative samples in 3D space as our reference images.

Experiment and Result
=====================

Materials and experiments {#sect:4.1}
-------------------------

Since there are no established benchmarks for cervical cell object detection in the community, we first establish a database consisting of 7,086 cervical microscopical images and based on which 48,587 object instance bounding boxes were labeled by experienced pathologists. Conforming to TBS categories [@nayar2015bethesda], we divide these objects into 11 categories, namely ASC-US (ascus), ASC-H (asch), low-grade squamous intraepithelial lesion (lsil), high-grade squamous intraepithelial lesion (hsil), squamous-cell carcinoma (scc), atypical glandular cells (agc), trichomonas (trich), candida (cand), flora, herps, actinomyces (actin). Figure \[fig:5\] shows some examples of each category in our database. Then we divide the dataset into training set $D_f$ which contains 6667 images, test set which contains 419 images for experiment. We randomly choose 762 images from the training dataset to form a small dataset of $D_s$. The number of categories in each dataset is shown in the Fig. \[fig:6\]

![image](figure5.pdf){width=".7\textwidth"}

![image](figure6.eps)

In all experiments, we used ResNet50 as backbone network with ImageNet pre-trained model. For reference images, we re-scale them such that their side is $w=h=224$ which is coincident with pre-trained model. The initial learning rate is 0.001, and then decreased by a factor of 10 at 35-th and 50-th epoch. Training is stopped after 60 epochs and the other parameters are the same as FPN [@lin2017feature]. The experiment is firstly trained on the $D_f$ to evaluate the performance on sufficient data. In our setting, the reference images are fixed in each training iteration for the stability of the training model. And test stage is the same.

As for the cervical cell images, annotators are prone to take a higher threshold when label the objects due to the low discrimination of them. At the same time, multiple nearby objects with the same category will be marked as one, so the performance of the model can not be well reflected by mAP. Therefore, the performance of the model is evaluated by using mAP and AR as a complement on test set. If the mAP does not decrease and the AR improves, it surely signifies the performance is improved. Herein, the results are reported in both mAP and AR. A summary of results can be found in Table \[tab:1\] and some detection results on the test set are shown in Fig.\[fig:7\]. .

[|c|c|c|c|c|c|c|c|]{}

------------------------------------------------------------------------

**model & ******

  ------------
    learning
   background
  ------------

& **independent mode & ******

  ------------------
      using all
   pyramid features
  ------------------

& **refine box & **balance loss & **mAP & **AR\
********

------------------------------------------------------------------------

`A` &$\surd$ &$\surd$ &$\surd$ &$\surd$ & &34.1 &53.3\

------------------------------------------------------------------------

`B` & &$\surd$ &$\surd$ &$\surd$ & &31.4 &49.3\

------------------------------------------------------------------------

`C` &$\surd$ &$\surd$ & &$\surd$ & &32.7 &50.8\

------------------------------------------------------------------------

`D` &$\surd$ & &$\surd$ &$\surd$ & &41.0 &51.3\

------------------------------------------------------------------------

`E` &$\surd$ & & &$\surd$ & &38.9 &49.8\

------------------------------------------------------------------------

`F` &$\surd$ &$\surd$ &$\surd$ & & &37.7 &51.1\

------------------------------------------------------------------------

`G` &$\surd$ &$\surd$ &$\surd$ & &$\surd$ &38.8 &52.3\

------------------------------------------------------------------------

`H` &$\surd$ & &$\surd$ &$\surd$ &$\surd$ &43.5 &58.9\

------------------------------------------------------------------------

`I` &$\surd$ &$\surd$ &$\surd$ &$\surd$ &$\surd$ &**43.7** &**60.7**\

  ----------------------------------------------------------------------------------- -- -- --
   **comparator & **$\ell_2$-distance &**parameterized $\ell_2$-distance & **concat\        
                     mAP &34.1(43.7)& 38.2(**44.5**) & 40.7(42.5)\                          
                     AR &53.3(60.7)& 56.8(**61.6**) & 49.1(58.1)\                           
                                       ********                                             
  ----------------------------------------------------------------------------------- -- -- --

Reference background
--------------------

We first evaluate our scheme to learn the background reference. Experiment shows our method is feasible (See model `A` in Table \[tab:1\]). It should be noted that because the prototype representation of the background category is learned from the prototype representation of other categories, the gradient propagation will also have some effect on the optimization of other prototype representation. In order to make sure whether this effect is beneficial, we stop gradient propagation at the fork position in Fig. \[fig:2\]. The performance of the model has declined with an mAP of 33.0% and a AR of 52.6%. In order to compare the effect of learning background, we randomly select some background samples from the proposals to obtain the features of background. The result is model `B` in Table \[tab:1\]. Besides generating the prototype of background, we try to remove it but the training fails to converge.

Prototype representations of categories
---------------------------------------

In our approach, as shown in Eq. \[pr\_avg\], we use all pyramid features to generate prototype representation of categories. Another choice is to only use the last level pyramid feature as the category of prototype, i.e. $S (\{F_i^l\}) = F^5_i$. As shown the results of model `A` comparing to model `C` and model `D` comparing to model `E` in Table \[tab:1\], using all pyramid features performs much better. Because it can combine features from multiple level pyramids, which not only have rich semantics but also take into account objects of different size. We also fuse different levels of pyramid features by using LSTM [@hochreiter1997long], but the speed is greatly reduced.

Head model {#section:4.3}
----------

As mentioned before, in independent module, the box regression function $B(\cdot,\cdot)$ is the same as baseline model because experiment found that removing one layer will make the result worse. The results show shared module (model `D`) performs much better than independent module (model `A`). Furthermore, we drop the operation of refining bounding box from the head. As shown in Table \[tab:1\], it’s weird that model `F` is better than model `A` which goes against common sense that fine-tuning bounding box twice is often better than just once. We conjecture that the importance of classification should be more important than bounding box regression in our model [@liang2018object]. So we add a weight coefficient $\lambda$ to balance the classification loss and bounding box regression loss. Here we select $\lambda=5$. The results of model `G`, `H` and `I` in Table \[tab:1\] confirm our analysis. By analyzing model `H` and model `I`, we find that the difference between them is not only classification and bbox regression is independent, but also the comparison classifier of model `H` is semi-parameter. After changing the comparison classifier of model `I` into semi-parameter(Fig. \[fig:1\] (b)), the results show that it is better than model `H`.

  ---------------------------------------------------- -- -- --
   **method & **fixed mode & **random mode & **t-SNE\        
               mAP &44.5 &42.8&**45.3**\                     
               AR & 61.6 &61.0& **62.8**\                    
                        ********                             
  ---------------------------------------------------- -- -- --

  : Different way of selecting the reference images.[]{data-label="tab:3"}

![image](figure7a.eps){height="3.2cm" width="4.3cm"}

![image](figure7b.eps){height="3.2cm" width="4.3cm"}

![image](figure7c.eps){height="3.2cm" width="4.3cm"}

![image](figure7d.eps){height="3.2cm" width="4.3cm"}

![image](figure7e.eps){height="3.2cm" width="4.3cm"}

![image](figure7f.eps){height="3.2cm" width="4.3cm"}

![image](figure7g.eps){height="3.2cm" width="4.3cm"}

![image](figure7h.eps){height="3.2cm" width="4.3cm"}

![image](figure7i.eps){height="3.2cm" width="4.3cm"}

![image](figure7j.eps){height="3.2cm" width="4.3cm"}

![image](figure7k.eps){height="3.2cm" width="4.3cm"}

![image](figure7l.eps){height="3.2cm" width="4.3cm"}

[|c|c|c|c|c|c|c|c|c|c|c|c|c|c|c|]{} **method & **dataset & **AR & **mAP & **ascu &**asch &**lsil &**hsil &**scc & **agc & **trich & **cand & **flora & **herps&**actin\
******************************

  ----------
   baseline
    model
  ----------

& $D_s$ &12.9&6.6&11.0&2.0&23.7&21.6&0.0&3.5&0.0&11.5&0.0&0.0&0.0\

  ------------
   Comparison
    detector
  ------------

&$D_s$ &**35.7**&**26.3**&10.5&1.7&42.8&32.3&0.8&40.5&37.5&24.1&6.9&45.0&46.6\

  ----------
   baseline
    model
  ----------

&$D_f$ &58.9&45.2&27.2&6.7&41.7&35.3&18.6&57.3&46.7&72.2&57.3&83.0&51.4\

  ------------
   Comparison
    detector
  ------------

&$D_f$ &**62.8**&**45.3**&29.1&7.8&43.0&37.8&19.3&56.2&50.4&62.2&59.4&64.4&68.3\

Optimizing comparison classifier
--------------------------------

We evaluate three distance metrics in the comparison classifier. The first is $\ell_2$-distance which means $d(P_m, F_i) = M(|F_i-P_m |^2) $. $M(\cdot)$ represents averaging function for tensor. The second is the parameterized $\ell_2$-distance, such as $d(P_m, F_i) = Conv_7(|F_i - P_m|^2)$. Similar to [@yang2018learning], we also try to make the model to learn the metric function instead of the predefined ones. According to the result of Table \[tab:2\], our model ultimately adopts parameterized $\ell_2$-distance. When $\lambda =5$, the result is shown in brackets. Combining with the results shown in Table \[tab:1\], it is universal that the balance trick can improve performance in our model. So we adopt this trick in all the next experiments.

Reference images sampling
-------------------------

We first evaluate the scheme of randomly choosing reference image which includes two tests. The first is to randomly choose 3 instances of each category (this number is limited by GPU’s memory) as the reference images (`fixed mode`). The second one is to randomly select 5 candidates of each category in those objects. Then the model randomly selected three of the five candidates as templates during training, but five in testing (`random mode`). The results are listed in Table \[tab:3\], which shows that compared with `random mode`, `fixed mode` performs better. Therefore when evaluate the scheme of choosing reference image by applying t-SNE, we also adopt the `fixed mode`. During the training of t-SNE, we adopt the following parameters setting, i.e. the hyper-parameters are 30 for perplexity, 1 for learning rate, and 10 for label supervision. Table \[tab:3\] shows that the selection reference image via t-SNE performs the best.

Performance on training dataset $D_f$ and $D_s$
-----------------------------------------------

As shown in Table \[tab:4\], Comparison detector has the almost same mAP as the baseline model when training on the $D_f$ dataset, but improves the AR by near 4 points. Due to the special annotating situation as described in Section \[sect:4.1\], some correct predictions may be identified as false positives. Therefore, there is an significant increase in AR, but little improvement in mAP. When training on the $D_s$, Comparison detector is completely superior to baseline model. It achieves a top result on the test set with a mAP of 26.3% compared to 6.6%, which indicates our method alleviates the over fitting problem to some extent. Prototype representation in this model is generated by reference images, however, it can be generated by other way, such as external memory. In the future work, We expect a better solution for the generation of prototype representations.

Conclusion
==========

In this work, we propose to apply contemporary CNN-based object detection methods for automated cervical cancer detection. To deal with the limited size of training samples, we develop the comparison classifier into the state-of-the-art two-stage object detection method based on the comparison with the reference images of each category. Instead of manually choosing the reference images of the background by some heuristic rules, we present a scheme to learn them form the data directly. We also investigate several important ingredients including the generation of prototype representations of each class and the design of head model for cervical cell detection. Experimental results show that compared with the baseline, our method improves the mAP by **19.7** points and the AR by **22.8** when trained on the small size training data, and achieves almost the same mAP but improves the AR by **3.9** when trained on the medium size training data. It should be noticed that our algorithm directly operate on the whole image rather than the extracted patches based on the nuclei and hereby only need one forward propagation for each image, making the inference very efficient. In addition, the proposed method is *flexible* to be intergraded into other proposal-based methods.

[^1]: This research was partially supported by the National Natural Science Foundation of China under Grant No. 61602522, the Natural Science Foundation of Hunan Province, China under Grant No.14JJ2008 and the Fundamental Research Funds of the Central Universities of Central South University under Grant No. 2018zzts577. (*Corresponding author: Yao Xiang*.)

[^2]: Y. Liang, Z. Tang, M. Yan, J. Chen and Y. Xiang are with the School of Computer Science and Engineering, Central South University, Hunan 410083, China. E-mail: {yxliang,yao.xiang}@csu.edu.cn.

[^3]: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.

[^4]: <https://cs.adelaide.edu.au/~carneiro/isbi14_challenge/index.html>

[^5]: <https://cs.adelaide.edu.au/~zhi/isbi15_challenge/index.html>
