Unsupervised Distance-Based OutlierDetection Using Nearest NeighboursAlgorithm on Distributed Approach: Survey
---
paper_title: Ranking Outliers Using Symmetric Neighborhood Relationship

paper_content:

Mining outliers in database is to find exceptional objects that deviate from the rest of the data set. Besides classical outlier analysis algorithms, recent studies have focused on mining local outliers, i.e., the outliers that have density distribution significantly different from their neighborhood. The estimation of density distribution at the location of an object has so far been based on the density distribution of its k-nearest neighbors [2,11]. However, when outliers are in the location where the density distributions in the neighborhood are significantly different, for example, in the case of objects from a sparse cluster close to a denser cluster, this may result in wrong estimation. To avoid this problem, here we propose a simple but effective measure on local outliers based on a symmetric neighborhood relationship. The proposed measure considers both neighbors and reverse neighbors of an object when estimating its density distribution. As a result, outliers so discovered are more meaningful. To compute such local outliers efficiently, several mining algorithms are developed that detects top-n outliers based on our definition. A comprehensive performance evaluation and analysis shows that our methods are not only efficient in the computation but also more effective in ranking outliers.
---
paper_title: A Data Stream Outlier Delection Algorithm Based on Reverse K Nearest Neighbors

paper_content:

A new data stream outlier detection algorithm SODRNN is proposed based on reverse nearest neighbors. We deal with the sliding window model, where outlier queries are performed in order to detect anomalies in the current window. The update of insertion or deletion only needs one scan of the current window, which improves efficiency. The capability of queries at arbitrary time on the whole current window is achieved by Query Manager procedure, which can capture the phenomenon of concept drift of data stream in time. Results of experiments conducted on both synthetic and real data sets show that SODRNN algorithm is both effective and efficient.
---
paper_title: Interpreting and Unifying Outlier Scores

paper_content:

Outlier scores provided by different outlier models differ widely in their meaning, range, and contrast between different outlier models and, hence, are not easily comparable or interpretable. We propose a unification of outlier scores provided by various outlier models and a translation of the arbitrary “outlier factors” to values in the range [0, 1] interpretable as values describing the probability of a data object of being an outlier. As an application, we show that this unification facilitates enhanced ensembles for outlier detection.
---
paper_title: Outlier detection score based on ordered distance difference

paper_content:

Outlier Detection is one of the most important topics in data mining and knowledge discovery in databases. It is to find a methodology to detect instances in a dataset that do not conform to the rest of the dataset. Local Outlier Factor is one of the earlier outlier detection score. In this paper, we propose a new approach for parameter-free outlier detection algorithm to compute Ordered Distance Difference Outlier Factor. We formulate a new outlier score for each instance by considering the difference of ordered distances. Then, we use this value to compute an outlier score. We use a score of each instance to provide a degree of outlier and compare it with LOF. Our algorithm can produce OOF in Θ (n2) without parameter.
---
paper_title: A New Local Distance-Based Outlier Detection Approach for Scattered Real-World Data

paper_content:

Detecting outliers which are grossly different from or inconsistent with the remaining dataset is a major challenge in real-world KDD applications. Existing outlier detection methods are ineffective on scattered real-world datasets due to implicit data patterns and parameter setting issues. We define a novel Local Distance-based Outlier Factor (LDOF) to measure the outlier-ness of objects in scattered datasets which addresses these issues. LDOF uses the relative location of an object to its neighbours to determine the degree to which the object deviates from its neighbourhood. We present theoretical bounds on LDOF's false-detection probability. Experimentally, LDOF compares favorably to classical KNN and LOF based outlier detection. In particular it is less sensitive to parameter values.
---
paper_title: Statistical selection of relevant subspace projections for outlier ranking

paper_content:

Outlier mining is an important data analysis task to distinguish exceptional outliers from regular objects. For outlier mining in the full data space, there are well established methods which are successful in measuring the degree of deviation for outlier ranking. However, in recent applications traditional outlier mining approaches miss outliers as they are hidden in subspace projections. Especially, outlier ranking approaches measuring deviation on all available attributes miss outliers deviating from their local neighborhood only in subsets of the attributes. In this work, we propose a novel outlier ranking based on the objects deviation in a statistically selected set of relevant subspace projections. This ensures to find objects deviating in multiple relevant subspaces, while it excludes irrelevant projections showing no clear contrast between outliers and the residual objects. Thus, we tackle the general challenges of detecting outliers hidden in subspaces of the data. We provide a selection of subspaces with high contrast and propose a novel ranking based on an adaptive degree of deviation in arbitrary subspaces. In thorough experiments on real and synthetic data we show that our approach outperforms competing outlier ranking approaches by detecting outliers in arbitrary subspace projections.
---
paper_title: LOF: identifying density-based local outliers

paper_content:

For many KDD applications, such as detecting criminal activities in E-commerce, finding the rare instances or the outliers, can be more interesting than finding the common patterns. Existing work in outlier detection regards being an outlier as a binary property. In this paper, we contend that for many scenarios, it is more meaningful to assign to each object a degree of being an outlier. This degree is called the local outlier factor (LOF) of an object. It is local in that the degree depends on how isolated the object is with respect to the surrounding neighborhood. We give a detailed formal analysis showing that LOF enjoys many desirable properties. Using real-world datasets, we demonstrate that LOF can be used to find outliers which appear to be meaningful, but can otherwise not be identified with existing approaches. Finally, a careful performance evaluation of our algorithm confirms we show that our approach of finding local outliers can be practical.
---
paper_title: Ranking Outliers Using Symmetric Neighborhood Relationship

paper_content:

Mining outliers in database is to find exceptional objects that deviate from the rest of the data set. Besides classical outlier analysis algorithms, recent studies have focused on mining local outliers, i.e., the outliers that have density distribution significantly different from their neighborhood. The estimation of density distribution at the location of an object has so far been based on the density distribution of its k-nearest neighbors [2,11]. However, when outliers are in the location where the density distributions in the neighborhood are significantly different, for example, in the case of objects from a sparse cluster close to a denser cluster, this may result in wrong estimation. To avoid this problem, here we propose a simple but effective measure on local outliers based on a symmetric neighborhood relationship. The proposed measure considers both neighbors and reverse neighbors of an object when estimating its density distribution. As a result, outliers so discovered are more meaningful. To compute such local outliers efficiently, several mining algorithms are developed that detects top-n outliers based on our definition. A comprehensive performance evaluation and analysis shows that our methods are not only efficient in the computation but also more effective in ranking outliers.
---

Title: Unsupervised Distance-Based Outlier Detection Using Nearest Neighbors Algorithm on Distributed Approach: Survey

Section 1: INTRODUCTION
Description 1: Introduce the concepts of outlier detection, definitions, reasons for outliers, and various anomaly detection modes.

Section 2: RELATED WORK
Description 2: Summarize the existing approaches and techniques for outlier detection, including various algorithms and their characteristics.

Section 3: Local outlier factor (LOF)
Description 3: Describe the Local Outlier Factor approach, how it works, and its application in identifying outliers.

Section 4: Local distance based outlier factor (LDOF)
Description 4: Discuss the Local Distance-Based Outlier Factor method and its mechanism for measuring outlierness in scattered datasets.

Section 5: Influenced Outlierness (INFLO)
Description 5: Explain the Influenced Outlierness method, including how it works and the scenarios where it is useful.

Section 6: Disadvantages
Description 6: Outline the major disadvantages of the existing methods, such as false positive/negative rates and computational inefficiencies.

Section 7: Description of the Proposed System
Description 7: Detail the proposed system architecture, including data collection, preprocessing, partitioning, outlier detection, and performance evaluation.

Section 8: Data collection and data preprocessing
Description 8: Explain the process of data collection from standard datasets and the preprocessing steps applied to the raw data.

Section 9: Data partitioning
Description 9: Describe how preprocessed data is partitioned among various nodes and the role of the supervisor node.

Section 10: Outlier detection
Description 10: Discuss the outlier detection techniques proposed, including KNN Algorithm with ABOD and INFLO methods in a distributed environment.

Section 11: Performance Evaluation and Result Visualization
Description 11: Provide an overview of the performance evaluation metrics and methods used, along with the result visualization techniques.

Section 12: EXPERIMENTAL SETUP AND EVALUATION
Description 12: Describe the experimental setup, including datasets used, system specifications, and the implementation process.

Section 13: CONCLUSION
Description 13: Summarize the findings, improvements proposed, and suggest future directions for research and implementation.