On Integrating Information Visualization Techniques into Data Mining: A Review
---
paper_title: Data Mining: Practical Machine Learning Tools and Techniques

paper_content:

Data Mining: Practical Machine Learning Tools and Techniques offers a thorough grounding in machine learning concepts as well as practical advice on applying machine learning tools and techniques in real-world data mining situations. This highly anticipated third edition of the most acclaimed work on data mining and machine learning will teach you everything you need to know about preparing inputs, interpreting outputs, evaluating results, and the algorithmic methods at the heart of successful data mining. Thorough updates reflect the technical changes and modernizations that have taken place in the field since the last edition, including new material on Data Transformations, Ensemble Learning, Massive Data Sets, Multi-instance Learning, plus a new version of the popular Weka machine learning software developed by the authors. Witten, Frank, and Hall include both tried-and-true techniques of today as well as methods at the leading edge of contemporary research. *Provides a thorough grounding in machine learning concepts as well as practical advice on applying the tools and techniques to your data mining projects *Offers concrete tips and techniques for performance improvement that work by transforming the input or output in machine learning methods *Includes downloadable Weka software toolkit, a collection of machine learning algorithms for data mining tasks-in an updated, interactive interface. Algorithms in toolkit cover: data pre-processing, classification, regression, clustering, association rules, visualization
---
paper_title: A Preliminary Discourse On The Study Of Natural Philosophy

paper_content:

Thank you very much for reading a preliminary discourse on the study of natural philosophy. Maybe you have knowledge that, people have look numerous times for their chosen books like this a preliminary discourse on the study of natural philosophy, but end up in harmful downloads. Rather than reading a good book with a cup of coffee in the afternoon, instead they cope with some infectious bugs inside their laptop.
---
paper_title: The Bagplot: A Bivariate Boxplot

paper_content:

Abstract We propose the bagplot, a bivariate generalization of the univariate boxplot. The key notion is the half space location depth of a point relative to a bivariate dataset, which extends the univariate concept of rank. The “depth median” is the deepest location, and it is surrounded by a “bag” containing the n/2 observations with largest depth. Magnifying the bag by a factor 3 yields the “fence” (which is not plotted). Observations between the bag and the fence are marked by a light gray loop, whereas observations outside the fence are flagged as outliers. The bagplot visualizes the location, spread, correlation, skewness, and tails of the data. It is equivariant for linear transformations, and not limited to elliptical distributions. Software for drawing the bagplot is made available for the S-Plus and MATLAB environments. The bagplot is illustrated on several datasets—for example, in a scatterplot matrix of multivariate data.
---
paper_title: Variations of Box Plots

paper_content:

Abstract Box plots display batches of data. Five values from a set of data are conventionally used; the extremes, the upper and lower hinges (quartiles), and the median. Such plots are becoming a widely used tool in exploratory data analysis and in preparing visual summaries for statisticians and nonstatisticians alike. Three variants of the basic display, devised by the authors, are described. The first visually incorporates a measure of group size; the second incorporates an indication of rough significance of differences between medians; the third combines the features of the first two. These techniques are displayed by examples.
---
paper_title: A colour scheme for the display of astronomical intensity images

paper_content:

I describe a colour scheme that is appropriate for the screen display of intensity images. This -- unlike many currently available schemes -- is designed to be monotonically increasing in terms of its perceived brightness. Also, when printed on a black and white postscript printer, the scheme results in a greyscale with monotonically increasing brightness. This scheme has recently been incorporated into the radio astronomical analysis packages CASA and AIPS.
---
paper_title: ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps

paper_content:

AbstractChoosing effective colour schemes for thematic maps is surprisingly difficult. ColorBrewer is an online tool designed to take some of the guesswork out of this process by helping users select appropriate colour schemes for their specific mapping needs by considering: the number of data classes; the nature of their data (matched with sequential, diverging and qualitative schemes); and the end-use environment for the map (e.g., CRT, LCD, printed, projected, photocopied). ColorBrewer contains 'learn more' tutorials to help guide users, prompts them to test-drive colour schemes as both map and legend, and provides output in five colour specification systems.
---
paper_title: Interactive volume rendering using multi-dimensional transfer functions and direct manipulation widgets

paper_content:

Most direct volume renderings produced today employ one-dimensional transfer functions, which assign color and opacity to the volume based solely on the single scalar quantity which comprises the dataset. Though they have not received widespread attention, multi-dimensional transfer functions are a very effective way to extract specific material boundaries and convey subtle surface properties. However, identifying good transfer functions is difficult enough in one dimension, let alone two or three dimensions. This paper demonstrates an important class of three-dimensional transfer functions for scalar data (based on data value, gradient magnitude, and a second directional derivative), and describes a set of direct manipulation widgets which make specifying such transfer functions intuitive and convenient. We also describe how to use modern graphics hardware to interactively render with multi-dimensional transfer functions. The transfer functions, widgets, and hardware combine to form a powerful system for interactive volume exploration.
---
paper_title: Multivariate Data Glyphs: Principles and Practice

paper_content:

In the context of data visualization, a glyph is a visual representation of a piece of data where the attributes of a graphical entity are dictated by one or more attributes of a data record. For example, the width and height of a box could be determined by a student’s score on the midterm and final exam for a course, while the box’s color might indicate the genderof the student.Thedefinitionabove is ratherbroad, as it can cover such visual elements as the markers in a scatterplot, the bars of a histogram, or even an entire line plot. However, a narrower definition would not be sufficient to capture the wide range of data visualization techniques that have been developed over the centuries that are termed glyphs.
---
paper_title: The Use of Faces to Represent Points in k-Dimensional Space Graphically

paper_content:

Abstract A novel method of representing multivariate data is presented. Each point in k-dimensional space, k≤18, is represented by a cartoon of a face whose features, such as length of nose and curvature of mouth, correspond to components of the point. Thus every multivariate observation is visualized as a computer-drawn face. This presentation makes it easy for the human mind to grasp many of the essential regularities and irregularities present in the data. Other graphical representations are described briefly.
---
paper_title: Superquadric tensor glyphs

paper_content:

Tensor field visualization is a challenging task due in part to the multi-variate nature of individual tensor samples. Glyphs convey tensor variables by mapping the tensor eigenvectors and eigenvalues to the orientation and shape of a geometric primitive, such as a cuboid or ellipsoid. Though widespread, cuboids and ellipsoids have problems of asymmetry and visual ambiguity. Cuboids can display misleading orientation for tensors with underlying rotational symmetry. Ellipsoids differing in shape can be confused, from certain viewpoints, because of similarities in profile and shading. This paper addresses the problems of asymmetry and ambiguity with a new tunable continuum of glyphs based on superquadric surfaces. Superquadric tensor glyphs enjoy the necessary symmetry properties of ellipsoids, while also imitating cuboids and cylinders to better convey shape and orientation, where appropriate. The new glyphs are demonstrated on fields of diffusion tensors from the human brain.
---
paper_title: Glyph-Based Visualization: Foundations, Design Guidelines, Techniques, and Applications

paper_content:

This state of the art report focuses on glyph-based visualization, a common form of visual design where a data set is depicted by a collection of visual objects referred to as glyphs. Its major strength is that patterns of multivariate data involving more than two attribute dimensions can often be more readily perceived in the context of a spatial relationship, whereas many techniques for spatial data such as direct volume rendering find difficult to depict with multivariate or multi-field data, and many techniques for non-spatial data such as parallel coordinates are less able to convey spatial relationships encoded in the data. This report fills several major gaps in the literature, drawing the link between the fundamental concepts in semiotics and the broad spectrum of glyph-based visualization, reviewing existing design guidelines and implementation techniques, and surveying the use of glyph-based visualization in many applications.
---
paper_title: Enhancing scatterplot matrices for data with ordering or spatial attributes

paper_content:

The scatterplot matrix is one of the most common methods used to project multivariate data onto two dimensions for display. While each off-diagonal plot maps a pair of non-identical dimensions, there is no prescribed mapping for the diagonal plots. In this paper, histograms, 1D plots and 2D plots are drawn in the diagonal plots of the scatterplots matrix. In 1D plots, the data are assumed to have order, and they are projected in this order. In 2D plots, the data are assumed to have spatial information, and they are projected onto locations based on these spatial attributes using color to represent the dimension value. The plots and the scatterplots are linked together by brushing. Brushing on these alternate visualizations will affect the selected data in the regular scatterplots, and vice versa. Users can also navigate to other visualizations, such as parallel coordinates and glyphs, which are also linked with the scatterplot matrix by brushing. Ordering and spatial attributes can also lie used as methods of indexing and organizing data. Users can select an ordering span or a spatial region by interacting with 1D plots or with 2D plots, and then observe the characteristics of the selected data subset. 1D plots and 2D plots provide the ability to explore the ordering and spatial attributes, while other views are for viewing the abstract data. In a sense, we are linking what are traditionally seen as scientific visualization methods with methods from the information visualization and statistical graphics fields. We validate the usefulness of this integration by providing two case studies, time series data analysis and spatial data analysis.
---
paper_title: SpRay: A visual analytics approach for gene expression data

paper_content:

We present a new application, SpRay, designed for the visual exploration of gene expression data. It is based on an extension and adaption of parallel coordinates to support the visual exploration of large and high-dimensional datasets. In particular, we investigate the visual analysis of gene expression data as generated by micro-array experiments; We combine refined visual exploration with statistical methods to a visual analytics approach that proved to be particularly successful in this application domain. We will demonstrate the usefulness on several multidimensional gene expression datasets from different bioinformatics applications.
---
paper_title: Interactive data mining with 3D-parallel-coordinate-trees

paper_content:

Parallel coordinates are an established technique to visualize high-dimensional data, in particular for data mining purposes. A major challenge is the ordering of axes, as any axis can have at most two neighbors when placed in parallel on a 2D plane. By extending this concept to a 3D visualization space we can place several axes next to each other. However, finding a good arrangement often does not necessarily become easier, as still not all axes can be arranged pairwise adjacently to each other. Here, we provide a tool to explore complex data sets using 3D-parallel-coordinate-trees, along with a number of approaches to arrange the axes.
---
paper_title: Interactive hierarchical dimension ordering, spacing and filtering for exploration of high dimensional datasets

paper_content:

Large number of dimensions not only cause clutter in multi-dimensional visualizations, but also make it difficult for users to navigate the data space. Effective dimension management, such as dimension ordering, spacing and filtering, is critical for visual exploration of such datasets. Dimension ordering and spacing explicitly reveal dimension relationships in arrangement-sensitive multidimensional visualization techniques, such as parallel coordinates, star glyphs, and pixel-oriented techniques. They facilitate the visual discovery of patterns within the data. Dimension filtering hides some of the dimensions to reduce clutter while preserving the major information of the dataset. In this paper, we propose an interactive hierarchical dimension ordering, spacing and filtering approach, called DOSFA. DOSFA is based on dimension hierarchies derived from similarities among dimensions. It is scalable multi-resolution approach making dimensional management a tractable task. On the one hand, it automatically generates default settings for dimension ordering, spacing and filtering. On the other hand, it allows users to efficiently control all aspects of this dimension management process via visual interaction tools for dimension hierarchy manipulation. A case study visualizing a dataset containing over 200 dimensions reveals high dimensional visualization techniques.
---
paper_title: Analysis and Interpretation of Visual Hierarchical Heavy Hitters of Binary Relations

paper_content:

The emerging field of visual analytics changes the way we model, gather, and analyze data. Current data analysis approaches suggest to gather as much data as possible and then focus on goal and process oriented data analysis techniques. Visual analytics changes this approach and the methodology to interpret the results becomes the key issue. ::: ::: This paper contributes with a method to interpret visual hierarchical heavy hitters (VHHHs). We show how to analyze data on the general level and how to examine specific areas of the data. We identify five common patterns that build the interpretation alphabet of VHHHs. We demonstrate our method on three different real world datasets and show the effectiveness of our approach.
---
paper_title: Drawing graphs nicely using simulated annealing

paper_content:

The paradigm of simulated annealing is applied to the problem of drawing graphs “nicely.” Our algorithm deals with general undirected graphs with straight-line edges, and employs several simple criteria for the aesthetic quality of the result. The algorithm is flexible, in that the relative weights of the criteria can be changed. For graphs of modest size it produces good results, competitive with those produced by other methods, notably, the “spring method” and its variants.
---
paper_title: Analysis and Visualization of Network Data using JUNG

paper_content:

The JUNG (Java Universal Network/Graph) Framework is a free, open-source software library that provides a common and extendible language for the manipulation, analysis, and visualization of data that can be represented as a graph or network. It is written in the Java programming language, allowing JUNG-based applications to make use of the extensive built-in capabilities of the Java Application Programming Interface (API), as well as those of other existing third-party Java libraries. We describe the design, and some details of the implementation, of the JUNG architecture, and provide illustrative examples of its use.
---
paper_title: Complex Networks: Structure, Robustness and Function

paper_content:

1. Introduction Part I. Random Network Models: 2. The Erdos-Renyi models 3. Observations in real-world networks 4. Models for complex networks 5. Growing network models Part II. Structure and Robustness of Complex Networks: 6. Distances in scale-free networks - the ultra small world 7. Self-similarity in complex networks 8. Distances in geographically embedded networks 9. The network's structure - the generating function method 10. Percolation on complex networks 11. Structure of random directed networks - the bow tie 12. Introducing weights - bandwidth allocation and multimedia broadcasting Part III. Network Function - Dynamics and Applications: 13. Optimization of the network structure 14. Epidemiological models 15. Immunization 16. Thermodynamic models on networks 17. Spectral properties, transport, diffusion and dynamics 18. Searching in networks 19. Biological networks and network motifs Part IV. Appendices References Index.
---
paper_title: Handbook of graph drawing and visualization

paper_content:

Planarity Testing and Embedding Maurizio Patrignani Crossings and Planarization Christoph Buchheim, Markus Chimani, Carsten Gutwenger, Michael Junger, and Petra Mutzel Symmetric Graph Drawing Peter Eades and Seok-Hee Hong Proximity Drawings Giuseppe Liotta Tree Drawing Algorithms Adrian Rusu Planar Straight-Line Drawing Algorithms Luca Vismara Planar Orthogonal and Polyline Drawing Algorithms Christian A. Duncan and Michael T. Goodrich Spine and Radial Drawings Emilio Di Giacomo, Walter Didimo, and Giuseppe Liotta Circular Drawing Algorithms Janet M. Six and Ioannis G. Tollis Rectangular Drawing Algorithms Takao Nishizeki and Md. Saidur Rahman Simultaneous Embedding of Planar Graphs Thomas Blasius, Stephen G. Kobourov, and Ignaz Rutter Force-Directed Drawing Algorithms Stephen G. Kobourov Hierarchical Drawing Algorithms Patrick Healy and Nikola S. Nikolov Three-Dimensional Drawings Vida Dujmovic and Sue Whitesides Labeling Algorithms Konstantinos G. Kakoulis and Ioannis G. Tollis Graph Markup Language (GraphML) Ulrik Brandes, Markus Eiglsperger, Jurgen Lerner, and Christian Pich The Open Graph Drawing Framework (OGDF) Markus Chimani, Carsten Gutwenger, Michael Junger, Gunnar W. Klau, Karsten Klein, and Petra Mutzel GDToolkit Giuseppe Di Battista and Walter Didimo PIGALE Hubert de Fraysseix and Patrice Ossona de Mendez Biological Networks Christian Bachmaier, Ulrik Brandes, and Falk Schreiber Computer Security Olga Ohrimenko, Charalampos Papamanthou, and Bernardo Palazzi Graph Drawing for Data Analytics Stephen G. Eick Graph Drawing and Cartography Alexander Wolff Graph Drawing in Education Stina Bridgeman Computer Networks Giuseppe Di Battista and Massimo Rimondini Social Networks Ulrik Brandes, Linton C. Freeman, and Dorothea Wagner Index
---
paper_title: Vizster: visualizing online social networks

paper_content:

Recent years have witnessed the dramatic popularity of online social networking services, in which millions of members publicly articulate mutual "friendship" relations. Guided by ethnographic research of these online communities, we have designed and implemented a visualization system for playful end-user exploration and navigation of large scale online social networks. Our design builds upon familiar node link network layouts to contribute customized techniques for exploring connectivity in large graph structures, supporting visual search and analysis, and automatically identifying and visualizing community structures. Both public installation and controlled studies of the system provide evidence of the system's usability, capacity for facilitating discovery, and potential for fun and engaged social activity
---
paper_title: Visual Discovery of Network Patterns of Interaction between Attributes

paper_content:

Visual discovery of network patterns of interaction between attributes in a data set identifies emergent networks between myriads of individual data items and utilises special algorithms that aid visualisation of `emergent' patterns and trends in the linkage. It complements conventional data mining methods, which assume the independence between the attributes and the independence between the values of these attributes. The approach complements analytical data mining techniques where the rules or definitions of what might constitute an exception are able to be known and specified ahead of time. For example, in the analysis of transaction data there are no known suspicious transactions. This chapter presents a human-centred visual data mining methodology that addresses the issues of depicting implicit relationships between data attributes and/or specific values of these attributes. Different aspects of the approach is demonstrated through the reflection of the analytical process in two cases: one looking at fraudulent activity which will be difficult, if not impossible to detect with conventional exception detection methods, and the other one looking at exploring a large data set of low level communication data. The chapter argues that for many problems, a `discovery' phase in the investigative process based on visualisation and human cognition is a logical precedent to, and complement of, more automated `exception detection' phases.
---
paper_title: A Visualization Approach for Frauds Detection in Financial Market

paper_content:

The traditional solutions to the stock market security are not sufficient in identifying attackers and further attack plans from the analysis of existing events.Therefore, it is difficult for analysts to prevent future unexpected events or frauds by only monitoring the realtime trading information. The event-driven fraud detection in financial market could not help analysts to find attack plans and the further intention of attackers. This paper proposed a new framework of visual analytics for stock market security. The proposed solution consists of two stages: 1) Visual Surveillance of Market Performance, and 2) Behavior-Driven Visual Analysis of Trading Networks.In the first stage, we use a 3D treemaps to monitor the real-time stock market performance and to identify a particular stock that produced an unusual trading pattern. We then move to the next stage: social network visualization to conduct behavior-driven visual analysis of suspected pattern. Through the visual analysis of social (or trading)network, analysts may finally identify the attackers (the sources of the fraud), and further attack plans.
---
paper_title: Visualizing Topic Models

paper_content:

Managing large collections of documents is an important problem for many areas of science, industry, and culture. Probabilistic topic modeling offers a promising solution. Topic modeling is an unsupervised machine learning method that learns the underlying themes in a large collection of otherwise unorganized documents. This discovered structure summarizes and organizes the documents. However, topic models are high-level statistical tools—a user must scrutinize numerical distributions to understand and explore their results. In this paper, we present a method for visualizing topic models. Our method creates a navigator of the documents, allowing users to explore the hidden structure that a topic model discovers. These browsing interfaces reveal meaningful patterns in a collection, helping end-users explore and understand its contents in new ways. We provide open source software of our method.
---
paper_title: Dynamic topic models

paper_content:

A family of probabilistic time series models is developed to analyze the time evolution of topics in large document collections. The approach is to use state space models on the natural parameters of the multinomial distributions that represent the topics. Variational approximations based on Kalman filters and nonparametric wavelet regression are developed to carry out approximate posterior inference over the latent topics. In addition to giving quantitative, predictive models of a sequential corpus, dynamic topic models provide a qualitative window into the contents of a large document collection. The models are demonstrated by analyzing the OCR'ed archives of the journal Science from 1880 through 2000.
---
paper_title: TextArc: Showing Word Frequency and Distribution in Text

paper_content:

TextArc is an alternate view of a text, tailored to expose the frequency and distribution of the words of an entire text on a single page or screen. In texts having no markup or metainformation, one of the quickest ways of getting a feeling for the content of a text is to scan through the words that are used most frequently. Knowing the distribution of those words in the text can support another level of understanding, e.g. helping to reveal chapters in a text that concentrate on a specific subject. A structure and method of displaying an entire text on a single page or screen is presented. It reveals both frequency and distribution, and provides a well-understood and organized space that works as a background for other tools.
---
paper_title: Text analysis and knowledge mining system

paper_content:

Large text databases potentially contain a great wealth of knowledge. However, text represents factual information (and information about the author's communicative intentions) in a complex, rich, and opaque manner. Consequently, unlike numerical and fixed field data, it cannot be analyzed by standard statistical data mining methods. Relying on human analysis results in either huge workloads or the analysis of only a tiny fraction of the database. We are working on text mining technology to extract knowledge from very large amounts of textual data. Unlike information retrieval technology that allows a user to select documents that meet the user's requirements and interests, or document clustering technology that organizes documents, we focus on finding valuable patterns and rules in text that indicate trends and significant features about specific topics. By applying our prototype system named TAKMI (Text Analysis and Knowledge Mining) to textual databases in PC help centers, we can automatically detect product failures; determine issues that have led to rapid increases in the number of calls and their underlying reasons; and analyze help center productivity and changes in customers' behavior involving a particular product, without reading any of the text. We have verified that our framework is also effective for other data such as patent documents.
---
paper_title: A self-organizing semantic map for information retrieval

paper_content:

Self-organizing Semantic Map for Information Retrieval Xia Lin Dagobert Soergel Gary Marchionini College of Library and Information Services University of Maryland College Park, A neural network’s unsupervised learning algorithm, Kohonen’s feature map, is applied to constructing a selforganizing semantic map for information retrieval. The semantic map visualizes semantic relationships between input documents, and has properties of economic representation of data with their interrelationships. The potentials of the semantic map include using the map as a retrieval interface for an online bibliographic system. A prototype system that demonstrates this potential is described.
---
paper_title: Classification trees with bivariate splits

paper_content:

We extend the recursive partitioning approach to classifier learning to use more complex types of split at each decision node. The new split types we permit are bivariate and can thus be interpreted visually in plots and tables. In order to find optimal splits of these new types, a new split criterion is introduced that allows the development of divide-and-conquer type algorithms. Two experiments are presented in which the bivariate trees—both with the Gini split criterion and with the new split criterion—are compared to a traditional tree-growing procedure. With the Gini criterion, the bivariate trees show a slight improvement in predictive accuracy and a considerable improvement in tree size over univariate trees. Under the new split criterion, accuracy is also improved, but there is no consistent improvement in tree size.
---
paper_title: HD-Eye: visual mining of high-dimensional data

paper_content:

Clustering in high-dimensional databases poses an important problem. However, we can apply a number of different clustering algorithms to high-dimensional data. The authors consider how an advanced clustering algorithm combined with new visualization methods interactively clusters data more effectively. Experiments show these techniques improve the data mining process.
---
paper_title: Interactive Model Prototyping in Visualization Space

paper_content:

Researching formal models that explain selected natural phenomena of interest is a central aspect of most scientific work. A tested and confirmed model can be the key to classification, knowledge crystallization, and prediction. With this paper we propose a new approach to rapidly draft, fit and quantify model prototypes in visualization space. We also show that these models can provide important insights and accurate metrics about the original data. Using our technique, which is similar to the statistical concept of de-trending, data that behaves according to the model is de-emphasized, leaving only outliers and potential model flaws for further inspection. Moreover, we provide several techniques to assist the user in the process of prototyping such models. We demonstrate the usability of this approach in the context of the analysis of streaming process data from the Norwegian oil and gas industry, and on weather data, investigating the distribution of temperatures over the course of a year.
---
paper_title: A Visualization to Investigate and Give Feedback to Classifiers

paper_content:

Classification is a core technique of data mining. The performance of a classifier might be improved by user feedback especially when a stable classifier cannot be generated from the available training data. Classification processes and classifier decisions are sometimes hard to grasp. However, for giving feedback users need to understand a classifier’s decision and the results. Visualizations can boost the understanding of classification processes and results even for non-expert users. In the past, several classifier visualizations were proposed, however they are either restricted to a limited set of classifiers [Chodos and Zaiane 2008] or no feedback mechanisms are supported [Rheingans and desJardins 2000]. In this work we present a visualization system to investigate and give feedback to all classifiers whose predictions can be interpreted as probability distribution over classes. We provide an intuitive and simple application to observe classification processes and results even for non-expert users. Users can give feedback to classifiers directly in the visualization with well-known interaction techniques like drag and drop.
---
paper_title: Ensemble Methods in Machine Learning

paper_content:

Ensemble methods are learning algorithms that construct a set of classifiers and then classify new data points by taking a (weighted) vote of their predictions. The original ensemble method is Bayesian averaging, but more recent algorithms include error-correcting output coding, Bagging, and boosting. This paper reviews these methods and explains why ensembles can often perform better than any single classifier. Some previous studies comparing ensemble methods are reviewed, and some new experiments are presented to uncover the reasons that Adaboost does not overfit rapidly.
---
paper_title: EnsembleMatrix: interactive visualization to support machine learning with multiple classifiers

paper_content:

Machine learning is an increasingly used computational tool within human-computer interaction research. While most researchers currently utilize an iterative approach to refining classifier models and performance, we propose that ensemble classification techniques may be a viable and even preferable alternative. In ensemble learning, algorithms combine multiple classifiers to build one that is superior to its components. In this paper, we present EnsembleMatrix, an interactive visualization system that presents a graphical view of confusion matrices to help users understand relative merits of various classifiers. EnsembleMatrix allows users to directly interact with the visualizations in order to explore and build combination models. We evaluate the efficacy of the system and the approach in a user study. Results show that users are able to quickly combine multiple classifiers operating on multiple feature sets to produce an ensemble classifier with accuracy that approaches best-reported performance classifying images in the CalTech-101 dataset.
---
paper_title: A survey of data mining and knowledge discovery software tools

paper_content:

Knowledge discovery in databases is a rapidly growing field, whose development is driven by strong research interests as well as urgent practical, social, and economical needs. While the last few years knowledge discovery tools have been used mainly in research environments, sophisticated software products are now rapidly emerging. In this paper, we provide an overview of common knowledge discovery tasks and approaches to solve these tasks. We propose a feature classification scheme that can be used to study knowledge and data mining software. This scheme is based on the software's general characteristics, database connectivity, and data mining characteristics. We then apply our feature classification scheme to investigate 43 software products, which are either research prototypes or commercially available. Finally, we specify features that we consider important for knowledge discovery software to possess in order to accommodate its users effectively, as well as issues that are either not addressed or insufficiently solved yet.
---
paper_title: WEKA: a machine learning workbench

paper_content:

WEKA is a workbench for machine learning that is intended to aid in the application of machine learning techniques to a variety of real-world problems, in particular, those arising from agricultural and horticultural domains. Unlike other machine learning projects, the emphasis is on providing a working environment for the domain specialist rather than the machine learning expert. Lessons learned include the necessity of providing a wealth of interactive tools for data manipulation, result visualization, database linkage, and cross-validation and comparison of rule sets, to complement the basic machine learning tools. >
---
paper_title: KNIME: The Konstanz Information Miner

paper_content:

The Konstanz Information Miner is a modular environment, which enables easy visual assembly and interactive execution of a data pipeline. It is designed as a teaching, research and collaboration platform, which enables simple integration of new algorithms and tools as well as data manipulation or visualization methods in the form of new modules or nodes. In this paper we describe some of the design aspects of the underlying architecture and briefly sketch how new nodes can be incorporated.
---
paper_title: Orange: From Experimental Machine Learning to Interactive Data Mining

paper_content:

Orange (www.ailab.si/orange) is a suite for machine learning and data mining. For researchers in machine learning, Orange offers scripting to easily prototype new algorithms and experimental procedures. For explorative data analysis, it provides a visual programming framework with emphasis on interactions and creative combinations of visual components.
---
paper_title: A survey and taxonomy of approaches for mining software repositories in the context of software evolution

paper_content:

A comprehensive literature survey on approaches for mining software repositories (MSR) in the context of software evolution is presented. In particular, this survey deals with those investigations that examine multiple versions of software artifacts or other temporal information. A taxonomy is derived from the analysis of this literature and presents the work via four dimensions: the type of software repositories mined (what), the purpose (why), the adopted/invented methodology used (how), and the evaluation method (quality). The taxonomy is demonstrated to be expressive (i.e., capable of representing a wide spectrum of MSR investigations) and effective (i.e., facilitates similarities and comparisons of MSR investigations). Lastly, a number of open research issues in MSR that require further investigation are identified.
---
paper_title: Using Visual Interpretation of Small Ensembles in Microarray Analysis

paper_content:

Many different classification models and techniques have been employed on gene expression data. These computational methods are in rapid and continuous evolution and there is no clear consensus on which methods are best to cope with the complex microarray data analysis. Currently ensembles of classifiers are regarded as one of the best classification techniques as they can achieve excellent classification accuracy in comparison to single classifiers methods. One of their main drawbacks is their incomprehensibility. This paper addresses the important issue of the tradeoff between accuracy and comprehensibility when building ensembles and proposes a novel visual technique for interactive interpretation of the knowledge from the small ensembles consisting of only a few decision trees. This way we can achieve better accuracy compared to single classifier, but still maintain a certain level of comprehensibility in small ensembles. The results show that our small ensembles outperform the single classifiers and still retain comprehensibility. Our study also points out that in order to take advantage of our proposed method we need more effective small ensemble building techniques.
---
paper_title: EVALUATION: FROM PRECISION, RECALL AND F-MEASURE TO ROC, INFORMEDNESS, MARKEDNESS & CORRELATION

paper_content:

Commonly used evaluation measures including Recall, Precision, F-Factor and Rand Accuracy are biased and should not be used without clear understanding of the biases, and corresponding identification of chance or base case levels of the statistic. Using these measures a system that performs worse in the objective sense of Informedness, can appear to perform better under any of these commonly used measures. We discuss several concepts and measures that reflect the probability that prediction is informed versus chance. Informedness and introduce Markedness as a dual measure for the probability that prediction is marked versus chance. Finally we demonstrate elegant connections between the concepts of Informedness, Markedness, Correlation and Significance as well as their intuitive relationships with Recall and Precision, and outline the extension from the dichotomous case to the general multi-class case. .
---
paper_title: Cost curves: An improved method for visualizing classifier performance

paper_content:

This paper introduces cost curves, a graphical technique for visualizing the performance (error rate or expected cost) of 2-class classifiers over the full range of possible class distributions and misclassification costs. Cost curves are shown to be superior to ROC curves for visualizing classifier performance for most purposes. This is because they visually support several crucial types of performance assessment that cannot be done easily with ROC curves, such as showing confidence intervals on a classifier's performance, and visualizing the statistical significance of the difference in performance of two classifiers. A software tool supporting all the cost curve analysis described in this paper is available from the authors.
---
paper_title: A novel scalable multi-class ROC for effective visualization and computation

paper_content:

This paper introduces a new cost function for evaluating the multi-class classifier. The new cost function facilitates both a way to visualize the performance (expected cost) of the multi-class classifier and a summary of the misclassification costs. This function overcomes the limitations of ROC in not being able to represent the classifier performance graphically when there are more than two classes. Here we present a new scalable method for producing a scalar measurement that is used to compare the performance of the multi-class classifier. We mathematically demonstrate that our technique can capture small variations in classifier performance.
---
paper_title: Visual Methods for Analyzing Probabilistic Classification Data

paper_content:

Multi-class classifiers often compute scores for the classification samples describing probabilities to belong to different classes. In order to improve the performance of such classifiers, machine learning experts need to analyze classification results for a large number of labeled samples to find possible reasons for incorrect classification. Confusion matrices are widely used for this purpose. However, they provide no information about classification scores and features computed for the samples. We propose a set of integrated visual methods for analyzing the performance of probabilistic classifiers. Our methods provide insight into different aspects of the classification results for a large number of samples. One visualization emphasizes at which probabilities these samples were classified and how these probabilities correlate with classification error in terms of false positives and false negatives. Another view emphasizes the features of these samples and ranks them by their separation power between selected true and false classifications. We demonstrate the insight gained using our technique in a benchmarking classification dataset, and show how it enables improving classification performance by interactively defining and evaluating post-classification rules.
---
paper_title: Manual Controls For High-Dimensional Data Projections

paper_content:

Abstract Projections of high-dimensional data onto low-dimensional subspaces provide insightful views for understanding multivariate relationships. This article discusses how to manually control the variable contributions to the projection. The user has control of the way a particular variable contributes to the viewed projection and can interactively adjust the variable's contribution. These manual controls complement the automatic views provided by a grand tour, or a guided tour, and give greatly improved flexibility to data analysts.
---
paper_title: Visualizing Class Probability Estimators

paper_content:

Inducing classifiers that make accurate predictions on future data is a driving force for research in inductive learning. However, also of importance to the users is how to gain information from the models produced. Unfortunately, some of the most powerful inductive learning algorithms generate “black boxes”—that is, the representation of the model makes it virtually impossible to gain any insight into what has been learned. This paper presents a technique that can help the user understand why a classifier makes the predictions that it does by providing a two-dimensional visualization of its class probability estimates. It requires the classifier to generate class probabilities but most practical algorithms are able to do so (or can be modified to this end).
---
paper_title: Gaining insights into support vector machine pattern classifiers using projection-based tour methods

paper_content:

This paper discusses visual methods that can be used to understand and interpret the results of classification using support vector machines (SVM) on data with continuous real-valued variables. SVM induction algorithms build pattern classifiers by identifying a maximal margin separating hyperplane from training examples in high dimensional pattern spaces or spaces induced by suitable nonlinear kernel transformations over pattern spaces. SVM have been demonstrated to be quite effective in a number of practical pattern classification tasks. Since the separating hyperplane is defined in terms of more than two variables it is necessary to use visual techniques that can navigate the viewer through high-dimensional spaces. We demonstrate the use of projection-based tour methods to gain useful insights into SVM classifiers with linear kernels on 8-dimensional data.
---
paper_title: Nomograms for visualizing support vector machines

paper_content:

We propose a simple yet potentially very effective way of visualizing trained support vector machines. Nomograms are an established model visualization technique that can graphically encode the complete model on a single page. The dimensionality of the visualization does not depend on the number of attributes, but merely on the properties of the kernel. To represent the effect of each predictive feature on the log odds ratio scale as required for the nomograms, we employ logistic regression to convert the distance from the separating hyperplane into a probability. Case studies on selected data sets show that for a technique thought to be a black-box, nomograms can clearly expose its internal structure. By providing an easy-to-interpret visualization the analysts can gain insight and study the effects of predictive factors.
---
paper_title: Nomograms for Visualization of Naive Bayesian Classifier

paper_content:

Besides good predictive performance, the naive Bayesian classifier can also offer a valuable insight into the structure of the training data and effects of the attributes on the class probabilities. This structure may be effectively revealed through visualization of the classifier. We propose a new way to visualize the naive Bayesian model in the form of a nomogram. The advantages of the proposed method are simplicity of presentation, clear display of the effects of individual attribute values, and visualization of confidence intervals. Nomograms are intuitive and when used for decision support can provide a visual explanation of predicted probabilities. And finally, with a nomogram, a naive Bayesian model can be printed out and used for probability prediction without the use of computer or calculator.
---
paper_title: Visualizing information on a sphere

paper_content:

We describe a method for the visualization of information units on spherical domains which is employed in the banking industry for risk analysis, stock prediction and other tasks. The system is based on a quantification of the similarity of related objects that governs the parameters of a mass-spring system. Unlike existing approaches we initialize all information units onto the inner surface of two concentric spheres and attach them with springs to the outer sphere. Since the spring stiffnesses correspond to the computed similarity measures, the system converges into an energy minimum which reveals multidimensional relations and adjacencies in terms of spatial neighborhoods. Depending on the application scenario our approach supports different topological arrangements of related objects. In order to cope with large data sets we propose a blobby clustering mechanism that enables encapsulation of similar objects by implicit shapes. In addition, we implemented various interaction techniques allowing semantic analysis of the underlying data sets. Our prototype system IVORY is written in Java, and its versatility is illustrated by an example from financial service providers.
---
paper_title: BicOverlapper: A tool for bicluster visualization

paper_content:

Summary: BicOverlapper is a tool to visualise biclusters from gene expression matrices in a way that helps to compare biclustering methods, to unravel trends and to highlight relevant genes and conditions. A visual approach can complement biological and statistical analysis and reduce the time spent by specialists interpreting the results of biclustering algorithms. The technique is based on a force-directed graph where biclusters are represented as flexible overlapped groups of genes and conditions. Availability: The BicOverlapper software and supplementary material are available at http://vis.usal.es/bicoverlapper.
---
paper_title: Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling

paper_content:

12 Pathology and Microbiology, and 13 Diffuse large B-cell lymphoma (DLBCL), the most common subtype of non-Hodgkin's lymphoma, is clinically heterogeneous: 40% of patients respond well to current therapy and have prolonged survival, whereas the remainder succumb to the disease. We proposed that this variability in natural history reflects unrecognized molecular heterogeneity in the tumours. Using DNA microarrays, we have conducted a systematic characterization of gene expression in B-cell malignancies. Here we show that there is diversity in gene expression among the tumours of DLBCL patients, apparently reflecting the variation in tumour proliferation rate, host response and differentiation state of the tumour. We identified two molecularly distinct forms of DLBCL which had gene expression patterns indicative of different stages of B-cell differentiation. One type expressed genes characteristic of germinal centre B cells ('germinal centre B-like DLBCL'); the second type expressed genes normally induced during in vitro activation of peripheral blood B cells ('activated B-like DLBCL'). Patients with germinal centre B-like DLBCL had a significantly better overall survival than those with activated B-like DLBCL. The molecular classification of tumours on the basis of gene expression can thus identify previously undetected and clinically significant subtypes of cancer.
---
paper_title: H-BLOB: a hierarchical visual clustering method using implicit surfaces

paper_content:

We present a new hierarchical clustering and visualization algorithm called H-BLOB, which groups and visualizes cluster hierarchies at multiple levels-of-detail. Our method is fundamentally different to conventional clustering algorithms, such as C-means, K-means, or linkage methods that are primarily designed to partition a collection of objects into subsets sharing similar attributes. These approaches usually lack an efficient level-of-detail strategy that breaks down the visual complexity of very large datasets for visualization. In contrast, our method combines grouping and visualization in a two stage process constructing a hierarchical setting. In the first stage a cluster tree is computed making use of an edge contraction operator. Exploiting the inherent hierarchical structure of this tree, a second stage visualizes the clusters by computing a hierarchy of implicit surfaces. We believe that H-BLOB is especially suited for the visualization of very large datasets and for visual decision making in information visualization. The versatility of the algorithm is demonstrated using examples from visual data mining.
---
paper_title: The TwoKey Plot for Multiple Association Rules Control

paper_content:

Association rules have been popular in theory, though it is unclear how much success they have had in practice. Very many association rules are found in any application by any approach and they require effective pruning and filtering. There has been much research in this area recently, but less with the goal of providing a global overview and summary of all rules, which may then be used to explore the rules and to evaluate their worth. The unusual feature of association rules is that those with the highest objective values for the two key criteria (support and confidence) are not usually those with the most subjective interest (because we know the obvious results already). The TwoKey plot is a way of displaying all discovered association rules at once, while also providing the means to review and manage them. It is a powerful tool in order to get a first overview of the distribution of confidence and support. Features such as separate groups of rules or outliers are detected immediately. By exploiting various ancestor relationship structures among the rules, we can use the TwoKey Plot also as a visual assessment tool, closely related to pruning methods - e.g. those proposed by Bing Liu (1999). The concept will be illustrated using the interactive software MARC (Multiple Association Rules Control).
---
paper_title: Mining association rules between sets of items in large databases

paper_content:

We are given a large database of customer transactions. Each transaction consists of items purchased by a customer in a visit. We present an efficient algorithm that generates all significant association rules between items in the database. The algorithm incorporates buffer management and novel estimation and pruning techniques. We also present results of applying this algorithm to sales data obtained from a large retailing company, which shows the effectiveness of the algorithm.
---
paper_title: A colour scheme for the display of astronomical intensity images

paper_content:

I describe a colour scheme that is appropriate for the screen display of intensity images. This -- unlike many currently available schemes -- is designed to be monotonically increasing in terms of its perceived brightness. Also, when printed on a black and white postscript printer, the scheme results in a greyscale with monotonically increasing brightness. This scheme has recently been incorporated into the radio astronomical analysis packages CASA and AIPS.
---
paper_title: A survey of data mining and knowledge discovery software tools

paper_content:

Knowledge discovery in databases is a rapidly growing field, whose development is driven by strong research interests as well as urgent practical, social, and economical needs. While the last few years knowledge discovery tools have been used mainly in research environments, sophisticated software products are now rapidly emerging. In this paper, we provide an overview of common knowledge discovery tasks and approaches to solve these tasks. We propose a feature classification scheme that can be used to study knowledge and data mining software. This scheme is based on the software's general characteristics, database connectivity, and data mining characteristics. We then apply our feature classification scheme to investigate 43 software products, which are either research prototypes or commercially available. Finally, we specify features that we consider important for knowledge discovery software to possess in order to accommodate its users effectively, as well as issues that are either not addressed or insufficiently solved yet.
---
paper_title: Visualizing information on a sphere

paper_content:

We describe a method for the visualization of information units on spherical domains which is employed in the banking industry for risk analysis, stock prediction and other tasks. The system is based on a quantification of the similarity of related objects that governs the parameters of a mass-spring system. Unlike existing approaches we initialize all information units onto the inner surface of two concentric spheres and attach them with springs to the outer sphere. Since the spring stiffnesses correspond to the computed similarity measures, the system converges into an energy minimum which reveals multidimensional relations and adjacencies in terms of spatial neighborhoods. Depending on the application scenario our approach supports different topological arrangements of related objects. In order to cope with large data sets we propose a blobby clustering mechanism that enables encapsulation of similar objects by implicit shapes. In addition, we implemented various interaction techniques allowing semantic analysis of the underlying data sets. Our prototype system IVORY is written in Java, and its versatility is illustrated by an example from financial service providers.
---
paper_title: ColorBrewer.org: An Online Tool for Selecting Colour Schemes for Maps

paper_content:

AbstractChoosing effective colour schemes for thematic maps is surprisingly difficult. ColorBrewer is an online tool designed to take some of the guesswork out of this process by helping users select appropriate colour schemes for their specific mapping needs by considering: the number of data classes; the nature of their data (matched with sequential, diverging and qualitative schemes); and the end-use environment for the map (e.g., CRT, LCD, printed, projected, photocopied). ColorBrewer contains 'learn more' tutorials to help guide users, prompts them to test-drive colour schemes as both map and legend, and provides output in five colour specification systems.
---
paper_title: Visualizing Class Probability Estimators

paper_content:

Inducing classifiers that make accurate predictions on future data is a driving force for research in inductive learning. However, also of importance to the users is how to gain information from the models produced. Unfortunately, some of the most powerful inductive learning algorithms generate “black boxes”—that is, the representation of the model makes it virtually impossible to gain any insight into what has been learned. This paper presents a technique that can help the user understand why a classifier makes the predictions that it does by providing a two-dimensional visualization of its class probability estimates. It requires the classifier to generate class probabilities but most practical algorithms are able to do so (or can be modified to this end).
---
paper_title: ImageNet classification with deep convolutional neural networks

paper_content:

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0%, respectively, which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully connected layers we employed a recently developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
---
paper_title: A novel scalable multi-class ROC for effective visualization and computation

paper_content:

This paper introduces a new cost function for evaluating the multi-class classifier. The new cost function facilitates both a way to visualize the performance (expected cost) of the multi-class classifier and a summary of the misclassification costs. This function overcomes the limitations of ROC in not being able to represent the classifier performance graphically when there are more than two classes. Here we present a new scalable method for producing a scalar measurement that is used to compare the performance of the multi-class classifier. We mathematically demonstrate that our technique can capture small variations in classifier performance.
---
paper_title: #FluxFlow: Visual Analysis of Anomalous Information Spreading on Social Media

paper_content:

We present FluxFlow, an interactive visual analysis system for revealing and analyzing anomalous information spreading in social media. Everyday, millions of messages are created, commented, and shared by people on social media websites, such as Twitter and Facebook. This provides valuable data for researchers and practitioners in many application domains, such as marketing, to inform decision-making. Distilling valuable social signals from the huge crowd's messages, however, is challenging, due to the heterogeneous and dynamic crowd behaviors. The challenge is rooted in data analysts' capability of discerning the anomalous information behaviors, such as the spreading of rumors or misinformation, from the rest that are more conventional patterns, such as popular topics and newsworthy events, in a timely fashion. FluxFlow incorporates advanced machine learning algorithms to detect anomalies, and offers a set of novel visualization designs for presenting the detected threads for deeper analysis. We evaluated FluxFlow with real datasets containing the Twitter feeds captured during significant events such as Hurricane Sandy. Through quantitative measurements of the algorithmic performance and qualitative interviews with domain experts, the results show that the back-end anomaly detection model is effective in identifying anomalous retweeting threads, and its front-end interactive visualizations are intuitive and useful for analysts to discover insights in data and comprehend the underlying analytical model.
---

Title: On Integrating Information Visualization Techniques into Data Mining: A Review

Section 1: INTRODUCTION
Description 1: Introduce the motivation behind integrating information visualization techniques into data mining, highlight the differences and synergies between the two fields, and summarize the goals and structure of the paper.

Section 2: RELATED WORK
Description 2: Review existing surveys and research related to visual analytics, visual data mining, and machine learning model-based visualization. Discuss how this paper differs from and contributes to the existing body of work. 

Section 3: VISUALIZATION FOR PRELIMINARY DATA ANALYSIS
Description 3: Describe visualization techniques for preliminary data analysis, focusing on methods for scalar value data visualization, scalar value distribution, visualizing scalar value along other dimensions, multi-dimensional data visualization, and specific relation visualizations like bipartite flow.

Section 4: Scalar value data visualization
Description 4: Review basic visualization techniques for visualizing single numbers, including comparison of values, distribution, and changes over external dimensions.

Section 5: Scalar value distribution
Description 5: Outline techniques for visualizing the distribution of a large group of scalar values, including histograms, probability density functions, and various forms of boxplots.

Section 6: visualizing scalar value along other dimensions
Description 6: Explain methods to study relations between scalar values and other dimensions, such as time-series analysis, scatterplots, dot charts, and spatial data visualizations like Choropleth Maps and level sets.

Section 7: Multi-dimensional data visualization
Description 7: Discuss techniques for multi-dimensional data visualization, including glyph-based vector visualization, scatterplot matrices, parallel coordinates, and 3D visual data mining.

Section 8: VISUALIZATION FOR MODEL CONSTRUCTION
Description 8: Investigate how InfoVis can enhance the process of building machine learning models, focusing on progressive construction, iterative prototyping, ensemble model construction, and pipeline construction.

Section 9: Progressive construction
Description 9: Cover visualization techniques for progressively built machine learning models, such as visual decision tree construction and OptiGrid clustering.

Section 10: Iterative prototyping
Description 10: Describe how feedback-driven visualization systems can iteratively refine models, including systems for classifiers and regression models, and ensemble classifiers.

Section 11: Interactive pipeline construction
Description 11: Discuss algorithm-agnostic approaches for interactive construction of data mining pipelines, highlighting tools like Weka, RapidMiner, and KNIME.

Section 12: VISUALIZATION FOR MODEL EVALUATION
Description 12: Explore techniques for visualizing and evaluating the performance of data mining models, focusing on classification performance, instance space visualization, factor analysis, and more.

Section 13: Classification
Description 13: Discuss visualization techniques for assessing the performance of classification models, including receiver operating characteristic (ROC) curves and tools for a posteriori classification results analysis.

Section 14: Instance space based visualization
Description 14: Elucidate methods for projecting and displaying high-dimensional data instances, including SPLOM and non-linear dimension reduction techniques.

Section 15: Factor analysis
Description 15: Review approaches to visualize the relationship between predictive factors and classification outcomes, such as nomograms and parallel coordinate plots.

Section 16: Clustering
Description 16: Describe visualization techniques for clustering, including BLOB and H-BLOB clustering, biclustering visualizations, and domain-specific applications.

Section 17: Association Rules
Description 17: Discuss methods to visualize association rules, focusing on encoding techniques that can accommodate large sets of rules, such as double-decker plots and parallel coordinates.

Section 18: Neuron Network
Description 18: Detail visualization techniques for neural networks, including visualization of self-organizing maps (SOM) and deep learning feature activations.

Section 19: Graphical Probabilistic Models
Description 19: Explain visualization approaches for probabilistic graphical models, highlighting the graphical nature of these models and their application in various software tools.

Section 20: DISCUSSION AND CONCLUSION
Description 20: Summarize the integration of information visualization techniques into data mining, reflect on the survey’s coverage and taxonomy, and discuss future research directions and practical applications.