Performance Visualization for Large-Scale Computing Systems: A Literature Review
---
paper_title: A performance visualization paradigm for data parallel computing

paper_content:

Observing the activities of a complex parallel computer system is no small feat, and relating these observations to program behavior is even harder. This paper presents a general measurement approach that is applicable to a large class of scalable programs and machines, specifically data parallel programs executing on distributed memory computer systems. The combined instrumentation and visualization paradigm, called VISTA (which stands for Visualization and Instrumentation of Scalable mulTicomputer Applications), is based on the author's experiences of programming and monitoring applications running on an nCUBE 2 computer and a MasPar MP-1 computer. The key is that performance data are treated similarly to any distributed data in the context of the data parallel programming model. Because of the data-parallel mapping of the program onto the machine, one can view the performance as it relates to each processor, processor cluster or processor ensemble and as it relates to the data structures of the program. The author illustrates the utility of VISTA by way of an example. >
---
paper_title: Measuring the robustness of a resource allocation

paper_content:

Parallel and distributed systems may operate in an environment that undergoes unpredictable changes causing certain system performance features to degrade. Such systems need robustness to guarantee limited degradation despite fluctuations in the behavior of its component parts or environment. This research investigates the robustness of an allocation of resources to tasks in parallel and distributed systems. The main contributions are 1) a mathematical description of a metric for the robustness of a resource allocation with respect to desired system performance features against multiple perturbations in multiple system and environmental conditions, and 2) a procedure for deriving a robustness metric for an arbitrary system. For illustration, this procedure is employed to derive robustness metrics for three example distributed systems. Such a metric can help researchers evaluate a given resource allocation for robustness against uncertainties in specified perturbation parameters.
---
paper_title: Visualizing the performance of parallel programs

paper_content:

ParaGraph, a software tool that provides a detailed, dynamic, graphical animation of the behavior of message-passing parallel programs and graphical summaries of their performance, is presented. ParaGraph animates trace information from actual runs to depict behavior and obtain the performance summaries. It provides twenty-five perspectives on the same data, lending insight that might otherwise be missed. ParaGraph's features are described, its use is explained, its software design is briefly discussed, and its displays are examined in some detail. Future work on ParaGraph is indicated. >
---
paper_title: Stochastic robustness metric and its use for static resource allocations

paper_content:

This research investigates the problem of robust static resource allocation for distributed computing systems operating under imposed Quality of Service (QoS) constraints. Often, such systems are expected to function in a physical environment replete with uncertainty, which causes the amount of processing required to fluctuate substantially over time. Determining a resource allocation that accounts for this uncertainty in a way that can provide a probabilistic guarantee that a given level of QoS is achieved is an important research problem. The stochastic robustness metric proposed in this research is based on a mathematical model where the relationship between uncertainty in system parameters and its impact on system performance are described stochastically. The utility of the established metric is then exploited in the design of optimization techniques based on greedy and iterative approaches that address the problem of resource allocation in a large class of distributed systems operating on periodically updated data sets. The performance results are presented for a simulated environment that replicates a heterogeneous cluster-based radar data processing center. A mathematical performance lower bound is presented for comparison analysis of the heuristic results. The lower bound is derived based on a relaxation of the Integer Linear Programming formulation for a given resource allocation problem.
---
paper_title: A taxonomy of program visualization systems

paper_content:

A taxonomy of program visualization systems that is based on a model of program visualization that maps programs to graphical representations is presented. The taxonomy is illustrated with three program visualization systems representative of research trends: Zeus, Tango, and Pavane. >
---
paper_title: A performance visualization paradigm for data parallel computing

paper_content:

Observing the activities of a complex parallel computer system is no small feat, and relating these observations to program behavior is even harder. This paper presents a general measurement approach that is applicable to a large class of scalable programs and machines, specifically data parallel programs executing on distributed memory computer systems. The combined instrumentation and visualization paradigm, called VISTA (which stands for Visualization and Instrumentation of Scalable mulTicomputer Applications), is based on the author's experiences of programming and monitoring applications running on an nCUBE 2 computer and a MasPar MP-1 computer. The key is that performance data are treated similarly to any distributed data in the context of the data parallel programming model. Because of the data-parallel mapping of the program onto the machine, one can view the performance as it relates to each processor, processor cluster or processor ensemble and as it relates to the data structures of the program. The author illustrates the utility of VISTA by way of an example. >
---
paper_title: The paradyn parallel performance measurement tools

paper_content:

Paradyn is a tool for measuring the performance of large-scale parallel programs. Our goal in designing a new performance tool was to provide detailed, flexible performance information without incurring the space (and time) overhead typically associated with trace-based tools. Paradyn achieves this goal by dynamically instrumenting the application and automatically controlling this instrumentation in search of performance problems. Dynamic instrumentation lets us defer insertion until the moment it is needed (and remove it when it is no longer needed); Paradyn's Performance Consultant decides when and where to insert instrumentation. >
---
paper_title: Performance analysis challenges and framework for high-performance reconfigurable computing

paper_content:

Reconfigurable computing (RC) applications employing both microprocessors and FPGAs have potential for large speedup when compared with traditional (software) parallel applications. However, this potential is marred by the additional complexity of these dual-paradigm systems, making it difficult to identify performance bottlenecks and achieve desired performance. Performance analysis concepts and tools are well researched and widely available for traditional parallel applications but are lacking in RC, despite being of great importance due to the applications' increased complexity. In this paper, we explore challenges and present new techniques in automated instrumentation, runtime measurement, and visualization of RC application behavior. We also present ideas for integration with conventional performance analysis tools to create a unified tool for RC applications as well as our initial framework for FPGA instrumentation and measurement. Results from a case study are provided using a prototype of this new tool.
---
paper_title: High-level performance modeling of task-based algorithms

paper_content:

Performing modeling and visualization of task-based parallel algorithms is challenging. Libraries such as Intel Threading Building Blocks (TBB) and Microsoft's Parallel Patterns Library provide high-level algorithms that are implemented using low-level tasks. Current tools present performance at this lower level. Developers like to tune and debug at the same level as the coding abstraction, so in this paper we propose tools and a two step methodology that target this level of abstraction. In the first step, the system level metrics of utilization and overhead are collected to determine if performance is acceptable. If a problem is suspected, the second step of our methodology projects these metrics on to the algorithms contained in the application. Using these projections many common performance issues can be quickly diagnosed. We demonstrate our methodology using a prototype implementation that is integrated with the Intel Threading Building Blocks library. We show the flexibility of the approach by analyzing three applications, including a client-server benchmark that uses a parallel_for nested within a parallel pipeline.
---
paper_title: The visualization of parallel systems: an overview

paper_content:

Abstract We present an overview of visualization tools for parallel systems focusing on parallel debuggers, performance evaluation systems, and program visualization systems. Our goal in this paper is to summarize the issues addressed throughout the entire process of generating a visualization, highlighting the areas in which particular systems have made important contributions. Our discussion is focused along two levels: the visualization task being performed, and the purpose of the visualization. This article also serves as a bibliographic compendium of existing research on the visualization of parallel systems.
---
paper_title: Visualizing the performance of parallel programs

paper_content:

ParaGraph, a software tool that provides a detailed, dynamic, graphical animation of the behavior of message-passing parallel programs and graphical summaries of their performance, is presented. ParaGraph animates trace information from actual runs to depict behavior and obtain the performance summaries. It provides twenty-five perspectives on the same data, lending insight that might otherwise be missed. ParaGraph's features are described, its use is explained, its software design is briefly discussed, and its displays are examined in some detail. Future work on ParaGraph is indicated. >
---
paper_title: Parallel program visualization using SIEVE.1

paper_content:

In this paper we introduce a new model for the design of performance analysis and visualization tools. The system integrates static code analysis, relational database designs and a spreadsheet model of interactive programming. This system provides a general methodology for visualizing parallel program executions by providing correlation to the source program, a convenient method for rapidly developing new visualizations of the program and a framework for performance studies. The basic semantics of this model are described and example applications of visualizing the execution of two different parallel programs are considered.
---
paper_title: Pajé, an interactive visualization tool for tuning multi-threaded parallel applications

paper_content:

Abstract This paper describes Paje, an interactive visualization tool for displaying the execution of parallel applications where a potentially large number of communicating threads of various life-times execute on each node of a distributed memory parallel system. Paje is capable of representing a wide variety of interactions between threads. The main characteristics of Paje, interactivity and scalability, are exemplified by the performance tuning of a molecular dynamics application. In order to be easily extensible, the architecture of the system was based on components which are connected in a data flow graph to produce a given visualization tool. Innovative components were designed, in addition to “classical” components existing in similar visualization systems, to support scalability and interactivity.
---
paper_title: A taxonomy of program visualization systems

paper_content:

A taxonomy of program visualization systems that is based on a model of program visualization that maps programs to graphical representations is presented. The taxonomy is illustrated with three program visualization systems representative of research trends: Zeus, Tango, and Pavane. >
---
paper_title: Debugging heterogeneous distributed systems using event-based models of behavior

paper_content:

Event Based Behavioral Abstraction (EBBA) is a high-level debugging approach which treats debugging as a process of creating models of actual behavior from the activity of the system and comparing these to models of expected system behavior. The differences between the actual and expected models are used to characterize erroneous system behavior and direct further investigation. A set of EBBA-based tools has been implemented that users can employ to construct libraries of behavior models and investigate the behavior of an errorful system through these models. EBBA evolves naturally as a cooperative distributed program that can take better advantage of computational power available in a network computer system to enhance debugging tool transparency, reduce latency and uncertainty for fundamental debugging activities and accommodate diverse, heterogeneous architectures.
---
paper_title: Performance Measurement, Visualization and Modeling of Parallel and Distributed Programs using the AIMS Toolkit

paper_content:

Writing large-scale parallel and distributed scientific applications that make optimum use of the multiprocessor is a challenging problem. Typically, computational resources are underused due to performance failures in the application being executed. Performance-tuning tools are essential for exposing these performance failures and for suggesting ways to improve program performance. In this paper, we first address fundamental issues in building useful performance-tuning tools and then describe our experience with the AIMS toolkit for tuning parallel and distributed programs on a variety of platforms. AIMS supports source-code instrumentation, run-time monitoring, graphical execution profiles, performance indices and automated modeling techniques as ways to expose performance problems of programs. Using several examples representing a broad range of scientific applications, we illustrate AIMS' effectiveness in exposing performance problems in parallel and distributed programs.
---
paper_title: Integrated Visualization of Parallel Program Performance Data

paper_content:

Abstract Performance tuning a parallel application involves integrating performance data from many components of the system, including the message passing library, performance monitoring tool, resource manager, operating system, and the application itself. The current practice of visualizing these data streams using a separate, customized tool for each source is inconvenient from a usability perspective, and there is no easy way to visualize the data in an integrated fashion. We demonstrate a solution to this problem using Devise, a generic visualization tool which is designed to allow an arbitrary number of different but related data streams to be integrated and explored visually in a flexible manner. We display data emanating from a variety of sources side by side in three case studies. First we interface the Paradyn parallel performance tool and Devise, using two simple data export modules and Paradyn's simple visualization interface. We show several Devise/Paradyn visualizations which are useful for performance tuning parallel codes, and which incorporate data from Unix utilities and application output. Next we describe the visualization of trace data from a parallel application running in a Condor cluster of workstations. Finally we demonstrate the utility of Devise visualizations in a study of Condor cluster activity.
---
paper_title: Flexible performance visualization of parallel and distributed applications �

paper_content:

Performance debugging of parallel and distributed applications can benefit from behavioral visualization tools helping to capture the dynamics of the executions of applications. The Paje generic tool presented in this article provides interactive and scalable behavioral visualizations; because of its genericity, it can be used unchanged in a large variety of contexts.
---
paper_title: Visualizing the performance of parallel programs

paper_content:

ParaGraph, a software tool that provides a detailed, dynamic, graphical animation of the behavior of message-passing parallel programs and graphical summaries of their performance, is presented. ParaGraph animates trace information from actual runs to depict behavior and obtain the performance summaries. It provides twenty-five perspectives on the same data, lending insight that might otherwise be missed. ParaGraph's features are described, its use is explained, its software design is briefly discussed, and its displays are examined in some detail. Future work on ParaGraph is indicated. >
---
paper_title: A visualization tool for analyzing cluster performance data

paper_content:

This paper describes a unique visualization tool that has been used to analyze performance of the Cplant(tm) clusters at Sandia National Laboratories.As commodity cluster systems grow in size and complexity, understanding performance issues becomes more and more difficult.We have developed a tool that facilitates visual performance analysis within the context of the physical and runtime environment of a system. Combining an abstract system model with color-coding for both performance and job information enables quick fault isolation as well as insight into complex system behavior.
---
paper_title: Cone Trees: animated 3D visualizations of hierarchical information

paper_content:

The task of managing and accessing large information spaces is a problem in large scale cognition. Enwrging technologies for 3D visualization and interactive aninlaiion offer potential solutions to this problenl, especially when the structure of the information can be visualized. We describe one of these Information Visualtzaiion techniques, called the Cone Tree, which is used for visualizing hierarchical information structures. The hierarchy is presented in 3D to nlaxilnize effective use of available screen space and enable visualization of the whole structure. Interactive aninlation is used (,o shift sonle of the user’s cognitive load to the human perceptual system,
---
paper_title: Scalable thread visualization for debugging data races in OpenMP programs

paper_content:

It is important to debug unintended data races in OpenMP programs efficiently, because such programs are often complex and longrunning. Previous tools for detecting the races does not provide any effective facility for understanding the complexity of threads involved in the reported races. This paper presents a thread visualization tool to present a partial order of threads executed in the traced programs with a scalable graph of abstract threads upon a three-dimensional cone. The scalable thread visualization is proved to be effective in debugging races using a set of synthetic programs.
---
paper_title: Performance Measurement, Visualization and Modeling of Parallel and Distributed Programs using the AIMS Toolkit

paper_content:

Writing large-scale parallel and distributed scientific applications that make optimum use of the multiprocessor is a challenging problem. Typically, computational resources are underused due to performance failures in the application being executed. Performance-tuning tools are essential for exposing these performance failures and for suggesting ways to improve program performance. In this paper, we first address fundamental issues in building useful performance-tuning tools and then describe our experience with the AIMS toolkit for tuning parallel and distributed programs on a variety of platforms. AIMS supports source-code instrumentation, run-time monitoring, graphical execution profiles, performance indices and automated modeling techniques as ways to expose performance problems of programs. Using several examples representing a broad range of scientific applications, we illustrate AIMS' effectiveness in exposing performance problems in parallel and distributed programs.
---
paper_title: A performance visualization paradigm for data parallel computing

paper_content:

Observing the activities of a complex parallel computer system is no small feat, and relating these observations to program behavior is even harder. This paper presents a general measurement approach that is applicable to a large class of scalable programs and machines, specifically data parallel programs executing on distributed memory computer systems. The combined instrumentation and visualization paradigm, called VISTA (which stands for Visualization and Instrumentation of Scalable mulTicomputer Applications), is based on the author's experiences of programming and monitoring applications running on an nCUBE 2 computer and a MasPar MP-1 computer. The key is that performance data are treated similarly to any distributed data in the context of the data parallel programming model. Because of the data-parallel mapping of the program onto the machine, one can view the performance as it relates to each processor, processor cluster or processor ensemble and as it relates to the data structures of the program. The author illustrates the utility of VISTA by way of an example. >
---
paper_title: Visualization of repetitive patterns in event traces

paper_content:

Performance Tracing has always been challenged by large amounts of trace data. Software tools for trace analysis and visualization successfully cope with ever growing trace sizes. Still, human perception is unable to "scale up" with the amounts of data. ::: ::: With a new model of trace visualization, we try to provide less data but additional information or rather more convenient information to human users. By marking regular repetition patterns and hiding the inner details, less complex visualization can offer better insight. At the same time potentially interesting irregular sections are revealed. ::: ::: The paper introduces the origin of repetition patterns and outlines the detection algorithm used. It demonstrates the new visualization scheme which has also been incorporated into Vampir NG as a prototype. Finally, it gives an outlook on further development and possible extensions.
---
paper_title: Visualizing complex dynamics in many-core accelerator architectures

paper_content:

While many-core accelerator architectures, such as today's Graphics Processing Units (GPUs), offer orders of magnitude more raw computing power than contemporary CPUs, their massive parallelism often produces complex dynamic behaviors even with the simplest applications. Using a fixed set of hardware or simulator performance counters to quantify behavior over a large interval of time such as an entire application execution run or program phase may not capture this behavior. Software and/or hardware designers may consequently miss out on opportunities to optimize for better performance. Similarly, significant effort may be expended to find metrics that explain anomalous behavior in architecture design studies. Moreover, the increasing complexity of applications developed for today's GPU has created additional difficulties for software developers when attempting to identify bottlenecks of an application for optimization. This paper presents a novel GPU performance visualization tool, AerialVision, to address these two problems. It interfaces with the GPGPU-Sim simulator to capture and visualize the dynamic behavior of a GPU architecture throughout an application run. Similar to existing performance analysis tools for CPUs, it can annotate individual lines of source code with performance statistics to simplify the bottleneck identification process. To provide further insight, AerialVision introduces a novel methodology to relate pathological dynamic architectural behaviors resulting in performance loss with the part of the source code that is responsible. By rapidly providing insight into complex dynamic behavior, AerialVision enables research on improving many-core accelerator architectures and will help ensure applications written for these architectures reach their full performance potential.
---
paper_title: Visualizing the performance of parallel programs

paper_content:

ParaGraph, a software tool that provides a detailed, dynamic, graphical animation of the behavior of message-passing parallel programs and graphical summaries of their performance, is presented. ParaGraph animates trace information from actual runs to depict behavior and obtain the performance summaries. It provides twenty-five perspectives on the same data, lending insight that might otherwise be missed. ParaGraph's features are described, its use is explained, its software design is briefly discussed, and its displays are examined in some detail. Future work on ParaGraph is indicated. >
---
paper_title: PET, a software monitoring toolkit for performance analysis of parallel embedded applications

paper_content:

Since the late 1980s a great deal of research has been dedicated to the development of software monitoring and visualization toolkits for parallel systems. These toolkits have traditionally been oriented to measuring and analyzing parallel scientific applications. However, nowadays, other types of parallel applications, using signal-processing or image-processing techniques, are becoming increasingly importance in the field of embedded computing. Such applications are executed on special parallel computers, normally known as embedded multiprocessors, and they exhibit structural and behavioral characteristics very different from scientific applications. Because of this, monitoring tools with specific characteristics are required for measuring and analyzing parallel embedded applications.In this paper we present performance execution tracer (PET), a monitoring and visualization toolkit specifically developed to measure and analyze this type of application. Special emphasis is placed on explaining how PET deals with the particular characteristics of these applications. In addition, in order to demonstrate the capabilities of PET, the measurement and analysis of a real application using this tool are described.
---
paper_title: Paje: An Extensible Environment for Visualizing Multi-Threaded Program Executions

paper_content:

PajE is an interactive visualization tool for displaying the execution of parallel applications where a (potentially) large number of communicating threads of various life-times execute on each node of a distributed memory parallel system. The main novelty of PajE is an original combination of three of the most desirable properties of visualization tools for parallel programs: extensibility, interactivity and scalability. This article mainly focuses on the extensibility property of PajE, ability to easily add new functionalities to the tool. PajE was designed as a data-flow graph of modular components to ease the replacement of existing modules or the implementation of new ones. In addition the genericity of PajE allows application programmers to tailor the visualization to their needs, by simply adding tracing orders to the programs being traced.
---
paper_title: Parallel program visualization using SIEVE.1

paper_content:

In this paper we introduce a new model for the design of performance analysis and visualization tools. The system integrates static code analysis, relational database designs and a spreadsheet model of interactive programming. This system provides a general methodology for visualizing parallel program executions by providing correlation to the source program, a convenient method for rapidly developing new visualizations of the program and a framework for performance studies. The basic semantics of this model are described and example applications of visualizing the execution of two different parallel programs are considered.
---
paper_title: Pajé, an interactive visualization tool for tuning multi-threaded parallel applications

paper_content:

Abstract This paper describes Paje, an interactive visualization tool for displaying the execution of parallel applications where a potentially large number of communicating threads of various life-times execute on each node of a distributed memory parallel system. Paje is capable of representing a wide variety of interactions between threads. The main characteristics of Paje, interactivity and scalability, are exemplified by the performance tuning of a molecular dynamics application. In order to be easily extensible, the architecture of the system was based on components which are connected in a data flow graph to produce a given visualization tool. Innovative components were designed, in addition to “classical” components existing in similar visualization systems, to support scalability and interactivity.
---
paper_title: Performance Measurement, Visualization and Modeling of Parallel and Distributed Programs using the AIMS Toolkit

paper_content:

Writing large-scale parallel and distributed scientific applications that make optimum use of the multiprocessor is a challenging problem. Typically, computational resources are underused due to performance failures in the application being executed. Performance-tuning tools are essential for exposing these performance failures and for suggesting ways to improve program performance. In this paper, we first address fundamental issues in building useful performance-tuning tools and then describe our experience with the AIMS toolkit for tuning parallel and distributed programs on a variety of platforms. AIMS supports source-code instrumentation, run-time monitoring, graphical execution profiles, performance indices and automated modeling techniques as ways to expose performance problems of programs. Using several examples representing a broad range of scientific applications, we illustrate AIMS' effectiveness in exposing performance problems in parallel and distributed programs.
---
paper_title: The paradyn parallel performance measurement tools

paper_content:

Paradyn is a tool for measuring the performance of large-scale parallel programs. Our goal in designing a new performance tool was to provide detailed, flexible performance information without incurring the space (and time) overhead typically associated with trace-based tools. Paradyn achieves this goal by dynamically instrumenting the application and automatically controlling this instrumentation in search of performance problems. Dynamic instrumentation lets us defer insertion until the moment it is needed (and remove it when it is no longer needed); Paradyn's Performance Consultant decides when and where to insert instrumentation. >
---
paper_title: Performance analysis challenges and framework for high-performance reconfigurable computing

paper_content:

Reconfigurable computing (RC) applications employing both microprocessors and FPGAs have potential for large speedup when compared with traditional (software) parallel applications. However, this potential is marred by the additional complexity of these dual-paradigm systems, making it difficult to identify performance bottlenecks and achieve desired performance. Performance analysis concepts and tools are well researched and widely available for traditional parallel applications but are lacking in RC, despite being of great importance due to the applications' increased complexity. In this paper, we explore challenges and present new techniques in automated instrumentation, runtime measurement, and visualization of RC application behavior. We also present ideas for integration with conventional performance analysis tools to create a unified tool for RC applications as well as our initial framework for FPGA instrumentation and measurement. Results from a case study are provided using a prototype of this new tool.
---
paper_title: A visualization tool for analyzing cluster performance data

paper_content:

This paper describes a unique visualization tool that has been used to analyze performance of the Cplant(tm) clusters at Sandia National Laboratories.As commodity cluster systems grow in size and complexity, understanding performance issues becomes more and more difficult.We have developed a tool that facilitates visual performance analysis within the context of the physical and runtime environment of a system. Combining an abstract system model with color-coding for both performance and job information enables quick fault isolation as well as insight into complex system behavior.
---
paper_title: Cone Trees: animated 3D visualizations of hierarchical information

paper_content:

The task of managing and accessing large information spaces is a problem in large scale cognition. Enwrging technologies for 3D visualization and interactive aninlaiion offer potential solutions to this problenl, especially when the structure of the information can be visualized. We describe one of these Information Visualtzaiion techniques, called the Cone Tree, which is used for visualizing hierarchical information structures. The hierarchy is presented in 3D to nlaxilnize effective use of available screen space and enable visualization of the whole structure. Interactive aninlation is used (,o shift sonle of the user’s cognitive load to the human perceptual system,
---
paper_title: Tree-maps: a space-filling approach to the visualization of hierarchical information structures

paper_content:

A method for visualizing hierarchically structured information is described. The tree-map visualization technique makes 100% use of the available display space, mapping the full hierarchy onto a rectangular region in a space-filling manner. This efficient use of space allows very large hierarchies to be displayed in their entirety and facilitates the presentation of semantic information. Tree-maps can depict both the structure and content of the hierarchy. However, the approach is best suited to hierarchies in which the content of the leaf nodes and the structure of the hierarchy are of primary importance, and the content information associated with internal nodes is largely derived from their children.<<ETX>>
---
paper_title: Triva: Interactive 3D visualization for performance analysis of parallel applications

paper_content:

The successful execution of parallel applications in grid infrastructures depends directly on a performance analysis that takes into account the grid characteristics, such as the network topology and resources location. This paper presents Triva, a software analysis tool that implements a novel technique to visualize the behavior of parallel applications. The proposed technique explores 3D graphics to show the application behavior together with a description of the resources, highlighting communication patterns, the network topology and a visual representation of a logical organization of the resources. We have used a real grid infrastructure to execute and trace applications composed of thousands of processes.
---
paper_title: Treemaps for workload visualization

paper_content:

The author uses treemaps to present the state of work in a grid or other distributed computing environments, providing users with a view of overall performance and allowing performance analysts to explore data in a creative manner.
---
paper_title: Cone Trees: animated 3D visualizations of hierarchical information

paper_content:

The task of managing and accessing large information spaces is a problem in large scale cognition. Enwrging technologies for 3D visualization and interactive aninlaiion offer potential solutions to this problenl, especially when the structure of the information can be visualized. We describe one of these Information Visualtzaiion techniques, called the Cone Tree, which is used for visualizing hierarchical information structures. The hierarchy is presented in 3D to nlaxilnize effective use of available screen space and enable visualization of the whole structure. Interactive aninlation is used (,o shift sonle of the user’s cognitive load to the human perceptual system,
---
paper_title: Enterprise Network Monitoring Using Treemaps

paper_content:

Treemaps may provide significant advantages over tabular data in corporate enterprise applications, due to their inherent ability to support users' integration of multiple dimensions of information. This study investigated the usability of treemaps for enterprise system administrators who monitor servers and web applications. Manipulated factors included hierarchy representation, data scale, and comparison with unsorted tables. The treemap hierarchy representations differed significantly in their support of the identification, comparison, and analysis tasks, but were significantly faster and more accurate than tabular data views. Treemap learnability was at least as successful as for tables. Performance differences between treemaps and tables increased with increasing size of datasets. Users' subjective ratings overwhelmingly supported treemaps over tabular data views. These results suggest that treemaps should be included as a standard graphical component in enterprise-level data analysis and monitoring ...
---
paper_title: A performance visualization paradigm for data parallel computing

paper_content:

Observing the activities of a complex parallel computer system is no small feat, and relating these observations to program behavior is even harder. This paper presents a general measurement approach that is applicable to a large class of scalable programs and machines, specifically data parallel programs executing on distributed memory computer systems. The combined instrumentation and visualization paradigm, called VISTA (which stands for Visualization and Instrumentation of Scalable mulTicomputer Applications), is based on the author's experiences of programming and monitoring applications running on an nCUBE 2 computer and a MasPar MP-1 computer. The key is that performance data are treated similarly to any distributed data in the context of the data parallel programming model. Because of the data-parallel mapping of the program onto the machine, one can view the performance as it relates to each processor, processor cluster or processor ensemble and as it relates to the data structures of the program. The author illustrates the utility of VISTA by way of an example. >
---
paper_title: Integrated Visualization of Parallel Program Performance Data

paper_content:

Abstract Performance tuning a parallel application involves integrating performance data from many components of the system, including the message passing library, performance monitoring tool, resource manager, operating system, and the application itself. The current practice of visualizing these data streams using a separate, customized tool for each source is inconvenient from a usability perspective, and there is no easy way to visualize the data in an integrated fashion. We demonstrate a solution to this problem using Devise, a generic visualization tool which is designed to allow an arbitrary number of different but related data streams to be integrated and explored visually in a flexible manner. We display data emanating from a variety of sources side by side in three case studies. First we interface the Paradyn parallel performance tool and Devise, using two simple data export modules and Paradyn's simple visualization interface. We show several Devise/Paradyn visualizations which are useful for performance tuning parallel codes, and which incorporate data from Unix utilities and application output. Next we describe the visualization of trace data from a parallel application running in a Condor cluster of workstations. Finally we demonstrate the utility of Devise visualizations in a study of Condor cluster activity.
---
paper_title: Integrated Visualization of Parallel Program Performance Data

paper_content:

Abstract Performance tuning a parallel application involves integrating performance data from many components of the system, including the message passing library, performance monitoring tool, resource manager, operating system, and the application itself. The current practice of visualizing these data streams using a separate, customized tool for each source is inconvenient from a usability perspective, and there is no easy way to visualize the data in an integrated fashion. We demonstrate a solution to this problem using Devise, a generic visualization tool which is designed to allow an arbitrary number of different but related data streams to be integrated and explored visually in a flexible manner. We display data emanating from a variety of sources side by side in three case studies. First we interface the Paradyn parallel performance tool and Devise, using two simple data export modules and Paradyn's simple visualization interface. We show several Devise/Paradyn visualizations which are useful for performance tuning parallel codes, and which incorporate data from Unix utilities and application output. Next we describe the visualization of trace data from a parallel application running in a Condor cluster of workstations. Finally we demonstrate the utility of Devise visualizations in a study of Condor cluster activity.
---
paper_title: A visualization tool for analyzing cluster performance data

paper_content:

This paper describes a unique visualization tool that has been used to analyze performance of the Cplant(tm) clusters at Sandia National Laboratories.As commodity cluster systems grow in size and complexity, understanding performance issues becomes more and more difficult.We have developed a tool that facilitates visual performance analysis within the context of the physical and runtime environment of a system. Combining an abstract system model with color-coding for both performance and job information enables quick fault isolation as well as insight into complex system behavior.
---
paper_title: Pajé, an interactive visualization tool for tuning multi-threaded parallel applications

paper_content:

Abstract This paper describes Paje, an interactive visualization tool for displaying the execution of parallel applications where a potentially large number of communicating threads of various life-times execute on each node of a distributed memory parallel system. Paje is capable of representing a wide variety of interactions between threads. The main characteristics of Paje, interactivity and scalability, are exemplified by the performance tuning of a molecular dynamics application. In order to be easily extensible, the architecture of the system was based on components which are connected in a data flow graph to produce a given visualization tool. Innovative components were designed, in addition to “classical” components existing in similar visualization systems, to support scalability and interactivity.
---
paper_title: A taxonomy of program visualization systems

paper_content:

A taxonomy of program visualization systems that is based on a model of program visualization that maps programs to graphical representations is presented. The taxonomy is illustrated with three program visualization systems representative of research trends: Zeus, Tango, and Pavane. >
---
paper_title: A performance visualization paradigm for data parallel computing

paper_content:

Observing the activities of a complex parallel computer system is no small feat, and relating these observations to program behavior is even harder. This paper presents a general measurement approach that is applicable to a large class of scalable programs and machines, specifically data parallel programs executing on distributed memory computer systems. The combined instrumentation and visualization paradigm, called VISTA (which stands for Visualization and Instrumentation of Scalable mulTicomputer Applications), is based on the author's experiences of programming and monitoring applications running on an nCUBE 2 computer and a MasPar MP-1 computer. The key is that performance data are treated similarly to any distributed data in the context of the data parallel programming model. Because of the data-parallel mapping of the program onto the machine, one can view the performance as it relates to each processor, processor cluster or processor ensemble and as it relates to the data structures of the program. The author illustrates the utility of VISTA by way of an example. >
---
paper_title: Visualizing complex dynamics in many-core accelerator architectures

paper_content:

While many-core accelerator architectures, such as today's Graphics Processing Units (GPUs), offer orders of magnitude more raw computing power than contemporary CPUs, their massive parallelism often produces complex dynamic behaviors even with the simplest applications. Using a fixed set of hardware or simulator performance counters to quantify behavior over a large interval of time such as an entire application execution run or program phase may not capture this behavior. Software and/or hardware designers may consequently miss out on opportunities to optimize for better performance. Similarly, significant effort may be expended to find metrics that explain anomalous behavior in architecture design studies. Moreover, the increasing complexity of applications developed for today's GPU has created additional difficulties for software developers when attempting to identify bottlenecks of an application for optimization. This paper presents a novel GPU performance visualization tool, AerialVision, to address these two problems. It interfaces with the GPGPU-Sim simulator to capture and visualize the dynamic behavior of a GPU architecture throughout an application run. Similar to existing performance analysis tools for CPUs, it can annotate individual lines of source code with performance statistics to simplify the bottleneck identification process. To provide further insight, AerialVision introduces a novel methodology to relate pathological dynamic architectural behaviors resulting in performance loss with the part of the source code that is responsible. By rapidly providing insight into complex dynamic behavior, AerialVision enables research on improving many-core accelerator architectures and will help ensure applications written for these architectures reach their full performance potential.
---

Title: Performance Visualization for Large-Scale Computing Systems: A Literature Review

Section 1: Introduction
Description 1: Introduce the topic of performance visualization and its importance in large-scale computing systems, outlining the objectives and scope of the paper.

Section 2: Approach to Performance Visualization
Description 2: Discuss the general approach to performance visualization, including key steps such as instrumentation, measurement, data analysis, and visualization.

Section 3: Classification of Visualization Techniques
Description 3: Review various visualization techniques from 21 performance visualization systems, classified according to Card's framework of information visualization.

Section 4: Simple Visual Structures
Description 4: Describe the use of simple visual structures like statistical charts and diagrams in performance visualization.

Section 5: Macro-micro Composite View
Description 5: Explain micro-macro composition, where both global pictures and local details are provided in visualization.

Section 6: Timeline Views
Description 6: Discuss the use of timeline views to describe runtime behaviors and performance statistics over time.

Section 7: Information Typography
Description 7: Detail the concept of information typography, which uses 2D scatterplots and overlays to present performance data.

Section 8: Information Landscape
Description 8: Explain information landscapes, highlighting their use of the third spatial dimension and their application in performance visualization.

Section 9: Trees and Networks
Description 9: Discuss the use of trees and networks to visualize hierarchical information structures and networking relationships.

Section 10: Composed Structures
Description 10: Describe composed structures, which integrate multiple simple structures to provide a comprehensive view of performance data.

Section 11: Interactive Visual Structures
Description 11: Outline various interactive visual structures that allow user manipulation to customize and explore visualizations.

Section 12: Focus + Context Abstractions
Description 12: Explain the use (or lack) of focus + context abstractions in performance visualization systems, suggesting possible directions for future design.

Section 13: Future Work
Description 13: Summarize the key issues discussed, and suggest future research directions to address challenges in performance visualization for exascale computing systems.