A Survey of Embedded Software Profiling Methodologies
---
paper_title: Software Performance Estimation in MPSoC Design

paper_content:

Estimation tools are a key component of system-level methodologies, enabling a fast design space exploration. Estimation of software performance is essential in current software-dominated embedded systems. This work proposes an integrated methodology for system design and performance analysis. An analytic approach based on neural networks is used for high-level software performance estimation. At the functional level, this analytic tool enables a fast evaluation of the performance to be obtained with selected processors, which is an essential task for the definition of a "golden" architecture. From this architectural definition, a tool that refines hardware and software interfaces produces a bus-functional model. A virtual prototype is then generated from the bus-functional model, providing a global, cycle-accurate simulation model and offering several features for design validation and detailed performance analysis. Our work thus combines an analytic approach at functional level and a simulation-based approach at bus functional level. This provides an adequate trade-off between estimation time and precision. A multiprocessor platform implementing an MPEG4 encoder is used as case study, and the analytic estimation results in errors only up to 17% when compared to the virtual platform simulation. On the other hand, the analytic estimation takes only 17 seconds, against 10 minutes using the cycle-accurate simulation model.
---
paper_title: Fine-grained application source code profiling for ASIP design

paper_content:

Current Application Specific Instruction set Processor (ASIP) design methodologies are mostly based on iterative architecture exploration that uses Architecture Description Languages (ADLs) and retargetable software development tools. However, for improved design efficiency, additional pre-architecture exploration tools are required to help narrow-down the huge design space and making coarsegrained Instruction Set Architecture (ISA) decisions before detailed ADL modeling. Extensive application code profiling is the key in such early design stages. Based on a novel code instrumentation technology, we present a microprofiling approach that fills the current gap between source-level and instruction-level profilers and combines their advantages w.r.t. speed and accuracy. We show how the microprofiler is embedded into an advanced ASIP design flow and justify its use in a case study to design an MP3 decoder ASIP.
---
paper_title: gprof: a call graph execution profiler

paper_content:

We extended the UNIX® system's profiler by gathering arcs in the call graph of a program. Here is it 20 years later and this profiler is still in daily use. Why is that? It's not because there aren't well-known areas for improvement.Large complex programs are composed of many small routines that implement abstractions for the routines that call them. To be useful, an execution profiler must attribute execution time in a way that is significant for the logical structure of a program as well as for its textual decomposition. This data must then be displayed to the user in a convenient and informative way. The gprof profiler accounts for the running time of called routines in the running time of the routines that call them. The design and use of this profiler is described.
---
paper_title: Performance Data Collection: A Hybrid Approach

paper_content:

As the complexity of embedded software systems grows, performance profiling becomes more and more important. Performance profiling of embedded software systems requires data collection with low overhead and high information completeness. Performance profiling consists of monitoring a software system during execution and then analyzing the obtained data. There are two ways to collect profiling data: either event tracing through code instrumentation or statistical sampling. Event tracing may be more intrusive but allows the profiler to record all events of interest. Statistical sampling may be less intrusive to software system execution, but cannot provide complete execution information. Our position is that data collection on embedded software systems should be performed using a hybrid approach that combines the completeness of event tracing with the low cost of statistical sampling. The following sections expand this position.
---
paper_title: Profiling Tools for FPGA-Based Embedded Systems: Survey and Quantitative Comparison

paper_content:

Profiling tools are computer-aided design (CAD) tools that help in determining the computationally intensive portions in software. Embedded systems consist of hardware and software components that execute concurrently and efficiently to execute a specific task or application. Profiling tools are used by embedded system designers to choose computationally intensive functions for hardware implementation and acceleration. In this paper we review and compare various existing profiling tools for FPGA-based embedded systems. We then describe Airwolf, an FPGAbased profiling tool. We present a quantitative comparison of Airwolf and a well known software-based profiling tool, GNU gprof. Four software benchmarks were used to obtain profiling results using Airwolf and gprof. We show that Airwolf provides up to 66.2% improvement in accuracy of profiled results and reduces the run time performance overhead, caused by software-based profiling tools, by up to 41.3%. The results show that Airwolf provides accurate profiling results with minimal overhead and it can help the designers of FPGA-based embedded systems in identifying the computationally intensive portions of software code for hardware implementation and acceleration.
---
paper_title: Software Performance Estimation in MPSoC Design

paper_content:

Estimation tools are a key component of system-level methodologies, enabling a fast design space exploration. Estimation of software performance is essential in current software-dominated embedded systems. This work proposes an integrated methodology for system design and performance analysis. An analytic approach based on neural networks is used for high-level software performance estimation. At the functional level, this analytic tool enables a fast evaluation of the performance to be obtained with selected processors, which is an essential task for the definition of a "golden" architecture. From this architectural definition, a tool that refines hardware and software interfaces produces a bus-functional model. A virtual prototype is then generated from the bus-functional model, providing a global, cycle-accurate simulation model and offering several features for design validation and detailed performance analysis. Our work thus combines an analytic approach at functional level and a simulation-based approach at bus functional level. This provides an adequate trade-off between estimation time and precision. A multiprocessor platform implementing an MPEG4 encoder is used as case study, and the analytic estimation results in errors only up to 17% when compared to the virtual platform simulation. On the other hand, the analytic estimation takes only 17 seconds, against 10 minutes using the cycle-accurate simulation model.
---
paper_title: Applying neural networks to performance estimation of embedded software

paper_content:

High-level performance estimation of embedded software implemented in a particular processor is essential for a fast design space exploration, when the designer needs to evaluate different processor architectures (and their different versions) and also different task allocations in a multiprocessor system. The development of fast and adequate performance estimators is required to achieve the necessary speed in this design phase. However, advanced architectures present many features, such as pipelines, branch prediction mechanisms, and caches, which have a non-linear impact on the execution time, which thus becomes hard to evaluate using simple linear methods. In order to cope with this problem, this paper presents a high-level performance estimator based on a neural network, which easily adapts to the non-linear behaviour of the execution time in advanced architectures and presents a speed-up up to 190 times in comparison with cycle-accurate simulators, using the PowerPC 750 as target architecture. A method for automatic domain classification is proposed to group applications with similar characteristics, resulting in an increase of the estimation precision. For the PowerPC 750, the mean estimation error has been reduced from 7.90% to 6.41% thanks to domain-specific estimators. This precision level and the fast estimation time are suitable for high-level design space exploration, when different architectures or processor versions and different task allocations need to be evaluated in a fast way.
---
paper_title: Applying neural networks to performance estimation of embedded software

paper_content:

High-level performance estimation of embedded software implemented in a particular processor is essential for a fast design space exploration, when the designer needs to evaluate different processor architectures (and their different versions) and also different task allocations in a multiprocessor system. The development of fast and adequate performance estimators is required to achieve the necessary speed in this design phase. However, advanced architectures present many features, such as pipelines, branch prediction mechanisms, and caches, which have a non-linear impact on the execution time, which thus becomes hard to evaluate using simple linear methods. In order to cope with this problem, this paper presents a high-level performance estimator based on a neural network, which easily adapts to the non-linear behaviour of the execution time in advanced architectures and presents a speed-up up to 190 times in comparison with cycle-accurate simulators, using the PowerPC 750 as target architecture. A method for automatic domain classification is proposed to group applications with similar characteristics, resulting in an increase of the estimation precision. For the PowerPC 750, the mean estimation error has been reduced from 7.90% to 6.41% thanks to domain-specific estimators. This precision level and the fast estimation time are suitable for high-level design space exploration, when different architectures or processor versions and different task allocations need to be evaluated in a fast way.
---
paper_title: Multiprocessor System-on-Chip Profiling Architecture: Design and Implementation

paper_content:

With the growing needs for advanced functionalities in modern embedded systems, it is now necessary to integrate multiple processors in the system, preferably on a single chip, to support the required computing complexity. The problem is that such multiprocessor system-on-chip (MPSoC) architecture is very complex and its internal behavior is very difficult to track. An effective tool for profiling the behavior of the MPSoC system is in great need. Such a tool is very useful during system design for exploiting various options and identifying potential bottlenecks. In this paper, we introduce the MultiProcessor Profiling Architecture (MPPA) -- a general framework for profiling MPSoC embedded systems. The MPPA framework entails the use of FPGA emulation for the target system, the embedding of performance counters for recording system events, and the development of OS drivers for collecting the profiled data. To demonstrate its use, we show the implementation of an MPSoC emulation system based on Leon3 cores following the MPPA framework. We also show how the MPPA framework and the emulator help the designers to identify performance problems and improve their MPSoC embedded system design.
---
paper_title: Performance Data Collection: A Hybrid Approach

paper_content:

As the complexity of embedded software systems grows, performance profiling becomes more and more important. Performance profiling of embedded software systems requires data collection with low overhead and high information completeness. Performance profiling consists of monitoring a software system during execution and then analyzing the obtained data. There are two ways to collect profiling data: either event tracing through code instrumentation or statistical sampling. Event tracing may be more intrusive but allows the profiler to record all events of interest. Statistical sampling may be less intrusive to software system execution, but cannot provide complete execution information. Our position is that data collection on embedded software systems should be performed using a hybrid approach that combines the completeness of event tracing with the low cost of statistical sampling. The following sections expand this position.
---
paper_title: Profiling Tools for FPGA-Based Embedded Systems: Survey and Quantitative Comparison

paper_content:

Profiling tools are computer-aided design (CAD) tools that help in determining the computationally intensive portions in software. Embedded systems consist of hardware and software components that execute concurrently and efficiently to execute a specific task or application. Profiling tools are used by embedded system designers to choose computationally intensive functions for hardware implementation and acceleration. In this paper we review and compare various existing profiling tools for FPGA-based embedded systems. We then describe Airwolf, an FPGAbased profiling tool. We present a quantitative comparison of Airwolf and a well known software-based profiling tool, GNU gprof. Four software benchmarks were used to obtain profiling results using Airwolf and gprof. We show that Airwolf provides up to 66.2% improvement in accuracy of profiled results and reduces the run time performance overhead, caused by software-based profiling tools, by up to 41.3%. The results show that Airwolf provides accurate profiling results with minimal overhead and it can help the designers of FPGA-based embedded systems in identifying the computationally intensive portions of software code for hardware implementation and acceleration.
---
paper_title: Software Performance Estimation in MPSoC Design

paper_content:

Estimation tools are a key component of system-level methodologies, enabling a fast design space exploration. Estimation of software performance is essential in current software-dominated embedded systems. This work proposes an integrated methodology for system design and performance analysis. An analytic approach based on neural networks is used for high-level software performance estimation. At the functional level, this analytic tool enables a fast evaluation of the performance to be obtained with selected processors, which is an essential task for the definition of a "golden" architecture. From this architectural definition, a tool that refines hardware and software interfaces produces a bus-functional model. A virtual prototype is then generated from the bus-functional model, providing a global, cycle-accurate simulation model and offering several features for design validation and detailed performance analysis. Our work thus combines an analytic approach at functional level and a simulation-based approach at bus functional level. This provides an adequate trade-off between estimation time and precision. A multiprocessor platform implementing an MPEG4 encoder is used as case study, and the analytic estimation results in errors only up to 17% when compared to the virtual platform simulation. On the other hand, the analytic estimation takes only 17 seconds, against 10 minutes using the cycle-accurate simulation model.
---
paper_title: Applying neural networks to performance estimation of embedded software

paper_content:

High-level performance estimation of embedded software implemented in a particular processor is essential for a fast design space exploration, when the designer needs to evaluate different processor architectures (and their different versions) and also different task allocations in a multiprocessor system. The development of fast and adequate performance estimators is required to achieve the necessary speed in this design phase. However, advanced architectures present many features, such as pipelines, branch prediction mechanisms, and caches, which have a non-linear impact on the execution time, which thus becomes hard to evaluate using simple linear methods. In order to cope with this problem, this paper presents a high-level performance estimator based on a neural network, which easily adapts to the non-linear behaviour of the execution time in advanced architectures and presents a speed-up up to 190 times in comparison with cycle-accurate simulators, using the PowerPC 750 as target architecture. A method for automatic domain classification is proposed to group applications with similar characteristics, resulting in an increase of the estimation precision. For the PowerPC 750, the mean estimation error has been reduced from 7.90% to 6.41% thanks to domain-specific estimators. This precision level and the fast estimation time are suitable for high-level design space exploration, when different architectures or processor versions and different task allocations need to be evaluated in a fast way.
---
paper_title: gprof: a call graph execution profiler

paper_content:

We extended the UNIX® system's profiler by gathering arcs in the call graph of a program. Here is it 20 years later and this profiler is still in daily use. Why is that? It's not because there aren't well-known areas for improvement.Large complex programs are composed of many small routines that implement abstractions for the routines that call them. To be useful, an execution profiler must attribute execution time in a way that is significant for the logical structure of a program as well as for its textual decomposition. This data must then be displayed to the user in a convenient and informative way. The gprof profiler accounts for the running time of called routines in the running time of the routines that call them. The design and use of this profiler is described.
---
paper_title: Using reconfigurability to achieve real-time profiling for hardware/software codesign

paper_content:

Embedded systems combine a processor with dedicated logic to meet design specifications at a reasonable cost. The attempt to amalgamate two distinct design environments introduces many problems, one being how to partition a single design for the two platforms to achieve the best performance with the least effort. Since the latest FPGA technology allows the integration of soft or hard CPU cores with dedicated logic on a single chip, this presents new opportunities for addressing hardware/software codesign issues in the FPGA design process by utilizing the reconfigurable environment.This paper introduces SnoopP, a non-intrusive, real time, profiling tool. The user is able to obtain a clock cycle accurate profile of the real time performance of a software program running on a soft-core processor instantiated on an FPGA. SnoopP is an essential tool for hardware/software codesign on a reconfigurable platform. It allows the user to quickly obtain accurate profiling information that may greatly influence the partitioning of the design.
---
paper_title: Profiling Tools for FPGA-Based Embedded Systems: Survey and Quantitative Comparison

paper_content:

Profiling tools are computer-aided design (CAD) tools that help in determining the computationally intensive portions in software. Embedded systems consist of hardware and software components that execute concurrently and efficiently to execute a specific task or application. Profiling tools are used by embedded system designers to choose computationally intensive functions for hardware implementation and acceleration. In this paper we review and compare various existing profiling tools for FPGA-based embedded systems. We then describe Airwolf, an FPGAbased profiling tool. We present a quantitative comparison of Airwolf and a well known software-based profiling tool, GNU gprof. Four software benchmarks were used to obtain profiling results using Airwolf and gprof. We show that Airwolf provides up to 66.2% improvement in accuracy of profiled results and reduces the run time performance overhead, caused by software-based profiling tools, by up to 41.3%. The results show that Airwolf provides accurate profiling results with minimal overhead and it can help the designers of FPGA-based embedded systems in identifying the computationally intensive portions of software code for hardware implementation and acceleration.
---
paper_title: FPGA-based software profiler for Hardware/Software co-design

paper_content:

Embedded systems are a mixture of software running on a microprocessor and application-specific hardware. Hardware/Software co-design requires an appropriate profiler to detect the functions that contribute to a large percentage of program execution. Software based profiling tools, such as the well-known GNU gprof profiler, integrates an extra code with the software program to be profiled causing a significant performance overhead. To address this issue, this paper proposes a software profiler called AddressTracer. This profiler is an adaptation of a non-intrusive, real time profiler called SnoopP. The AddressTracer is accurately able to evaluate the performance matrices of any specific software function. A software benchmark, Secure Hash Algorithm (SHA), is profiled using AddressTracer and other software profiling tools, Airwolf, and GNU software profiling tool (gprof), for a quantitative comparison and their performance overhead are studied. The achieved results show that AddressTracer provides accurate profiling results with no performance overhead. Airwolf causes a very low remarkable performance overhead compared with that incurred by gprof.
---
paper_title: Frequent loop detection using efficient nonintrusive on-chip hardware

paper_content:

Dynamic software optimization methods are becoming increasingly popular for improving software performance and power. The first step in dynamic optimization consists of detecting frequently executed code, or "critical regions." Most previous critical region detectors have been targeted to desktop processors. We introduce a critical region detector targeted to embedded processors, with the unique features of being very size and power efficient and being completely nonintrusive to the software's execution-features needed in timing-sensitive embedded systems. Our detector not only finds the critical regions, but also determines their relative frequencies, a potentially important feature for selecting among alternative dynamic optimization methods. Our detector uses a tiny cache-like structure coupled with a small amount of logic. We provide results of extensive explorations across 19 embedded system benchmarks. We show that highly accurate results can be achieved with only a 0.02 percent power overhead, acceptable size overhead; and zero runtime overhead. Our detector is currently being used as part of a dynamic hardware/software partitioning approach, but is applicable to a wide variety of situations.
---
paper_title: Non-intrusive dynamic application profiling for multitasked applications

paper_content:

Application profiling -- the process of monitoring an application to determine the frequency of execution within specific regions -- is an essential step within the design process for many software and hardware systems. Profiling is often a critical step within hardware/software partitioning utilized to determine the critical kernels of an application. In this paper, we present a non-intrusive dynamic application profiler (DAProf) capable of profiling an executing application by monitoring the application's short backwards branches, function calls, function returns, as well as efficiently detecting context switches to provide accurate characterization of the frequently executed loops within multitasked applications. DAProf can accurately profile multiple tasks within a software application with 98.5% accuracy using as little as 10% additional area compared to an ARM9 processor.
---
paper_title: Non-intrusive dynamic application profiler for detailed loop execution characterization

paper_content:

Application profiling - the process of monitoring an application to determine the frequency of execution within specific regions - is an essential step within the design process for many software and hardware systems. In this paper, we present an efficient innovative, non-intrusive dynamic application profiler (DAProf) capable of profiling an executing application by monitoring the application's short backwards branches and providing detailed profiling statistics for characterizing loop execution behavior. DAProf is ideally suited for hardware/software partitioning approaches in which detailed loop execution information is needed to provide accurate performance estimates. DAProf provides a profiling accuracy of greater than 90% with only an 11% area overhead compared to a small ARM9.
---
paper_title: Efficient hardware-based nonintrusive dynamic application profiling

paper_content:

Application profiling—the process of monitoring an application to determine the frequency of execution within specific regions—is an essential step within the design process for many software and hardware systems. Profiling is often a critical step within hardware/software partitioning utilized to determine the critical kernels of an application. In this article, we present an innovative, nonintrusive dynamic application profiler (DAProf) capable of profiling an executing application by monitoring the application's short backward branches, function calls, and function returns. The resulting profile information provides an accurate characterization of the frequently executed loops within the application providing a breakdown of loop executions versus loop iterations per execution. DAProf achieves excellent profiling accuracy with an average accuracy of 98p for loop executions, 97p for average iterations per execution, and 95p for percentage of execution time. In addition, the presented dynamic application profiler incurs as little as 11p area overhead compared to an ARM9 microprocessor. DAProf is ideally suited for rapidly profiling software applications and dynamic optimization approaches such as dynamic hardware/software partitioning in which detailed loop execution information is needed to provide accurate performance estimates.
---
paper_title: Multiprocessor System-on-Chip Profiling Architecture: Design and Implementation

paper_content:

With the growing needs for advanced functionalities in modern embedded systems, it is now necessary to integrate multiple processors in the system, preferably on a single chip, to support the required computing complexity. The problem is that such multiprocessor system-on-chip (MPSoC) architecture is very complex and its internal behavior is very difficult to track. An effective tool for profiling the behavior of the MPSoC system is in great need. Such a tool is very useful during system design for exploiting various options and identifying potential bottlenecks. In this paper, we introduce the MultiProcessor Profiling Architecture (MPPA) -- a general framework for profiling MPSoC embedded systems. The MPPA framework entails the use of FPGA emulation for the target system, the embedding of performance counters for recording system events, and the development of OS drivers for collecting the profiled data. To demonstrate its use, we show the implementation of an MPSoC emulation system based on Leon3 cores following the MPPA framework. We also show how the MPPA framework and the emulator help the designers to identify performance problems and improve their MPSoC embedded system design.
---
paper_title: Maximizing system performance: using reconfigurability to monitor system communications

paper_content:

Commercial FPGA companies now provide tools that allow users to implement designs comprising soft-core processors and modules of dedicated logic. If a designer chooses to partition a system into multiple processors and hardware modules, tools and techniques for design analysis are necessary to understand system performance. This work introduces WOoDSTOCK, a tool that profiles system performance by adding monitors to the circuit running in real time on the chip. The user is able to generate a system specific profiler tailored to monitor the communication links between the different computing elements. This provides a macroscopic picture of system performance, which highlights the computing elements that cause bottlenecks in the design.
---
paper_title: Fine-grained application source code profiling for ASIP design

paper_content:

Current Application Specific Instruction set Processor (ASIP) design methodologies are mostly based on iterative architecture exploration that uses Architecture Description Languages (ADLs) and retargetable software development tools. However, for improved design efficiency, additional pre-architecture exploration tools are required to help narrow-down the huge design space and making coarsegrained Instruction Set Architecture (ISA) decisions before detailed ADL modeling. Extensive application code profiling is the key in such early design stages. Based on a novel code instrumentation technology, we present a microprofiling approach that fills the current gap between source-level and instruction-level profilers and combines their advantages w.r.t. speed and accuracy. We show how the microprofiler is embedded into an advanced ASIP design flow and justify its use in a case study to design an MP3 decoder ASIP.
---
paper_title: Software Performance Estimation in MPSoC Design

paper_content:

Estimation tools are a key component of system-level methodologies, enabling a fast design space exploration. Estimation of software performance is essential in current software-dominated embedded systems. This work proposes an integrated methodology for system design and performance analysis. An analytic approach based on neural networks is used for high-level software performance estimation. At the functional level, this analytic tool enables a fast evaluation of the performance to be obtained with selected processors, which is an essential task for the definition of a "golden" architecture. From this architectural definition, a tool that refines hardware and software interfaces produces a bus-functional model. A virtual prototype is then generated from the bus-functional model, providing a global, cycle-accurate simulation model and offering several features for design validation and detailed performance analysis. Our work thus combines an analytic approach at functional level and a simulation-based approach at bus functional level. This provides an adequate trade-off between estimation time and precision. A multiprocessor platform implementing an MPEG4 encoder is used as case study, and the analytic estimation results in errors only up to 17% when compared to the virtual platform simulation. On the other hand, the analytic estimation takes only 17 seconds, against 10 minutes using the cycle-accurate simulation model.
---

Title: A Survey of Embedded Software Profiling Methodologies

Section 1: INTRODUCTION
Description 1: In this section, provide an overview of the importance of design space exploration in embedded systems and introduce the need for software profiling tools, including a brief comparison between simulation and profiling.

Section 2: CLASSIFICATION OF EMBEDDED SOFTWARE PROFILING TECHNIQUES
Description 2: Identify common principles in existing embedded software profiling methodologies and develop a comprehensive classification for such approaches.

Section 3: Embedded Software Profiling Approaches
Description 3: Present various profiling approaches that have been explored by researchers in academic and industrial fields, with a focus on the methodologies used for embedded software performance estimation.

Section 4: Discussion
Description 4: Summarize and compare the different profiling methodologies based on key classification criteria introduced in Section II, highlighting the accuracy, overhead, and scope of each approach.

Section 5: Conclusion
Description 5: Conclude the survey by summarizing the key insights from the comparison of profiling methodologies and suggesting new research directions based on the limitations identified in the discussion.