Performance evaluation of component-based software systems: A survey
---
paper_title: Nieuwenhuis Performance comparison of middleware threading strategies

paper_content:

The spectacular growth of E-business applications on the Internet has boosted the development of middleware technology. Middleware is software that manages interactions between applications distributed across a heterogeneous computing environment. In the competitive E-business market the ability to deliver a high and predictable performance of E-business applications is crucial to avoid customer churn, and thus loss of revenue. This raises the need for service providers to be able to predict and control performance. The performance of middleware-based applications depends strongly on the choice of the so-called threading strategy, describing how the middleware layer handles competing method invocation requests. The goal of this paper is to provide an understanding of the impact of threading strategies on the performance of middlewarebased applications. To this end, we (1) develop new quantitative models for the performance of middleware under different threading strategies, (2) perform extensive test lab experiments to compare the performance under different threading strategies, and (3) explain the experimental results by relating them to the quantitative models. As such, this paper provides new and fundamental insight in the impact of threading strategies on the performance of E-business applications.
---
paper_title: The Future of Software Performance Engineering

paper_content:

Performance is a pervasive quality of software systems; everything affects it, from the software itself to all underlying layers, such as operating system, middleware, hardware, communication networks, etc. Software Performance Engineering encompasses efforts to describe and improve performance, with two distinct approaches: an early-cycle predictive model-based approach, and a late-cycle measurement-based approach. Current progress and future trends within these two approaches are described, with a tendency (and a need) for them to converge, in order to cover the entire development cycle.
---
paper_title: Making the Business Case for Software Performance Engineering

paper_content:

Shrinking budgets and increased fiscal accountability mean that management needs a sound financial justification before committing funds to software process improvements such as Software Performance Engineering (SPE). Preparing a business case for SPE can demonstrate that the commitment is financially worthwhile and win support for an SPE initiative. This paper presents an introduction to the use of business case analysis to justify investing in SPE to reduce costs due to performance failures. A case study illustrates how to perform a financial analysis and calculate a projected return on investment.
---
paper_title: Process algebra for performance evaluation

paper_content:

This paper surveys the theoretical developments in the field of stochastic process algebras, process algebras where action occurrences may be subject to a delay that is determined by a random variable. A huge class of resource-sharing systems - like large-scale computers, client-server architectures, networks - can accurately be described using such stochastic specification formalisms. The main emphasis of this paper is the treatment of operational semantics, notions of equivalence, and (sound and complete) axiomatisations of these equivalences for different types of Markovian process algebras, where delays are governed by exponential distributions. Starting from a simple actionless algebra for describing time-homogeneous continuous-time Markov chains, we consider the integration of actions and random delays both as a single entity (like in known Markovian process algebras like TIPP, PEPA and EMPA) and as separate entities (like in the timed process algebras timed CSP and TCCS). In total we consider four related calculi and investigate their relationship to existing Markovian process algebras. We also briefly indicate how one can profit from the separation of time and actions when incorporating more general, non-Markovian distributions.
---
paper_title: Model-based performance prediction in software development: a survey

paper_content:

Over the last decade, a lot of research has been directed toward integrating performance analysis into the software development process. Traditional software development methods focus on software correctness, introducing performance issues later in the development process. This approach does not take into account the fact that performance problems may require considerable changes in design, for example, at the software architecture level, or even worse at the requirement analysis level. Several approaches were proposed in order to address early software performance analysis. Although some of them have been successfully applied, we are still far from seeing performance analysis integrated into ordinary software development. In this paper, we present a comprehensive review of recent research in the field of model-based performance prediction at software development time in order to assess the maturity of the field and point out promising research directions.
---
paper_title: Component Software: Beyond Object-Oriented Programming

paper_content:

From the Publisher: ::: Component Software: Beyond Object-Oriented Programming explains the technical foundations of this evolving technology and its importance in the software market place. It provides in-depth discussion of both the technical and the business issues to be considered, then moves on to suggest approaches for implementing component-oriented software production and the organizational requirements for success. The author draws on his own experience to offer tried-and-tested solutions to common problems and novel approaches to potential pitfalls. Anyone responsible for developing software strategy, evaluating new technologies, buying or building software will find Clemens Szyperski's objective and market-aware perspective of this new area invaluable.
---
paper_title: SAAM: a method for analyzing the properties of software architectures

paper_content:

While software architecture has become an increasingly important research topic in recent years, insufficient attention has been paid to methods for evaluation of these architectures. Evaluating architectures is difficult for two main reasons. First, there is no common language used to describe different architectures. Second, there is no clear way of understanding an architecture with respect to an organization's life cycle concerns -software quality concerns such as maintainability portability, modularity, reusability, and so forth. We address these shortcomings by describing three perspectives by which we can understand the description of a software architecture and then proposing a five-step method for analyzing software architectures called SAAM (Software Architecture Analysis Method). We illustrate the method by analyzing three separate user interface architectures with respect to the quality of modifiability. >
---
paper_title: Stochastic Petri Nets An Introduction To The Theory

paper_content:

Thank you very much for reading stochastic petri nets an introduction to the theory. Maybe you have knowledge that, people have look hundreds times for their favorite novels like this stochastic petri nets an introduction to the theory, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they cope with some malicious bugs inside their computer.
---
paper_title: ATAM: Method for Architecture Evaluation

paper_content:

Abstract : If a software architecture is a key business asset for an organization, then architectural analysis must also be a key practice for that organization. Why? Because architectures are complex and involve many design tradeoffs. Without undertaking a formal analysis process, the organization cannot ensure that the architectural decisions made-particularly those which affect the achievement of quality attribute such as performance, availability, security, and modifiability-are advisable ones that appropriately mitigate risks. In this report, some of the technical and organizational foundations for performing architectural analysis are discussed, the Architecture Tradeoff Analysis Method (ATAM), is presented. The ATAM is a technique for analyzing software architectures that has been developed and refined in practice over the past three years.
---
paper_title: Performance Modeling and Evaluation of Distributed Component-Based Systems Using Queueing Petri Nets

paper_content:

Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how queueing Petri net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of queueing Petri nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits
---
paper_title: Performance prediction of component-based systems : A survey from an engineering perspective

paper_content:

Performance predictions of component assemblies and the ability of obtaining system-level performance properties from these predictions are a cru- cial success factor when building trustworthy component-based systems. In order to achieve this goal, a collection of methods and tools to capture and analyze the performance of software systems has been developed. These methods and tools aim at helping software engineers by providing them with the capability to understand design trade-offs, optimize their design by identifying performance inhibitors, or predict a systems performance within a specified deployment envi- ronment. In this paper, we analyze the applicability of various performance pre- diction methods for the development of component-based systems and contrast their inherent strengths and weaknesses in different engineering problem scenar- ios. In so doing, we establish a basis to select an appropriate prediction method and to provide recommendations for future research activities, which could sig- nificantly improve the performance prediction of component-based systems.
---
paper_title: Model-Driven Software Development: Technology, Engineering, Management

paper_content:

Part I: Introduction. 1. Introduction. 2. MDSD - Basic Ideas and Terminology. 3. Case Study: A Typical Web Application. 4. Concept Formation. 5. Classification. Part II: Domain Architectures. 6. Metamodeling. 7. MDSD-Capable Target Architectures. 8. Building Domain Architectures. 9. Code Generation Techniques. 10. Model Transformation Techniques. 11. MDSD Tools: Roles, Architecture, Selection Criteria, and Pointers. 12. The MDA Standard. Part III: Processes and Engineering. 13. MDSD Process Building Blocks and Best Practices. 14. Testing. 15. Versioning. 16. Case Study: Embedded Component Infrastructures. 17. Case Study: An Enterprise System. Part IV: Management. 18. Decision Support. 1.9 Organizational Aspects. 20. Adoption Strategies for MDSD. References. Index.
---
paper_title: Software component models

paper_content:

Component-based Development (CBD) is an important emerging topic in Software Engineering, promising long sought after benefits like increased reuse and reduced time-to-market (and hence software production cost). However, there are at present many obstacles to overcome before CBD can succeed. For one thing, CBD success is predicated on a standardised market place for software components, which does not yet exist. In fact currently CBD even lacks a universally accepted terminology. Existing component models adopt different component definitions and composition operators. Therefore much research remains to be done. We believe that the starting point for this endeavour should be a thorough study of current component models, identifying their key characteristics and comparing their strengths and weaknesses. A desirable side-effect would be clarifying and unifying the CBD terminology. In this tutorial, we present a clear and concise exposition of all the current major software component models, including a taxonomy. The purpose is to distill and present knowledge of current software component models, as well as to present an analysis of their properties with respect to commonly accepted criteria for CBD. The taxonomy also provides a starting point for a unified terminology.
---
paper_title: Component Software: Beyond Object-Oriented Programming

paper_content:

From the Publisher: ::: Component Software: Beyond Object-Oriented Programming explains the technical foundations of this evolving technology and its importance in the software market place. It provides in-depth discussion of both the technical and the business issues to be considered, then moves on to suggest approaches for implementing component-oriented software production and the organizational requirements for success. The author draws on his own experience to offer tried-and-tested solutions to common problems and novel approaches to potential pitfalls. Anyone responsible for developing software strategy, evaluating new technologies, buying or building software will find Clemens Szyperski's objective and market-aware perspective of this new area invaluable.
---
paper_title: Performance Solutions A Practical Guide To Creating Responsive Scalable Software

paper_content:

Thank you for downloading performance solutions a practical guide to creating responsive scalable software. As you may know, people have search numerous times for their favorite books like this performance solutions a practical guide to creating responsive scalable software, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they cope with some malicious bugs inside their laptop.
---
paper_title: Model-based performance prediction in software development: a survey

paper_content:

Over the last decade, a lot of research has been directed toward integrating performance analysis into the software development process. Traditional software development methods focus on software correctness, introducing performance issues later in the development process. This approach does not take into account the fact that performance problems may require considerable changes in design, for example, at the software architecture level, or even worse at the requirement analysis level. Several approaches were proposed in order to address early software performance analysis. Although some of them have been successfully applied, we are still far from seeing performance analysis integrated into ordinary software development. In this paper, we present a comprehensive review of recent research in the field of model-based performance prediction at software development time in order to assess the maturity of the field and point out promising research directions.
---
paper_title: Component Software: Beyond Object-Oriented Programming

paper_content:

From the Publisher: ::: Component Software: Beyond Object-Oriented Programming explains the technical foundations of this evolving technology and its importance in the software market place. It provides in-depth discussion of both the technical and the business issues to be considered, then moves on to suggest approaches for implementing component-oriented software production and the organizational requirements for success. The author draws on his own experience to offer tried-and-tested solutions to common problems and novel approaches to potential pitfalls. Anyone responsible for developing software strategy, evaluating new technologies, buying or building software will find Clemens Szyperski's objective and market-aware perspective of this new area invaluable.
---
paper_title: Towards Automatic Construction of Reusable Prediction Models for Component-Based Performance Engineering

paper_content:

Performance predictions for software architectures can reveal performance bottlenecks and quantitatively support design decisions for different architectural alternatives. As software architects aim at reusing existing software components, their performance properties should be included into performance predictions without the need for manual modelling. However, most prediction approaches do not include automated support for modelling implemented components. Therefore, we propose a new reverse engineering approach, which generates Palladio performance models from Java code. In this paper, we focus on the static analysis of Java code, which we have implemented as an Eclipse plugin called Java2PCM. We evaluated our approach on a larger component-based software architecture, and show that a similar prediction accuracy can be achieved with generated models compared to completely manually specified ones.
---
paper_title: UML-Based Performance Modeling Framework for Component-Based Distributed Systems

paper_content:

We describe a performance modeling framework that can be used in the development and maintenance of component-based distributed systems, such as those based on the CORBA, EJB, and COM+ platforms. The purpose of the framework is to produce predictive performance models that can be used for obtaining performance related information on the target system at all stages of its life cycle. The framework defines a UML-based notation for describing performance models, and a set of special techniques for modeling component-based distributed systems. In addition, we present a transformation for converting the resulting models into a format that can be solved approximately for a number of relevant performance metrics.
---
paper_title: Model-based performance prediction in software development: a survey

paper_content:

Over the last decade, a lot of research has been directed toward integrating performance analysis into the software development process. Traditional software development methods focus on software correctness, introducing performance issues later in the development process. This approach does not take into account the fact that performance problems may require considerable changes in design, for example, at the software architecture level, or even worse at the requirement analysis level. Several approaches were proposed in order to address early software performance analysis. Although some of them have been successfully applied, we are still far from seeing performance analysis integrated into ordinary software development. In this paper, we present a comprehensive review of recent research in the field of model-based performance prediction at software development time in order to assess the maturity of the field and point out promising research directions.
---
paper_title: Efficient Performance Models in Component-Based Software Engineering

paper_content:

Performance evaluation of Component-Based software systems should be performed as early as possible during the software development life cycle. Unfortunately, a detailed quantitative analysis is often not possible during such stages, as only the system outline is available, with very little quantitative knowledge. In this paper we propose an approach based on Queueing Network analysis for performance evaluation of component-based software systems at the software architectural level. Our approach provides performance bounds which can be efficiently computed. Starting from annotated UML diagrams we compute bounds on the system throughput and response time without explicitly deriving or solving the underlying multichain and multiclass Queueing Network model. We illustrate with an example how the technique can be applied to answer many performance-related questions which may arise during the software design phase.
---
paper_title: CB-SPE Tool: Putting component-based performance engineering into practice

paper_content:

A crucial issue in the design of Component-Based (CB) applications is the ability to early guarantee that the system under development will satisfy its Quality of Service requirements. In particular, we need rigorous and easy-to-use techniques for predicting and analyzing the performance of the assembly based on the properties of the constituent components. To this purpose, we propose the CB-SPE framework: a compositional methodology for CB Software Performance Engineering (SPE) and its supporting tool. CB-SPE is based on, and adapts to a CB paradigm, the concepts and steps of the well-known SPE technology, using for input modeling the standard RT-UML PA profile. The methodology is compositional: it is first applied by the component developer at the component layer, achieving a parametric performance evaluation of the components in isolation; then, at the application layer, the system assembler is provided with a step-wise procedure for predicting the performance of the assembled components on the actual platform. We have developed the CB-SPE tool reusing as much as possible existing free tools. In this paper we present the realized framework, together with a simple application example.
---
paper_title: Performance modeling from software components

paper_content:

When software products are assembled from pre-defined components, performance prediction should be based on the components also. This supports rapid model-building, using previously calibrated sub-models or "performance components", in sync with the construction of the product. The specification of a performance component must be tied closely to the software component specification, but it also includes performance related parameters (describing workload characteristics and demands), and it abstracts the behaviour of the component in various ways (for reasons related to practical factors in performance analysis). A useful set of abstractions and parameters are already defined for layered performance modeling. This work extends them to accommodate software components, using a new XML-based language called Component-Based Modeling Language (CBML). With CBML, compatible components can be inserted into slots provided in a hierarchical component specification based on the UML component model.
---
paper_title: Model-Based performance prediction with the palladio component model

paper_content:

One aim of component-based software engineering (CBSE) is to enable the prediction of extra-functional properties, such as performance and reliability, utilising a well-defined composition theory. Nowadays, such theories and their accompanying prediction methods are still in a maturation stage. Several factors influencing extra-functional properties need additional research to be understood. A special problem in CBSE stems from its specific development process: Software components should be specified and implemented independent from their later context to enable reuse. Thus, extra-functional properties of components need to be specified in a parametric way to take different influence factors like the hardware platform or the usage profile into account. In our approach, we use the Palladio Component Model (PCM) to specify component-based software architectures in a parametric way. This model offers direct support of the CBSE development process by dividing the model creation among the developer roles. In this paper, we present our model and a simulation tool based on it, which is capable of making performance predictions. Within a case study, we show that the resulting prediction accuracy can be sufficient to support the evaluation of architectural design decisions.
---
paper_title: Performance modeling and prediction of enterprise JavaBeans with layered queuing network templates

paper_content:

Component technologies, such as Enterprise Java Beans (EJB) and .NET, are used in enterprise servers with requirements for high performance and scalability. This work considers performance prediction from the design of an EJB system, based on the modular structure of an application server and the application components. It uses layered queueing models, which are naturally structured around the software components. This paper describes a framework for constructing such models, based on layered queue templates for EJBs, and for their inclusion in the server. The resulting model is calibrated and validated by comparison with an actual system.
---
paper_title: Packaging Predictable Assembly

paper_content:

Significant economic and technical benefits accrue from the use of pre-existing and commercially available software components to develop new systems. However, challenges remain that, if not adequately addressed, will slow the adoption of software component technology. Chief among these are a lack of consumer trust in the quality of components, and a lack of trust in the quality of assemblies of components without extensive and expensive testing. This paper describes prediction-enabled component technology (PECT). A PECT results from integrating component technology with analysis models. An analysis model permits analysis and prediction of assembly-level properties prior to component composition, and, perhaps, prior to component acquisition. Analysis models also identify required component properties and their certifiable descriptions. Component technology supports and enforces the assumptions underlying analysis models; it also provides the medium for deploying PECT instances and PECT-compliant software components. This paper describes the structure of PECT. It discusses the means of establishing the predictive powers of a PECT so that consumers may obtain measurably bounded trust in both components and design-time predictions based on the use of these components. We demonstrate these ideas in a simple but illustrative model problem: predicting average endto-end latency of a 'soft' real time application built from off-the-shelf software components.
---
paper_title: Predicting the Behavior of a Highly Configurable Component Based Real-Time System

paper_content:

Software components and the technology supporting component based software engineering contribute greatly to the rapid development and configuration of systems for a variety of application domains. Such domains go beyond desktop office applications and information systems supporting e-commerce, but include systems having real-time performance requirements and critical functionality. Discussed in this paper are the results from an experiment that demonstrates the ability to predict deadline satisfaction of threads in a real-time system where the functionality performed is based on the configuration of the assembled software components. Presented is the method used to abstract the large, legacy code base of the system software and the application software components in the system; the model of those abstractions based on available architecture documentation and empirically-based, runtime observations; and the analysis of the predictions which yielded objective confidence in the observations and model created which formed the underlying basis for the predictions.
---
paper_title: From design to analysis models: a kernel language for performance and reliability analysis of component-based systems

paper_content:

To facilitate the use of non-functional analysis results in the selection and assembly of components for component-based systems, automatic prediction tools should be devised, to predict some overall quality attribute of the application without requiring extensive knowledge of analysis methodologies to the application designer. To achieve this goal, a key idea is to define a model transformation that takes as input some "design-oriented" model of the component assembly and produces as a result an "analysis-oriented" model that lends itself to the application of some analysis methodology. However, to actually devise such a transformation, we must face both the heterogeneous design level notations for component-based systems, and the variety of non-functional attributes and related analysis methodologies we could be interested in. In this perspective, we define a kernel language whose aim is to capture the relevant information for the analysis of non-functional attributes of component-based systems, with a focus on performance and reliability. Using this kernel language as a bridge between design-oriented and analysis-oriented notations we reduce the burden of defining a variety of direct transformations from the former to the latter to the less complex problem of defining transformations to/from the kernel language. The proposed kernel language is defined within the MOF (Meta-Object Facility) framework, to allow the exploitation of MOF-based model transformation facilities.
---
paper_title: Model-driven performance analysis

paper_content:

Model-Driven Engineering (MDE) is an approach to develop software systems by creating models and applying automated transformations to them to ultimately generate the implementation for a target platform. Although the main focus of MDE is on the generation of code, it is also necessary to support the analysis of the designs with respect to quality attributes such as performance. To complement the model-to-implementation path of MDE approaches, an MDE tool infrastructure should provide what we call model-driven analysis. This paper describes an approach to model-driven analysis based on reasoning frameworks. In particular, it describes a performance reasoning framework that can transform a design into a model suitable for analysis of real-time performance properties with different evaluation procedures including rate monotonic analysis and simulation. The concepts presented in this paper have been implemented in the PACC Starter Kit, a development environment that supports code generation and analysis from the same models.
---
paper_title: Performance prediction for black-box components using reengineered parametric behaviour models

paper_content:

In component-based software engineering, the response time of an entire application is often predicted from the execution durations of individual component services. However, these execution durations are specific for an execution platform (i.e. its resources such as CPU) and for a usage profile. Reusing an existing component on different execution platforms up to now required repeated measurements of the concerned components for each relevant combination of execution platform and usage profile, leading to high effort. This paper presents a novel integrated approach that overcomes these limitations by reconstructing behaviour models with platform-independent resource demands of bytecode components. The reconstructed models are parameterised over input parameter values. Using platform-specific results of bytecode benchmarking, our approach is able to translate the platform-independent resource demands into predictions for execution durations on a certain platform. We validate our approach by predicting the performance of a file sharing application.
---
paper_title: TESTEJB - A measurement framework for EJBs

paper_content:

Specification of Quality of Service (QoS) for components can only be done in relation to the QoS the components themselves are given by imported components. Developers as well as users need support in order to derive valid data for specification respectively for checking whether a selected component complies with its specification. In this paper we introduce the architecture of a measurement framework for EJBs giving such support and discuss in detail the measurement of the well understood property of response time.
---
paper_title: Snapshot of CCL : A Language for Predictable Assembly

paper_content:

Abstract : Construction and composition language (CCL) plays several roles in our approach to achieving automated predictable assembly. CCL is used to produce specifications that contain structural, behavioral, and analysis-specific information about component technologies, as well as components and assemblies in such technologies. These specifications are translated to one or more reasoning frameworks that analyze and predict the runtime properties of assemblies. CCL processors can also be used to automate many of the constructive activities of component-based development through various forms of program generation. Using a common specification for prediction and construction improves confidence that analysis models match implementations. This report presents a snapshot of CCL by examining a small example CCL specification.
---
paper_title: The COMQUAD component model: enabling dynamic selection of implementations by weaving non-functional aspects

paper_content:

The reliability of non-functional contracts is crucial for many software applications. This added to the increasing attention this issue lately received in software engineering. Another development in software engineering is toward component-based systems. The interaction of both, non-functional aspects and components, is a relatively new research area, which the COMQUAD project is focusing on.Our component model, presented in this paper, enables the specification and runtime support of non-functional aspects in component-based systems. At the same time, a clear separation of non-functional properties and functionally motivated issues is provided. We achieve this by extending the concepts of the existing component-based systems Enterprise JavaBeans (EJB) and CORBA Components (CCM). Non-functional aspects are described orthogonally to the application structure using descriptors, and are woven into the running application by the component container acting as a contract manager. The container implicitly instantiates component specifications and connects them according to the current requests. The selection of actual implementations depends on the particular client's non-functional requirements. This technique also enables adaptation based on the specific quantitative capabilities of the running system.In this paper we give a detailed description of the COMQUAD component model and the appropriate container support. We also provide a simple case study of a multimedia application for better understanding.
---
paper_title: Predicting real-time properties of component assemblies: a scenario-simulation approach

paper_content:

This work addresses the problem of predicting timing properties of multitasking component assemblies during the design phase. For real-time applications, it is of vital importance to guarantee that the timing requirements of an assembly are met. We propose a simulation-based approach for predicting the real-time behaviour of an assembly based on models of its constituent components. Our approach extends the scenario-based method in [J. Muskens et al. (2004)] by offering a system model that is tailored to the domain of real-time applications. Contributions of This work include the possibility to handle the following features: mutual exclusion, combinations of aperiodic and periodic tasks and synchronization constraints. The analytical approach we used in previous work cannot handle these features. Therefore, we introduce the simulation-based approach. Our simulator provides data about dynamic resource consumption and real-time properties like response time, blocking time and number of missed deadlines per task. We have validated our approach using a video-decoder application.
---
paper_title: Exploring performance trade-offs of a JPEG decoder using the deepcompass framework

paper_content:

Designing embedded systems for multiprocessor platforms requires early prediction and balancing of multiple system quality attributes. We present a design space exploration framework for component-based software systems that allows an architect to get insight into a space of possible design alternatives with further evaluation and comparison of these alternatives. The framework provides (a) tool-guided design of multiple alternatives of software and hardware architectures, (b) early design-time predictions of performance properties and identification of bottlenecks for each architectural alternative, and (c) evaluation of each alternative with respect to multi-objective trade-offs. The performance prediction technique employs modeling of individual components and composition of the models into a system model representing the system behaviour and resource usage. We illustrate the framework by a case study of a JPEG decoder application. For this system, we consider architectural alternatives, show their specification, and explore their trade-offs with respect to task latencies, resource utilization and system cost.
---
paper_title: The Method of Layers

paper_content:

Distributed applications are being developed that contain one or more layers of software servers. Software processes within such systems suffer contention delays both for shared hardware and at the software servers. The responsiveness of these systems is affected by the software design, the threading level and number of instances of software processes, and the allocation of processes to processors. The Method of Layers (MOL) is proposed to provide performance estimates for such systems. The MOL uses the mean value analysis (MVA) linearizer algorithm as a subprogram to assist in predicting model performance measures. >
---
paper_title: Performance and scalability of EJB applications

paper_content:

We investigate the combined effect of application implementation method, container design, and efficiency of communication layers on the performance scalability of J2EE application servers by detailed measurement and profiling of an auction site server.We have implemented five versions of the auction site. The first version uses stateless session beans, making only minimal use of the services provided by the Enterprise JavaBeans (EJB) container. Two versions use entity beans, one with container-managed persistence and the other with bean-managed persistence. The fourth version applies the session fasade pattern, using session beans as a fasade to access entity beans. The last version uses EJB 2.0 local interfaces with the session fasade pattern. We evaluate these different implementations on two popular open-source EJB containers with orthogonal designs. JBoss uses dynamic proxies to generate the container classes at run time, making an extensive use of reflection. JOnAS pre-compiles classes during deployment, minimizing the use of reflection at run time. We also evaluate the communication optimizations provided by each of these EJB containers.The most important factor in determining performance is the application implementation method. EJB applications with session beans perform as well as a Java servlets-only implementation and an order-of-magnitude better than most of the implementations based on entity beans. The fine-granularity access exposed by the entity beans limits scalability. Use of session fasade beans improves performance for entity beans, but only if local communication is very efficient or EJB 2.0 local interfaces are used. Otherwise, session fasade beans degrade performance.For the implementation using session beans, communication cost forms the major component of the execution time on the EJB server. The design of the container has little effect on performance. With entity beans, the design of the container becomes important. In particular, the cost of reflection affects performance. For implementations using session fasade beans, local communication cost is critically important. EJB 2.0 local interfaces improve the performance by avoiding the communication layers for local communications.
---
paper_title: Revel8or: Model Driven Capacity Planning Tool Suite

paper_content:

Designing complex multi-tier applications that must meet strict performance requirements is a challenging software engineering problem. Ideally, the application architect could derive accurate performance predictions early in the project life-cycle, leveraging initial application design-level models and a description of the target software and hardware platforms. To this end, we have developed a capacity planning tool suite for component-based applications, called Revel8tor. The tool adheres to the model driven development paradigm and supports benchmarking and performance prediction for J2EE, .Net and Web services platforms. The suite is composed of three different tools: MDAPerf, MDABench and DSLBench. MDAPerf allows annotation of design diagrams and derives performance analysis models. MDABench allows a customized benchmark application to be modeled in the UML 2.0 Testing Profile and automatically generates a deployable application, with measurement automatically conducted. DSLBench allows the same benchmark modeling and generation to be conducted using a simple performance engineering Domain Specific Language (DSL) in Microsoft Visual Studio. DSLBench integrates with Visual Studio and reuses its load testing infrastructure. Together, the tool suite can assist capacity planning across platforms in an automated fashion.
---
paper_title: Design-level performance prediction of component-based applications

paper_content:

Server-side component technologies such as Enterprise JavaBeans (EJBs), .NET, and CORBA are commonly used in enterprise applications that have requirements for high performance and scalability. When designing such applications, architects must select suitable component technology platform and application architecture to provide the required performance. This is challenging as no methods or tools exist to predict application performance without building a significant prototype version for subsequent benchmarking. In this paper, we present an approach to predict the performance of component-based server-side applications during the design phase of software development. The approach constructs a quantitative performance model for a proposed application. The model requires inputs from an application-independent performance profile of the underlying component technology platform, and a design description of the application. The results from the model allow the architect to make early decisions between alternative application architectures in terms of their performance and scalability. We demonstrate the method using an EJB application and validate predictions from the model by implementing two different application architectures and measuring their performance on two different implementations of the EJB platform.
---
paper_title: Mean-Value Analysis of Closed Multichain Queuing Networks

paper_content:

It is shown that mean queue sizes, mean waiting times, and throughputs in closed multiple-chain queuing networks which have product-form solution can be computed recursively without computing product terms and normalization constants. The resulting computational procedures have improved properties (avoidance of numerical problems and, in some cases, fewer operations) compared to previous algorithms. Furthermore, the new algorithms have a physically meaningful interpretation which provides the basis for heuristic extensions that allow the approximate solution of networks with a very large number of closed chains, and which is shown to be asymptotically valid for large chain populations.
---
paper_title: Early performance testing of distributed software applications

paper_content:

Performance characteristics, such as response time, through put and scalability, are key quality attributes of distributed applications. Current practice, however, rarely applies systematic techniques to evaluate performance characteristics. We argue that evaluation of performance is particularly crucial in early development stages, when important architectural choices are made. At first glance, this contradicts the use of testing techniques, which are usually applied towards the end of a project. In this paper, we assume that many distributed systems are built with middleware technologies, such as the Java 2 Enterprise Edition (J2EE) or the Common Object Request Broker Architecture (CORBA). These provide services and facilities whose implementations are available when architectures are defined. We also note that it is the middleware functionality, such as transaction and persistence services, remote communication primitives and threading policy primitives, that dominate distributed system performance Drawing on these observations, this paper presents a novel approach to performance testing of distributed applications. We propose to derive application-specific test cases from architecture designs so that performance of a distributed application can be tested using the middleware software at early stages of a development process. We report empirical results that support the viability of the approach.
---
paper_title: Performance prediction of component-based applications

paper_content:

One of the major problems in building large-scale enterprise systems is anticipating the performance of the eventual solution before it has been built. The fundamental software engineering problem becomes more difficult when the systems are built on component technology. This paper investigates the feasibility of providing a practical solution to this problem. An empirical approach is proposed to determine the performance characteristics of component-based applications by benchmarking and profiling. Based on observation, a model is constructed to act as a performance predictor for a class of applications based on the specific component technology. The performance model derived from empirical measures is necessary to make the problem tractable and the results relevant. A case study applies the performance model to an application prototype implemented by two component infrastructures: CORBA and J2EE.
---
paper_title: Theory of software reliability based on components

paper_content:

We present a foundational theory of software system reliability based on components. The theory describes how component developers can design and test their components to produce measurements that are later used by system designers to calculate composite system reliability — without implementation and test of the system being designed. The theory describes how to make component measurements that are independent of operational profiles, and how to incorporate the overall system-level operational profile into the system reliability calculations. In principle, the theory resolves the central problem of assessing a component, which is: a component developer cannot know how the component will be used and so cannot certify it for an arbitrary use; but if the component buyer must certify each component before using it, component-based development loses much of its appeal. This dilemma is resolved if the component developer does the certification and provides the results in such a way that the component buyer can factor in the usage information later, without repeating the certification. Our theory addresses the basic technical problems inherent in certifying components to be released for later use in an arbitrary system. Most component research has been directed at functional specification of software components; our theory addresses the other, equally important, side of the coin: component quality.
---
paper_title: Performance specification of software components

paper_content:

Component-based software engineering is concerned with predictability in both functional and performance behavior, though most formal techniques have typically focused their attention on the former. Reasoning about the (functional or performance) behavior of a component-based system must be compositional in order to be scalable. Compositional performance reasoning demands that components include performance specifications, in addition to descriptions of functional behavior. Unfortunately, as explained in this paper, classical techniques and notations for performance analysis are either unsuitable or unnatural to capture performance behaviors of generic software components. They fail to work in the presence of parameterization and layering. The paper introduces elements of a compositional approach to performance analysis using a detailed example. It explains that performance specification problems are so basic that there are unresolved research issues to be tackled even for the simplest reusable components. These issues must be tackled by any practical proposal for sound performance reasoning. Only then will software developers be able to engineer new systems by choosing and assembling components that best fit their performance (time and space) requirements.
---
paper_title: Tools and experiments supporting a testing-based theory of component composition

paper_content:

Development of software using off-the-shelf components seems to offer a chance for improving product quality and developer productivity. This article reviews a foundational testing-based theory of component composition, describes tools that implement the theory, and presents experiments with functional and nonfunctional component/system properties that validate the theory and illuminate issues in component composition. The context for this work is an ideal form of Component-Based Software Development (CBSD) supported by tools. Component developers describe their components by measuring approximations to functional and nonfunctional behavior on a finite collection of subdomains. Systems designers describe an application-system structure by the component connections that form it. From measured component descriptions and a system structure, a CAD tool synthesizes the system properties, predicting how the system will behave. The system is not built, nor are any test executions performed. Neither the component sources nor executables are needed by systems designers. From CAD calculations a designer can learn (approximately) anything that could be learned by testing an actual system implementation. The CAD tool is often more efficient than it would be to assemble and execute an actual system. Using tools that support an ideal separation between component- and system development, experiments were conducted to investigate two related questions: (1) To what extent can unit (that is, component) testing replace system testingq (2) What properties of software and subdomains influence the quality of subdomain testingq
---
paper_title: Software component composition: a subdomain-based testing-theory foundation

paper_content:

Composition of software elements into assemblies (systems) is a fundamental aspect of software development. It is an important strength of formal mathematical specification that the descriptions of elements can be precisely composed into the descriptions of assemblies. Testing, on the other hand, is usually thought to be ‘non-compositional.’ Testing provides information about any executable software element, but testing descriptions have not been combined to describe assemblies of elements. The underlying reason for the compositional deficiency of testing is that tests are samples. When two elements are composed, the input samples (test points) for the first lead to an output sample, but it does not match the input test points of the second, following element. The current interest in software components and component-based software development (CBSD) provides an ideal context for investigating elements and assemblies. In CBSD, the elements (components) are analysed without knowledge of the system(s) to be later assembled. A fundamental testing theory of component composition must use measured component properties (test results) to predict system properties. This paper proposes a testing-based theory of software component composition based on subdomains. It shows how to combine subdomain tests of components into testing predictions for arbitrarily complex assemblies formed by sequence, conditional, and iteration constructions. The basic construction of the theory applies to functional behaviour, but the theory can also predict the system's non-functional properties from component subdomain tests. Compared with the alternative of actually building and testing a system, theoretical predictions are computationally more efficient. The theory can also be described as an exercise in modelling. Components are replaced by abstractions derived from testing them, and these models are manipulated to model system behaviour. This article replaces a previously published version. DOI: 10.1002/stvr.368. Copyright © 2007 John Wiley & Sons, Ltd.
---
paper_title: Automating the performance management of component-based enterprise systems through the use of redundancy

paper_content:

Component technologies are increasingly being used for building enterprise systems, as they can address complex functionality and flexibility problems and reduce development and maintenance costs. Nonetheless, current component technologies provide little support for predicting and controlling the emerging performance of software systems that are assembled from distinct components.This paper presents a framework for automating the performance management of complex, component-based systems. The adopted approach is based on the alternate usage of multiple component variants with equivalent functional characteristics, each one optimized for a different running environment. A fully-automated framework prototype for J2EE is presented, along with results from managing a sample enterprise application on JBoss. A mechanism that uses monitoring data to learn and automatically improve the framework's management behaviour is proposed. The framework imposes no extra requirements on component providers, or on the component technologies.
---
paper_title: A framework for performance monitoring, modelling and prediction of component oriented distributed systems

paper_content:

We present a framework that can be used to identify performance issues in component-oriented distributed systems. The framework consists of a monitoring module, a modelling module and a prediction module, that are interrelated. The monitoring block extracts real-time performance data from a live or under development system. The modelling block generates UML models of the system showing where the performance problems are located and drives the monitoring process. The performance prediction block simulates different system-loads on the generated models and pinpoints possible performance issues. The technological focus is currently on Enterprise Java Beans systems.
---
paper_title: Performance Solutions A Practical Guide To Creating Responsive Scalable Software

paper_content:

Thank you for downloading performance solutions a practical guide to creating responsive scalable software. As you may know, people have search numerous times for their favorite books like this performance solutions a practical guide to creating responsive scalable software, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they cope with some malicious bugs inside their laptop.
---
paper_title: TESTEJB - A measurement framework for EJBs

paper_content:

Specification of Quality of Service (QoS) for components can only be done in relation to the QoS the components themselves are given by imported components. Developers as well as users need support in order to derive valid data for specification respectively for checking whether a selected component complies with its specification. In this paper we introduce the architecture of a measurement framework for EJBs giving such support and discuss in detail the measurement of the well understood property of response time.
---
paper_title: Detecting Performance Antipatterns in Component Based Enterprise Systems

paper_content:

We introduce an approach for automatic detection of performance antipatterns. The approach is based on a number of advanced monitoring and analysis techniques. The advanced analysis is used to identify relationships and patterns in the monitored data. This information is subsequently used to reconstruct a design model of the underlying system, which is loaded into a rule engine in order to identify predefined antipatterns. We give results of applying this approach to identify a number of antipatterns in two JEE applications. Finally, this work also categorises JEE antipatterns into categories based on the data needed to detect them.
---
paper_title: Performance prediction for black-box components using reengineered parametric behaviour models

paper_content:

In component-based software engineering, the response time of an entire application is often predicted from the execution durations of individual component services. However, these execution durations are specific for an execution platform (i.e. its resources such as CPU) and for a usage profile. Reusing an existing component on different execution platforms up to now required repeated measurements of the concerned components for each relevant combination of execution platform and usage profile, leading to high effort. This paper presents a novel integrated approach that overcomes these limitations by reconstructing behaviour models with platform-independent resource demands of bytecode components. The reconstructed models are parameterised over input parameter values. Using platform-specific results of bytecode benchmarking, our approach is able to translate the platform-independent resource demands into predictions for execution durations on a certain platform. We validate our approach by predicting the performance of a file sharing application.
---
paper_title: A model transformation approach for the early performance and reliability analysis of component-based systems

paper_content:

The adoption of a “high level” perspective in the design of a component-based application, without considering the specific features of some underlying supporting platform, has the advantage of focusingon the relevant architectural aspects and reasoning about them in a platform independent way, omitting unnecessary details that could even not be known at the earliest development stages.On the other hand, many of the details that are typically neglected in this high-level perspective must necessarily be taken into account to obtain a meaningful evaluation of different architectural choices in terms of extra-functional quality attributes, like performance or reliability. Toward the reconciliation of these two contrasting needs, we propose a model-based approach whose goal is to support the derivation of sufficiently detailed prediction models from high level models of component-based systems, focusing on the prediction of performance and reliability. We exploit for this purpose a refinement mechanism based on the use of model transformation techniques.
---
paper_title: Automatic inclusion of middleware performance attributes into architectural UML software models

paper_content:

Distributed systems often use a form of communication middleware to cope with different forms of heterogeneity, including geographical spreading of the components, different programming languages and platform architectures, etc. The middleware, of course, impact the architecture and the performance of the system. This paper presents a model transformation framework to automatically include the architectural impact and the overhead incurred by using a middleware layer between several system components. Using this framework, architects can model the system in a middleware-independent fashion. Accurate, middleware-aware models can then be obtained automatically using a middleware model repository. The actual transformation algorithm is presented in more detail. The resulting models can be used to obtain performance models of the system. From those performance models, early indications of the system performance can be extracted.
---
paper_title: Performance prediction for component compositions

paper_content:

A stepwise approach is proposed to predict the performance of component compositions. The approach considers the major factors influencing the performance of component compositions in sequence: component operations, activities, and composition of activities. During each step, various models - analytical, statistical, simulation - can be constructed to specify the contribution of each relevant factor to the performance of the composition. The architects can flexibly choose which model they use at each step in order to trade prediction accuracy against prediction effort. The approach is illustrated with an example about the performance prediction for an Automobile Navigation System.
---
paper_title: Component Composition with Parametric Contracts

paper_content:

We discuss compositionality in terms of (a) component interoperability and contractual use of components, (b) component adaptation and (c) prediction of properties of composite components. In particular, we present parametric component contracts as a framework treating the above mentioned facets of compositionality in a unified way. Parametric contracts compute component interfaces in dependency of context properties, such as available external services or the profile how the component will be used by its clients. Under well-specified conditions, parametric contracts yield interfaces offering interoperability to the component context (as they are component-specifically generated). Therefore, parametric contracts can be considered as adaptation mechanism, adapting a components providesor requires-interface depending on connected components. If non-functional properties are specified in a component provides interface, parametric contracts compute these nonfunctional properties in dependency of the environment.
---
paper_title: Software component models

paper_content:

Component-based Development (CBD) is an important emerging topic in Software Engineering, promising long sought after benefits like increased reuse and reduced time-to-market (and hence software production cost). However, there are at present many obstacles to overcome before CBD can succeed. For one thing, CBD success is predicated on a standardised market place for software components, which does not yet exist. In fact currently CBD even lacks a universally accepted terminology. Existing component models adopt different component definitions and composition operators. Therefore much research remains to be done. We believe that the starting point for this endeavour should be a thorough study of current component models, identifying their key characteristics and comparing their strengths and weaknesses. A desirable side-effect would be clarifying and unifying the CBD terminology. In this tutorial, we present a clear and concise exposition of all the current major software component models, including a taxonomy. The purpose is to distill and present knowledge of current software component models, as well as to present an analysis of their properties with respect to commonly accepted criteria for CBD. The taxonomy also provides a starting point for a unified terminology.
---
paper_title: How far are we from the definition of a common software performance ontology?

paper_content:

The recent approaches to software performance modeling and validation share the idea of annotating software models with information related to performance (e.g. operational profile) and transforming the annotated model into a performance model (e.g. a Stochastic Petri Net). Up to date, no standard has been defined to represent the information related to performance in software artifacts, although clear advantages in tool interoperability and model transformations would stem from it. This paper is aimed at questioning whether a software performance ontology (i.e. a standard set of concepts and relations) is achievable or not. We consider three meta-models defined for software performance, that are the Schedulability, Performance and Time profile of UML, the Core Scenario Model and the Software Performance Engineering meta-model. We devise two approaches to the creation of an ontology: (i) bottom-up, that extracts common knowledge from the meta-models, (ii) top-down, that is driven from a set of requirements.
---
paper_title: An empirical investigation of the effort of creating reusable models for performance prediction

paper_content:

Model-based performance prediction methods aim at evaluating the expected response time, throughput, and resource utilisation of a software system at design time, before implementation. Existing performance prediction methods use monolithic, throw-away prediction models or component-based, reusable prediction models. While it is intuitively clear that the development of reusable models requires more effort, the actual higher amount of effort has not been quantified or analysed systematically yet. To study the effort, we conducted a controlled experiment with 19 computer science students who predicted the performance of two example systems applying an established, monolithic method (Software Performance Engineering) as well as our own component-based method (Palladio). The results show that the effort of model creation with Palladio is approximately 1.25 times higher than with SPE in our experimental setting, with the resulting models having comparable prediction accuracy. Therefore, in some cases, the creation of reusable prediction models can already be justified, if they are reused at least once.
---
paper_title: Predicting the Behavior of a Highly Configurable Component Based Real-Time System

paper_content:

Software components and the technology supporting component based software engineering contribute greatly to the rapid development and configuration of systems for a variety of application domains. Such domains go beyond desktop office applications and information systems supporting e-commerce, but include systems having real-time performance requirements and critical functionality. Discussed in this paper are the results from an experiment that demonstrates the ability to predict deadline satisfaction of threads in a real-time system where the functionality performed is based on the configuration of the assembled software components. Presented is the method used to abstract the large, legacy code base of the system software and the application software components in the system; the model of those abstractions based on available architecture documentation and empirically-based, runtime observations; and the analysis of the predictions which yielded objective confidence in the observations and model created which formed the underlying basis for the predictions.
---
paper_title: Rule-based automatic software performance diagnosis and improvement

paper_content:

There are many advantages to analyzing performance at the design level, rather than waiting until system testing. However the necessary expertise in making and interpreting performance models may not be available, and the time for the analysis may be prohibitive. This work addresses both these difficulties through automation. Starting from an annotated specification in UML, it is possible to automatically derive a performance model. This work goes further to automate the performance analysis, and to explore design changes using diagnostic and design-change rules. The rules generate improved performance models which can be transformed back to an improved design. They untangle the effects of the system configuration (such as the allocation of processors) from limitations of the design, and they recommend both configuration and design improvements. This paper describes a prototype called Performance Booster (PB), which incorporates several rules, and demonstrates its feasibility by applying PB to the design of several case studies (tutorial and industrial). It also addresses how the changes at the level of a performance model should be implemented in the software.
---
paper_title: Performance Solutions A Practical Guide To Creating Responsive Scalable Software

paper_content:

Thank you for downloading performance solutions a practical guide to creating responsive scalable software. As you may know, people have search numerous times for their favorite books like this performance solutions a practical guide to creating responsive scalable software, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they cope with some malicious bugs inside their laptop.
---
paper_title: Exploring performance trade-offs of a JPEG decoder using the deepcompass framework

paper_content:

Designing embedded systems for multiprocessor platforms requires early prediction and balancing of multiple system quality attributes. We present a design space exploration framework for component-based software systems that allows an architect to get insight into a space of possible design alternatives with further evaluation and comparison of these alternatives. The framework provides (a) tool-guided design of multiple alternatives of software and hardware architectures, (b) early design-time predictions of performance properties and identification of bottlenecks for each architectural alternative, and (c) evaluation of each alternative with respect to multi-objective trade-offs. The performance prediction technique employs modeling of individual components and composition of the models into a system model representing the system behaviour and resource usage. We illustrate the framework by a case study of a JPEG decoder application. For this system, we consider architectural alternatives, show their specification, and explore their trade-offs with respect to task latencies, resource utilization and system cost.
---
paper_title: Performance prediction for black-box components using reengineered parametric behaviour models

paper_content:

In component-based software engineering, the response time of an entire application is often predicted from the execution durations of individual component services. However, these execution durations are specific for an execution platform (i.e. its resources such as CPU) and for a usage profile. Reusing an existing component on different execution platforms up to now required repeated measurements of the concerned components for each relevant combination of execution platform and usage profile, leading to high effort. This paper presents a novel integrated approach that overcomes these limitations by reconstructing behaviour models with platform-independent resource demands of bytecode components. The reconstructed models are parameterised over input parameter values. Using platform-specific results of bytecode benchmarking, our approach is able to translate the platform-independent resource demands into predictions for execution durations on a certain platform. We validate our approach by predicting the performance of a file sharing application.
---
paper_title: Performance prediction of component-based applications

paper_content:

One of the major problems in building large-scale enterprise systems is anticipating the performance of the eventual solution before it has been built. The fundamental software engineering problem becomes more difficult when the systems are built on component technology. This paper investigates the feasibility of providing a practical solution to this problem. An empirical approach is proposed to determine the performance characteristics of component-based applications by benchmarking and profiling. Based on observation, a model is constructed to act as a performance predictor for a class of applications based on the specific component technology. The performance model derived from empirical measures is necessary to make the problem tractable and the results relevant. A case study applies the performance model to an application prototype implemented by two component infrastructures: CORBA and J2EE.
---
paper_title: Predicting the performance of component-based software architectures with different usage profiles

paper_content:

Performance predictions aim at increasing the quality of software architectures during design time. To enable such predictions, specifications of the performance properties of individual components within the architecture are required. However, the response times of a component might depend on its configuration in a specific setting and the data send to or retrieved from it. Many existing prediction approaches for component-based systems neglect these influences. This paper introduces extensions to a performance specification language for components, the Palladio Component Model, to model these influences. The model enables to predict response times of different architectural alternatives. A case study on a component-based architecture for a web portal validates the approach and shows that it is capable of supporting a design decision in this scenario.
---
paper_title: Modelling of input-parameter dependency for performance predictions of component-based embedded systems

paper_content:

The guaranty of meeting the timing constraints during the design phase of real-time component-based embedded software has not been realized. To satisfy real-time requirements, we need to understand behaviour and resource usage of a system over time. In this paper, we address both aspects in detail by observing the influence of input data on the system behaviour and performance. We extend an existing scenario simulation approach that features the modelling of input parameter dependencies and simulating the execution of the models. The approach enables specification of the dependencies in the component models, as well as initialisation of the parameters in the application scenario model. This gives a component-based application designer an explorative possibility of going through all possible execution scenarios with different parameter initialisations, and finding the worst-case scenarios where the predicted performance does not satisfy the requirements. The identification of these scenarios is important because it avoids system redesign at the later stage. In addition, the conditional behaviour and resource usage modelling with respect to the input data provide more accurate prediction.
---
paper_title: An empirical investigation of the effort of creating reusable models for performance prediction

paper_content:

Model-based performance prediction methods aim at evaluating the expected response time, throughput, and resource utilisation of a software system at design time, before implementation. Existing performance prediction methods use monolithic, throw-away prediction models or component-based, reusable prediction models. While it is intuitively clear that the development of reusable models requires more effort, the actual higher amount of effort has not been quantified or analysed systematically yet. To study the effort, we conducted a controlled experiment with 19 computer science students who predicted the performance of two example systems applying an established, monolithic method (Software Performance Engineering) as well as our own component-based method (Palladio). The results show that the effort of model creation with Palladio is approximately 1.25 times higher than with SPE in our experimental setting, with the resulting models having comparable prediction accuracy. Therefore, in some cases, the creation of reusable prediction models can already be justified, if they are reused at least once.
---
paper_title: Software performance in the real world: personal lessons from the performance trauma team

paper_content:

In the nine years that we have been involved in software performance engineering (SPE) and performance testing engagements we have learned several things. Across numerous eCommerce applications and an enterprise CRM product suite, our knowledge base about the field of Software Performance Engineering is constantly evolving. The focus of this paper is what we have learned in the areas of SPE project management, performance testing, defining the scope of SPE projects, ITIL, post production performance support, and exploration of the boundaries of applied SPE. Is it really just about performance.
---
paper_title: The Future of Software Performance Engineering

paper_content:

Performance is a pervasive quality of software systems; everything affects it, from the software itself to all underlying layers, such as operating system, middleware, hardware, communication networks, etc. Software Performance Engineering encompasses efforts to describe and improve performance, with two distinct approaches: an early-cycle predictive model-based approach, and a late-cycle measurement-based approach. Current progress and future trends within these two approaches are described, with a tendency (and a need) for them to converge, in order to cover the entire development cycle.
---
paper_title: Performance Model Estimation and Tracking Using Optimal Filters

paper_content:

To update a performance model, its parameter values must be updated, and in some applications (such as autonomic systems) tracked continuously over time. Direct measurement of many parameters during system operation requires instrumentation which is impractical. Kalman filter estimators can track such parameters using other data such as response times and utilizations, which are readily observable. This paper adapts Kalman filter estimators for performance model parameters, evaluates the approximations which must be made, and develops a systematic approach to setting up an estimator. The estimator converges under easily verified conditions. Different queueing-based models are considered here, and the extension for state-based models (such as stochastic Petri nets) is straightforward.
---
paper_title: Predicting the Behavior of a Highly Configurable Component Based Real-Time System

paper_content:

Software components and the technology supporting component based software engineering contribute greatly to the rapid development and configuration of systems for a variety of application domains. Such domains go beyond desktop office applications and information systems supporting e-commerce, but include systems having real-time performance requirements and critical functionality. Discussed in this paper are the results from an experiment that demonstrates the ability to predict deadline satisfaction of threads in a real-time system where the functionality performed is based on the configuration of the assembled software components. Presented is the method used to abstract the large, legacy code base of the system software and the application software components in the system; the model of those abstractions based on available architecture documentation and empirically-based, runtime observations; and the analysis of the predictions which yielded objective confidence in the observations and model created which formed the underlying basis for the predictions.
---
paper_title: Next generation data centers: trends and implications

paper_content:

In this talk we will discuss next generation data centers and the important impact they will have upon enterprise applications. Specifically, we will discuss the technical and economical trends motivating the move towards large scale distributed data centers consisting of tens of thousands of servers and hundreds of petabytes of storage. We will explain the roles and advantages of virtual machines and other virtualization technologies in these environments and also explore how they exacerbate the complexity of management and achieving predictable application behavior.To better illustrate issues emerging in such environments we will describe early experiments we conducted with a 1000-processor utility rendering service created for DreamWorks Animation that was used to render the films Shrek II and Madagascar. We will discuss the lessons learned from this experience.Next, we consider the trend towards service oriented architectures for enterprise application platforms. Service orientation provides for more flexible and agile information technology systems but further increases the complexity of management and behavior. We will explore the implications of composing services dynamically using an SOA approach.These trends for enterprise application platforms and the trends towards next generation data centers have helped to drive our current research agenda. Our goal is to enable the flexibility and agility offered by these new technologies while enabling cost effective management, predictable behavior and improved quality of service. We will give an overview of our research on model driven design for enterprise application infrastructure, automated deployment, and operations of distributed application services executing in a virtualized, shared resource pool within these next generation data centers.Finally, we will summarize the implications, challenges, and opportunities posed by these trends on academic and industrial research. In particular, we consider the impact on software performance and software performance engineering and pose important unanswered questions.
---
paper_title: Rule-based automatic software performance diagnosis and improvement

paper_content:

There are many advantages to analyzing performance at the design level, rather than waiting until system testing. However the necessary expertise in making and interpreting performance models may not be available, and the time for the analysis may be prohibitive. This work addresses both these difficulties through automation. Starting from an annotated specification in UML, it is possible to automatically derive a performance model. This work goes further to automate the performance analysis, and to explore design changes using diagnostic and design-change rules. The rules generate improved performance models which can be transformed back to an improved design. They untangle the effects of the system configuration (such as the allocation of processors) from limitations of the design, and they recommend both configuration and design improvements. This paper describes a prototype called Performance Booster (PB), which incorporates several rules, and demonstrates its feasibility by applying PB to the design of several case studies (tutorial and industrial). It also addresses how the changes at the level of a performance model should be implemented in the software.
---
paper_title: Performance Solutions A Practical Guide To Creating Responsive Scalable Software

paper_content:

Thank you for downloading performance solutions a practical guide to creating responsive scalable software. As you may know, people have search numerous times for their favorite books like this performance solutions a practical guide to creating responsive scalable software, but end up in infectious downloads. Rather than reading a good book with a cup of tea in the afternoon, instead they cope with some malicious bugs inside their laptop.
---
paper_title: A framework for automated generation of architectural feedback from software performance analysis

paper_content:

A rather complex task in the performance analysis of software architectures has always been the interpretation of the analysis results and the generation of feedback that may help developers to improve their architecture with alternative "better performing" solutions. This is due, on one side, to the fact that performance analysis results may be rather complex to interpret (e.g., they are often collections of different indices) and, on the other side, to the problem of coupling the "right" architectural alternatives to results, that are the alternatives that allow to improve the performance by resolving critical issues in the architecture. In this paper we propose a framework to interpret the performance analysis results and to propose alternatives to developers that improve their architectural designs. The interpretation of results is based on the ability to automatically recognize performance anti-patterns in the software architecture. The whole process of result interpretation and generation of architectural alternatives is supported by a tool based on the Layered Queueing Network notation.
---
paper_title: Experimentation in software engineering: an introduction

paper_content:

The purpose of Experimentation in Software Engineering: An Introduction is to introduce students, teachers, researchers, and practitioners to experimentation and experimental evaluation with a focus on software engineering. The objective is, in particular, to provide guidelines for performing experiments evaluating methods, techniques and tools in software engineering. The introduction is provided through a process perspective. The focus is on the steps that we go through to perform experiments and quasi-experiments. The process also includes other types of empirical studies. The motivation for the book emerged from the need for support we experienced when turning our software engineering research more experimental. Several books are available which either treat the subject in very general terms or focus on some specific part of experimentation; most focus on the statistical methods in experimentation. These are important, but there were few books elaborating on experimentation from a process perspective, none addressing experimentation in software engineering in particular. The scope of Experimentation in Software Engineering: An Introduction is primarily experiments in software engineering as a means for evaluating methods, techniques and tools. The book provides some information regarding empirical studies in general, including both case studies and surveys. The intention is to provide a brief understanding of these strategies and in particular to relate them to experimentation. Experimentation in Software Engineering: An Introduction is suitable for use as a textbook or a secondary text for graduate courses, and for researchers and practitioners interested in an empirical approach to software engineering. (Less)
---
paper_title: Performance prediction for black-box components using reengineered parametric behaviour models

paper_content:

In component-based software engineering, the response time of an entire application is often predicted from the execution durations of individual component services. However, these execution durations are specific for an execution platform (i.e. its resources such as CPU) and for a usage profile. Reusing an existing component on different execution platforms up to now required repeated measurements of the concerned components for each relevant combination of execution platform and usage profile, leading to high effort. This paper presents a novel integrated approach that overcomes these limitations by reconstructing behaviour models with platform-independent resource demands of bytecode components. The reconstructed models are parameterised over input parameter values. Using platform-specific results of bytecode benchmarking, our approach is able to translate the platform-independent resource demands into predictions for execution durations on a certain platform. We validate our approach by predicting the performance of a file sharing application.
---
paper_title: TESTEJB - A measurement framework for EJBs

paper_content:

Specification of Quality of Service (QoS) for components can only be done in relation to the QoS the components themselves are given by imported components. Developers as well as users need support in order to derive valid data for specification respectively for checking whether a selected component complies with its specification. In this paper we introduce the architecture of a measurement framework for EJBs giving such support and discuss in detail the measurement of the well understood property of response time.
---
paper_title: Automatic inclusion of middleware performance attributes into architectural UML software models

paper_content:

Distributed systems often use a form of communication middleware to cope with different forms of heterogeneity, including geographical spreading of the components, different programming languages and platform architectures, etc. The middleware, of course, impact the architecture and the performance of the system. This paper presents a model transformation framework to automatically include the architectural impact and the overhead incurred by using a middleware layer between several system components. Using this framework, architects can model the system in a middleware-independent fashion. Accurate, middleware-aware models can then be obtained automatically using a middleware model repository. The actual transformation algorithm is presented in more detail. The resulting models can be used to obtain performance models of the system. From those performance models, early indications of the system performance can be extracted.
---
paper_title: Detecting Performance Antipatterns in Component Based Enterprise Systems

paper_content:

We introduce an approach for automatic detection of performance antipatterns. The approach is based on a number of advanced monitoring and analysis techniques. The advanced analysis is used to identify relationships and patterns in the monitored data. This information is subsequently used to reconstruct a design model of the underlying system, which is loaded into a rule engine in order to identify predefined antipatterns. We give results of applying this approach to identify a number of antipatterns in two JEE applications. Finally, this work also categorises JEE antipatterns into categories based on the data needed to detect them.
---
paper_title: Architecture-Based Software Reliability Analysis: Overview and Limitations

paper_content:

With the growing size and complexity of software applications, research in the area of architecture-based software reliability analysis has gained prominence. The purpose of this paper is to provide an overview of the existing research in this area, critically examine its limitations, and suggest ways to address the identified limitations
---
paper_title: A model-driven approach to performability analysis of dynamically reconfigurable component-based systems

paper_content:

Dynamic reconfiguration techniques appear promising to build component-based (C-B) systems for application domains that have strong adaptability requirements, like the mobile and the service-oriented computing domains. However, introducing dynamic reconfiguration features into a C-B application makes even more challenging the design and verification of functional and non functional requirements. Our goal is to support the model-based analysis of the effectiveness of reconfigurable C-B applications, with a focus on the assessment of the non-functional performance and reliability attributes. As a first step towards this end, we address the issue of selecting suitable analysis models for reconfigurable systems, suggesting to this end the use of joint performance and reliability (performability) models. Furthermore, we propose a model-driven approach to automatically transform a design model into an analysis model. For this purpose, we build on the existence of intermediate languages that have been proposed to facilitate this transformation and we extend one of them, to capture the core features (from a performance/reliability viewpoint) of a dynamically reconfigurable C-B system. Finally, we illustrate by a simple application example the main steps of the proposed approach.
---
paper_title: Hybrid performance modeling approach for network intensive distributed software

paper_content:

When designing and evaluating software architectures and network facilities for hosting demanding distributed applications, taking performance considerations into account is essential. A key factor in assessing the performance of such a distributed system is the network latency and its relation to the application behaviour. In this respect, it is important to include the performance impact of the network into the performance models used during the entire design cycle of the system.A framework is proposed that allows to model both the software and the network components separately and extracts a single set of performance estimates for the entire system. This has the advantage of allowing the network and software aspects to be modeled separately using the modeling languages and tools most suited to those system aspects. A case study is presented to illustrate the use of the framework and its usefulness in predicting system performance.
---
paper_title: Modelling of input-parameter dependency for performance predictions of component-based embedded systems

paper_content:

The guaranty of meeting the timing constraints during the design phase of real-time component-based embedded software has not been realized. To satisfy real-time requirements, we need to understand behaviour and resource usage of a system over time. In this paper, we address both aspects in detail by observing the influence of input data on the system behaviour and performance. We extend an existing scenario simulation approach that features the modelling of input parameter dependencies and simulating the execution of the models. The approach enables specification of the dependencies in the component models, as well as initialisation of the parameters in the application scenario model. This gives a component-based application designer an explorative possibility of going through all possible execution scenarios with different parameter initialisations, and finding the worst-case scenarios where the predicted performance does not satisfy the requirements. The identification of these scenarios is important because it avoids system redesign at the later stage. In addition, the conditional behaviour and resource usage modelling with respect to the input data provide more accurate prediction.
---

Title: Performance evaluation of component-based software systems: A survey
Section 1: Introduction
Description 1: This section provides an overview of the paper, discussing the goal of classifying performance prediction and measurement approaches for component-based software systems proposed over the last ten years. 

Section 2: Software Component Performance
Description 2: This section introduces the key terms and concepts needed to understand the challenges of performance evaluation methods for component-based systems.

Section 3: Performance Evaluation Methods
Description 3: This section summarizes the main approaches to performance evaluation for component-based software systems, including prediction approaches based on UML and proprietary meta-models, middleware-focused methods, and formal specification methods.

Section 4: Critical Reflection
Description 4: This section offers a critical analysis of the surveyed approaches, discussing their benefits and drawbacks, and details the factors impacting component performance and their descriptions.

Section 5: Future Directions
Description 5: This section outlines possible future research directions in the field of performance evaluation of component-based systems, including model expressiveness, support for runtime and dynamic architectures, and domain-specific methods.

Section 6: Conclusions
Description 6: This section concludes the survey, summarizing the current state of the field and the importance of developing mixed approaches that incorporate both measurements and modelings to handle the complexity of component-based systems.