A survey of service oriented architecture systems testing
---
paper_title: Runtime Integration and Testing for Highly Dynamic Service Oriented ICT Solutions -- An Industry Challenges Report

paper_content:

Modern Information and Communications Technology (ICT) solutions are often widely distributed and highly dynamic service oriented architectures (SOA) with stringent availability requirements. Availability implies that SOA must be reconfigured, updated and maintained during runtime, while retaining their overall operational integrity. This requires that much of the adaptation, integration, configuration and testing activities typically performed offline, during development time, now have to be done online, during runtime. Current component-based runtime platforms such as SOA realize the technological foundations for runtime reconfiguration and maintenance.However, because software engineering methodology has not kept pace with the rapid leap forward in platform technology, adequate methods, techniques and tools for dealing with runtime integration and testing are not yet available.This paper discusses the industry challenges and open issues of integrating and testing SOA infrastructures during runtime.
---
paper_title: Testing services and service-centric systems: challenges and opportunities

paper_content:

This paper provides users and system integrators with an overview of service-oriented architecture (SOA) testing's fundamental technical issues and solutions, focusing on Web services as a practical implementation of the SOA model. The paper discusses SOA testing across two dimensions: testing perspectives, wherein various stakeholders have different needs and raise different testing requirements; and testing level, wherein each SOA testing level poses unique challenges
---
paper_title: SOA Test Governance: Enabling Service Integration Testing across Organization and Technology Borders

paper_content:

The Service-Oriented Architecture (SOA) is the emerging paradigm in information technology. More than a true "architecture", SOA provides a general reference model for the development, deployment and management of distributed dynamic systems. Companies are progressively adopting service-oriented technology, because of its many (real or idealized) foreseen benefits, among which notably loose coupling and dynamic interoperability. Such benefits, however, can only be achieved through discipline and standardization: in this respect, SOA Governance qualifies a framework of policies, procedures, design rules and documentation standards to be enforced for ensuring that different services and components can successfully cooperate towards a shared business goal. What about testing of such composite SOA applications? Little attention has been devoted so far by researchers to SOA testing, but awareness is raising that existing techniques and tools are not adequate. Our position in this paper is that the establishment of a test governance framework is a key issue for enabling SOA testing at integration and system levels. We discuss the inter-relation between technical and "social" aspects of SOA application testing, and attempt a first abstract definition of a SOA Test Governance notion. We provide examples of proposed SOA testing approaches that, more or less explicitly, rely on a cross-organization agreed test process.
---
paper_title: Business Semantics Centric Reliability Testing for Web Services in BPEL

paper_content:

The Business Process Execution Language (BPEL) has become the standard in web service composition, so that been adopted by industries to create mission critical services. The business process reliability especially business process consistency when some services failed is critical to composite services especially involving various organizations. However, it is difficult to conduct business process reliability testing for composite services due to complex business process in service composition, remote deployed services and long time testing duration. Little research is done to test fault tolerance capabilities of composite services only from low level. This paper proposes a technique on how to test the reliability of composite service defined in BPEL from the view of business semantics with little cost using fault injection. We present an approach for reliability testing of web services by using service stubs with semantic faults instead of real services which can be placed at service provider side or service consumer side. Dislike existing studies, we focus on how to find the reliability problems associating with business process - called semantics as the problems are not pure coding error but faults related to business process. In addition, the behavior of composite services in BPEL is analyzed when there are faults in orchestrated services invoked. Finally, a case study is given to show the whole process of reliability testing for composite web service.
---
paper_title: Service-oriented testbed infrastructures and cross-domain federation for Future Internet research

paper_content:

This article outlines the necessity for federated service oriented testbeds in order to accommodate the requirements of Future Internet research. Extensive investigations have been carried out in the networking and grid computing area during the last decades. These activities have successfully instantiated a number of large scale collaborative network testbeds and network emulation testbeds. The article describes an architectural approach on how to design service oriented testbeds and federate them across the boundaries of administrative domains to provide testing and development platforms that serve the needs of Future Internet research not only on the networking layer but also in the service and control platform layers.
---
paper_title: Cloud Services Testing: An Understanding

paper_content:

There is a mammoth quantity of Cloud Services being deployed on the Cloud, nowadays. Everyday users and customers use these services to fulfil their needs. The use of Cloud Services is getting more and more complex with the pace of time. It is a possibility that several recurrent service requests cannot be fulfilled using just one Cloud service. Cloud services are usually composed manually, which is a time consuming and monotonous task. We can find several numbers of successful methods for automatic Cloud service composition, the main issue with that is the lack of test environment with some standards to compare and evaluate these methods. This research work is about a short survey to explore Cloud Services testing methods. This study compares several software testing researches and pose questions for further research work to find Cloud suited testing techniques for the software testers. This survey paper poses few questions to the Cloud computing research community to concentrate and find suited answers for software testing community.
---
paper_title: Test-Data Generation for Web Services Based on Contract Mutation

paper_content:

Software testing is one of the most important techniques used to assure the quality of Web Services at present. Test-data generation is an important topic in Web Services testing. The quality of test data will influence the efficiency and cost when testing Web Services. Based on the contract-based mutation testing technique, this paper presents a method of automated test-data generation for Web Services. First, according to the description information and contracts in WSDL documents of Web Services, initial test data are generated automatically by the random method. Then the test data are selected using contract mutation testing. This method can generate a test suite meeting a certain contract mutation score, which indicates the quality and efficiency of testing. Finally, we have developed a prototype on the Microsoft .NET platform, and carried out some experiments. The results have shown that the proposed method is effective in automated test-data generation for Web Services.
---
paper_title: An Improved Test Case Generation Method for Web Service Testing from WSDL-S and OCL with Pair-Wise Testing Technique

paper_content:

Web Services are popular solutions that have been in recent years to implement software in every business domain. They provide an effective way to reuse functionality, which reduces development time and cost, and increases software reliability. Software testing is a key issue of Web Services development in order to ensure quality of services. Software testing aims to discover faults or defects in software to determine where its behaviors do not correspond with software requirements or test case specifications. Software testing is used to evaluate functional correctness, reliability, robustness, and performance. We describe a Web Service contract with a Web Service Semantics Language (WSDL-S) and the Object Constraint Language (OCL). WSDL-S can be used to identify pre- and post- conditions of Web Service operation by referring service rule from OCL. This paper presents an approach for generating Web Service test cases using WSDL-S and OCL, while test case generation method is a pair-wise testing technique.
---
paper_title: WSDL-Based Automated Test Data Generation for Web Service

paper_content:

With the increase of the popularity of Web service, more and more Web applications are developed with this new kind of components. Web service quality validation and control is becoming very critical to vendors, brokers and application builders. Both brokers and users usually have to use black-box testing because design and implementation details of Web service are unavailable. This paper proposes a formal approach for automated test data generation for Web service single operation based on WSDL specification. Firstly, we define a formal model for the datatype of input element of the operation. Secondly, the paper presents an algorithm that derives the model from WSDL. Thirdly, the paper presents a method that generates test data for operations from the model. Examples of using this approach are given in order to give evidence of its usefulness.
---
paper_title: WS-TAXI: A WSDL-based Testing Tool for Web Services

paper_content:

Web Services (WSs) are the W3C-endorsed realization of the Service-Oriented Architecture (SOA). Since they are supposed to be implementation-neutral, WSs are typically tested black-box at their interface. Such an interface is generally specified in an XML-based notation called the WS Description Language (WSDL). Conceptually, these WSDL documents are eligible for fully automated WS test generation using syntax-based testing approaches. Towards such goal, we introduce the WS-TAXI framework, in which we combine the coverage of WS operations with data-driven test generation. In this paper we present an early-stage implementation of WS-TAXI, obtained by the integration of two existing softwares: soapUI, a popular tool for WS testing, and TAXI, an application we have previously developed for the automated derivation of XML instances from a XML schema. WS-TAXI delivers a complete suite of test messages ready for execution. Test generation is driven by basic coverage criteria and by the application of some heuristics. The application of WS-TAXI to a real case study gave encouraging results.
---
paper_title: WSDL-based automatic test case generation for Web services testing

paper_content:

Web services promote the specification based cooperation and collaboration among distributed applications in an open environment. To ensure the quality of the services that are published, bound, invoked and integrated at runtime, test cases have to be automatically generated and testing executed, monitored and analyzed at runtime. This paper presents the research to generate Web services test cases automatically based on the Web services specification language WSDL (Web Services Description Language), which carries the basic information of a service including its interface operations and the data transmitted. The WSDL file is first parsed and transformed into the structured DOM tree. Then, test cases are generated from two perspectives: test data generation and test operation generation. Test data are generated by analyzing the message data types according to standard XML schema syntax. Operation flows are generated based on the operation dependency analysis. Three types of dependencies are defined: input dependency, output dependency, and input/output dependency. Finally, the generated test cases are documented in XML based test files called service test specification.
---
paper_title: Testing WSDL_Based Web Service Automatically

paper_content:

Testing to ensure service QoS is a key problem for Web Service adoption. One of the key problems for software testing is designing the finite test suit tailored to test case generation efficiently. This paper focuses on Abstract Test Suite and Executable Test Suite analysis of WSDL_based Web Service and proposes automated WSDL_based Web Service testing. A XML_based test scripting language is discussed and Executable Test Suite can be derived from Abstract Test Suite automatically based on XML parser. Executable Test Suite consists of message Schema syntactic test group, Web Service interface test group, and interoperation test group. A test process is used in WSDL_based Web Service testing. The three key techniques adopted by the test process including the test case generation, test flow, and test data generation are analyzed.
---
paper_title: Automatic Test Case Generation for web service processes Using a SAT Solver

paper_content:

Such useful properties of web services as access from any platform, great interoperability with other web services, ability to combine several web services into a larger application relatively quickly have made them an important category of software systems. One of the techniques used to increase the quality of software is testing. The adequacy of test cases and possible automation of the testing process greatly influence the quality of the produced software and timeliness of the software development process. Even though a great deal of work has been done in adapting test case generation techniques to the peculiarities of web services (e.g. [11][12][13]) we believe our work makes a useful contribution in this area. This paper proposes a novel approach to generate test cases based on the process definition model of a web service. A process definition model defines a sequence of activities that can be performed by orchestrating the capabilities of a web service. A SAT solver (such as Alloy [10]) is used to extract the paths from the process definition model. These paths are used to generate test case specifications that will test all web service capabilities involved in a process. In our opinion the main contribution of the work is an application of a static analysis method for generation of test cases for a web service guided by a goodness metric of process coverage.
---
paper_title: Testing BPEL with Stream X-Machine

paper_content:

To generate test cases for the unit testing of business process written in BPEL, developers have to prepare input data for the BPEL process under test (PUT) and verification conditions for output data from the PUT. It could be a tedious task due to the complexity of the PUT which describes the Web flow of a distributed collaboration of individual service providers executing concurrently. This paper shows how Stream X-machine based testing techniques could be applied to automatically generate test cases for BPEL process.
---
paper_title: An EMF activity tree based BPEL defect pattern testing method

paper_content:

For testing BPEL defects efficiently, a novel BPEL defect pattern testing architecture based on the EMF activity tree technology is proposed. The EMF activity tree that is similar to abstract syntax tree is used to describe the BPEL service process structure. The mapping method from the DOM object tree of a BPEL file to the EMF activity tree and the recursive algorithm to generate an EMF activity tree are represented in detail. A typical EMF activity tree is shown and the visitor design pattern based traversal method is stated. The directions to enhance this technology are illustrated finally.
---
paper_title: Basis path test suite and testing process for WS-BPEL

paper_content:

Web services technology offers a WS-BPEL language for business process execution. The building block of WS-BPEL is those Web services components that collaborate to realize a certain function of the business process. Applications can now be built more easily by composing existing Web services into workflows; each workflow itself is also considered a composite Web service. As with other programs, basis path testing can be conducted on WS-BPEL processes in order to verify the execution of every node of the workflow. This paper discusses the generation of the test suite for basis path testing of WS-BPEL and an accompanying tool that can be used by service testers. The test suite consists of test cases, stubs of the constituent services in the workflow, and auxiliary state services that assist in the test; these are deployed when running a test on a particular WS-BPEL. The paper presents also a testing process for service testers. A business process of a market place is discussed as a case study.
---
paper_title: A model driven approach toward business process test case generation

paper_content:

Web service based business process has become the backbone of enterprise information system, and it is evolving all the time. Therefore, the process has to be tested thoroughly and repeatedly whenever it is changed. This paper proposes a model driven approach toward generating executable test case from the given business process. The approach is composite of three stages: defining a process under test based on the business process model, transforming from process under test to abstract test cases and from abstract test cases to executable test cases. To implement the approach, a business process model based on the BPEL specification and UML2.0 activity diagram and a test case model based on U2TP framework and TTCN-3 key concepts are defined to have the ability to define the business process and the abstract test case. In order to validate the approach, a loan request process is used to show how the approach work and how the test cases are generated.
---
paper_title: DFTT4CWS: A Testing Tool for Composite Web Services Based on Data-Flow

paper_content:

For large and complex composite Web services which are composed by BPEL and a set of WSDL and XSD documents, we urgently need the aid of the tools in designing and selecting test cases. However, most of mature testing tools which generate test data with the information of WSDL are used for single service while the research for composition remains in the realm of theory. So we apply the theory of data-flow testing and try to develop a testing tool for Web services composition which can automatically detects harmful data-flow anomalies and generate test paths according to all kinds of data flow coverage criteria.
---
paper_title: An Automatic Test Case Generation Framework for Web Services

paper_content:

BPEL (Business Process Execution Language) as a de-facto standard for web service orchestration has drawn particularly attention from researchers and industries. BPEL is a semi-formal flow language with complex features such as concurrency and hierarchy. To test a model thoroughly, we need to cover different execution scenarios. As is well known, it is tedious, time consuming, and error prone to design test cases manually, especially for complex modelling languages. Hence, it is desirable to apply existing model-based-testing techniques in the domain of web services. We proposed WSA (Web Service Automata) to be the operational semantics for BPEL. Based on WSA, we propose a model checking based test case generation framework for BPEL. The SPIN and NuSMV model checkers are used as the test generation engine, and the conventional structural test coverage criteria are encoded into LTL and CTL temporal logic. State coverage and transition coverage are used for BPEL control flow testing, and all-du-path coverage is used for BPEL data flow testing. Two levels of test cases can be generated to test whether the implementation of web services conforms to the BPEL behaviour and WSDL interface models. The generated test cases are executed on the JUnit test execution engine.
---
paper_title: A Graph-Search Based Approach to BPEL4WS Test Generation

paper_content:

Business Process Execution Language for Web Services (BPEL4WS) is a kind of concurrent programming languages with several special features that raise special challenges for verification and testing. This paper proposes a graph-search based approach to BPEL test case generation, which effectively deals with BPEL concurrency semantics. This approach defines an extension of CFG (Control Flow Graph) - BPEL Flow Graph (BFG) - to represent a BPEL program in a graphical model. Then concurrent test paths can be generated by traversing the BFG model, and test data for each path can be generated using a constraint solving method. Finally test paths and data are combined into complete test cases.
---
paper_title: A context-aware orchestrating and choreographic test framework for service-oriented applications

paper_content:

Service orientation is an approach to developing software applications by means of service orchestration and choreography. In such applications, a service interacts with not only other services but also its embedding environment via contexts. However, it is still unclear about how to develop effective testing techniques for service orchestration and choreography, particularly in the presence of contexts. In this article, we present a context-aware framework for testing service-oriented applications.
---
paper_title: Efficient Testing of Service-Oriented Applications Using Semantic Service Stubs

paper_content:

Service-oriented applications can be expensive to test because services are hosted remotely, are potentially shared among many users, and may have costs associated with their invocation. In this paper, we present an approach for reducing the costs of testing such applications. The key observation underlying our approach is that certain aspects of an application can be tested using locally deployed semantic service stubs, instead of actual remote services.A semantic service stub incorporates some of the service functionality, such as verifying preconditions and generating output messages based on post conditions. We illustrate how semantic stubs can enable the client test suite to be partitioned into subsets, some of which need not be executed using remote services. We also present a case study that demonstrates the feasibility of the approach, and potential cost savings for testing. The main benefits of our approach are that it can (1) reduce the number of test cases that need to be run to invoke remote services, (2) ensure that certain aspects of application functionality are well-tested before service integration occurs.
---
paper_title: Bringing white-box testing to Service Oriented Architectures through a Service Oriented Approach

paper_content:

The attractive feature of Service Oriented Architecture (SOA) is that pieces of software conceived and developed by independent organizations can be dynamically composed to provide richer functionality. The same reasons that enable flexible compositions, however, also prevent the application of some traditional testing approaches, making SOA validation challenging and costly. Web services usually expose just an interface, enough to invoke them and develop some general (black-box) tests, but insufficient for a tester to develop an adequate understanding of the integration quality between the application and the independent web services. To address this lack we propose an approach that makes web services more transparent to testers through the addition of an intermediary service that provides coverage information. The approach, named Service Oriented Coverage Testing (SOCT), provides testers with feedback about how much a service is exercised by their tests without revealing the service internals. In SOCT, testing feedback is offered itself as a service, thus preserving SOA founding principles of loose coupling and implementation neutrality. In this paper we motivate and define the SOCT approach, and implement an instance of it. We also perform a study to asses SOCT feasibility and provide a preliminary evaluation of its viability and value.
---
paper_title: WebMov: A Dedicated Framework for the Modelling and Testing of Web Services Composition

paper_content:

This paper presents a methodology and a set of tools for the modelling, validation and testing of Web service composition, conceived and developed within the French national project WebMov. This methodology includes several modelling techniques, based mainly on some variations of Timed Extended Finite State Machines (TEFSM) formalism, which provide a formal model of the BPEL description of Web services composition. These models are used as a reference for the application of different test generation and passive testing techniques for conformance and robustness checking. The whole WebMov methodology is integrated within a dedicated framework, composed by a set of tools that implement the model representation, the test generation and passive testing algorithms. This framework also permits the interaction of these tools to achieve specific modelling and testing activities in a complementary way. A case study based on a real service, a Travel Reservation Web Service, is presented as well as the results of the application of the proposed WebMov methodology and tools.
---
paper_title: Tabu Search and Genetic Algorithm to Generate Test Data for BPEL Program

paper_content:

To solve the combination explosion and the undecidable problem on feasibility of complete computation paths when testing the composed Web services programmed by Business Process Execution Language (BPEL), this paper presents a simple strategy for generating complete computation paths of BPEL program and an approach for generating test data for feasible complete computation paths based on tabu search and genetic algorithm (Tabu GA). Firstly, the basic structure graph is built from BPEL program based on an open source tool BPEL4WS2OWLS. Then the basic structure graph is converted to middle structure graph for facilitating to generate complete computation paths by selected test adequacy criteria, which a complete computation path is corresponding to operation sequence. Lastly, we apply Tabu GA into searching the whole input space which is made from all operation parameters of each operation sequence, and generating test data for each feasible complete computation path. Experiments show that the method could determine efficiently whether the complete computation path is feasible or not and generate automatically test data for each feasible complete computation path of BPEL program.
---
paper_title: Automatic test case selection and generation for regression testing of composite service based on extensible BPEL flow graph

paper_content:

Services are highly reusable, flexible and loosely coupled, which makes the evolution and the maintenance of composite services more complex. Evolution of BPEL composite service covers changes of processes, bindings and interfaces. In this paper, an approach is proposed to select and generate test cases during the evolution of BPEL composite service. The approach identifies the changes by using control-flow analysis technique and comparing the paths in new composite service version and the old one using extensible BPEL flow graph (or XBFG). Message flow is appended to the control flow so that XBFG can describe the behavior of composite service integrally. The binding and predicate constraint information added in XBFG elements can be used in path selection and test case generation. Theory analysis and case study both show that the approach is effective, and test cases coverage rate is high for the changes of processes, bindings and interfaces.
---
paper_title: Automatic test case selection for regression testing of composite service based on extensible BPEL flow graph

paper_content:

Highlights? We propose a flow graph to model BPEL-based composite service. ? XBFG path is introduced and analyzed for test case selection. ? A new classification for service modification is discussed. ? Five different versions of a running example are explored. Services are highly reusable, flexible and loosely coupled components whose changes make the evolution and maintenance of composite services more complex. The changes of composite service mainly cover three types, i.e., the processes, bindings, and interfaces. In this article, an approach is proposed to select test cases for regression testing of different versions of BPEL (business process execution language) composite service where these changes are involved. The approach identifies the changes by performing control flow analysis and comparing the paths in a new version of composite service with those in the old one using a kind of eXtensible BPEL flow graph (XBFG). Message sequence is appended to XBFG path so that XBFG can fully describe the behavior of composite service. The binding and predicate constraint information added in different XBFG elements can be used for path selection and even for test case generation. Both theoretic analysis and case study show that the proposed approach is effective.
---
paper_title: SOA : Testing and Self-Checking

paper_content:

RCOST - Research Centre on Software Technology - University ofSannioPalazzo ex Poste, Via Traiano - 82100 Benevento, ItalyAbstractThe dynamic nature of service-oriented architectures poses new challenges to systemvalidation. Traditional testing is unable to cope with certain aspects of a service–orientedsystem validation, essentially because of the impossibility to test all (often unforeseen)system’s conﬁgurations. On the other hand, run–time monito ring, while able to deal withthe intrinsic dynamism and adaptiveness of a service–orien ted system, are unable to provideconﬁdence that a system will behave correctly before it is ac tually deployed.In this paper we discuss the role of testing and monitoring to validate a service–orientedsystem and how they can be combined to increase the conﬁdence and reduce the cost ofvalidation.Key words: Web Services, Service–Oriented Architecture, ServiceTesting, Service Monitoring
---
paper_title: Towards an evaluation framework for SOA security testing tools

paper_content:

Service-Oriented Architecture (SOA) is a paradigm that organizes and uses distributed capabilities to bring together a technical solution to a business problem. Despite the large and increasing dependency on SOA by the enterprise, testing SOA systems is still a nascent and immature field. In particular, testing SOA applications from a security perspective is an essential yet underserved activity. This paper presents preliminary work towards an evaluation framework for SOA security testing tools, in order to address the question “Which testing tool(s) provide(s) the best value in testing SOA security with respect to our needs and context?”
---
paper_title: Business Semantics Centric Reliability Testing for Web Services in BPEL

paper_content:

The Business Process Execution Language (BPEL) has become the standard in web service composition, so that been adopted by industries to create mission critical services. The business process reliability especially business process consistency when some services failed is critical to composite services especially involving various organizations. However, it is difficult to conduct business process reliability testing for composite services due to complex business process in service composition, remote deployed services and long time testing duration. Little research is done to test fault tolerance capabilities of composite services only from low level. This paper proposes a technique on how to test the reliability of composite service defined in BPEL from the view of business semantics with little cost using fault injection. We present an approach for reliability testing of web services by using service stubs with semantic faults instead of real services which can be placed at service provider side or service consumer side. Dislike existing studies, we focus on how to find the reliability problems associating with business process - called semantics as the problems are not pure coding error but faults related to business process. In addition, the behavior of composite services in BPEL is analyzed when there are faults in orchestrated services invoked. Finally, a case study is given to show the whole process of reliability testing for composite web service.
---
paper_title: Dynamic monitoring framework for the SOA execution environment

paper_content:

The paper analyses the challenges involved in constructing a dynamic monitoring framework compliant with the requirements of SOA application monitoring. The specification of these requirements provides a starting point for our multilayer framework architecture. We describe its Monitoring Scenario and Instrumentation layers in detail. The approach aims at runtime monitoring of container-based SOA execution environments. The Instrumentation layer exploits interceptor sockets, on top of which a powerful attribute-based reconfigurable monitoring service is constructed. The proposed solution is application-agnostic and can be used for enterprise and computational applications. We present a case study which further explains the functionality of the system. A prototype implementation for OSGi containers is also briefly described. Preliminary performance evaluation results are outlined and discussed.
---
paper_title: Improving data perturbation testing techniques for Web services

paper_content:

The widespread use of service-oriented architectures (SOAs) and Web services in commercial software requires the adoption of development techniques to ensure the quality of Web services. Testing techniques and tools concern quality and play a critical role in accomplishing quality of SOA based systems. Existing techniques and tools for traditional systems are not appropriate to these new systems, making the development of Web services testing techniques and tools required. This article presents new testing techniques to automatically generate a set of test cases and data for Web services. The techniques presented here explore data perturbation of Web services messages upon data types, integrity and consistency. To support these techniques, a tool (GenAutoWS) was developed and applied to real problems.
---
paper_title: Automatic Timed Test Case Generation for Web Services Composition

paper_content:

In order to specify the composition of Web services, WSBPEL was defined as an orchestrating language by an international standards consortium. In this paper, we propose a method to test composite Web service described in BPEL. As a first step, the BPEL specification is transformed into an Intermediate Format (IF) model that is based on timed automata, which enables modeling of timing constraints.We defined a conformance relation between two timed automata(of implementation and specification) and then proposed an algorithm to generate test cases. Test case generation is based on simulation where the exploration is guided by test purposes. The proposed method was implemented in a set of tools which were applied to a common Web service as a case study.
---
paper_title: A Composable Framework for Test Automation of Service-Based Applications

paper_content:

Service Oriented Architecture (SOA) provides a new generation of software architectures which attracts attention as promising way for smooth integration of loosely coupled software applications. In addition to all benefits, it adds new dimensions to the complexity of designing, building, testing, deploying and maintaining Service-based applications. Research in verification and validation applied to compositions of web services still is not mature enough. In this paper we present a framework named TASSA for testing and validation of both functional and non-functional behavior of service-based applications. TASSA provides end-to-end testing of all three layers (Service layer, Service Composition and Coordination, Business Process) of service-based applications. The framework is based on SOA principles and is composable depending on user requirements. The software developers and particularly application integrators are lacking practical testing suites and respective testing methodologies to be used both at design time and run time of the service-based applications. TASSA framework has the goal to be both practical and complete in order to satisfy these needs. It aims to reduce the cost and improve the effectiveness of testing Service-based applications by automating the testing process.
---
paper_title: WSOTF: An Automatic Testing Tool for Web Services Composition

paper_content:

This paper presents an automatic conformance testing tool with timing constraints from a formal specification (TEFSM: Timed Extended Finite State Machine) of web services composition (WSOTF: Web Service composition, Online Testing Framework), that is implemented by an online testing algorithm. This algorithm combines simultaneously idea of test execution and debug to generate and simultaneously execute the test cases. In this tool, the complete test scenario (timed test case suite and data) is built during test execution. This tool focus on unit testing, it means that only the service composition under test is tested and all its partners will be simulated by the WSOTF. This tool also considers the timing constraints and synchronous time delay. We can also use this tool for debug that is not easy while we develop a composite of Web services.
---

Title: A Survey of Service Oriented Architecture Systems Testing

Section 1: INTRODUCTION
Description 1: Introduce the concept of SOA and outline the purpose of the paper, the structure, and the areas of focus which include testing challenges and existing solutions.

Section 2: SOA SYSTEMS TESTING CHALLENGES
Description 2: Discuss the unique challenges of testing SOA systems, highlighting issues related to dynamicity, service code observability, control, trust, new testing aspects, test cost, and different stakeholders.

Section 3: Functional Testing Challenges
Description 3: Detail the specific challenges associated with functional testing in SOA systems, including difficulties with input/output, asynchronous operations, system size, interface issues, and complex test data generation.

Section 4: Non-Functional Testing Challenges
Description 4: Describe the challenges related to non-functional testing in SOA systems, focusing on availability, performance, applicability, maintenance, SLA parameters, security, and test reliability.

Section 5: Existing Solutions 
Description 5: Enumerate the solutions and methodologies proposed in existing research to address functional and non-functional testing challenges in SOA systems, including automation, monitoring, ESB capabilities, and improvements in SOA testbeds.

Section 6: Summarized
Description 6: Summarize the findings related to SOA testing methods, showing different levels and aspects of test coverage in tables.

Section 7: SOA SYSTEMS TESTING
Description 7: Outline the methods of SOA system testing in both functional and non-functional areas, detailing unit testing, integration testing, regression testing, and the role of automatic test data generation.

Section 8: Unit Testing
Description 8: Discuss the research and methods pertaining to unit testing of SOA systems, focusing on automation and WSDL-based testing.

Section 9: Integration Testing
Description 9: Provide insights into how integration testing is performed in SOA systems, including BPEL-based testing, test frameworks, data flow testing, and the generation of test cases.

Section 10: Regression Testing
Description 10: Explain how regression testing is done in SOA systems using approaches like XBFG and checking service evolution.

Section 11: Non-Functional Testing 
Description 11: Address non-functional requirements and mechanisms in SOA testing, highlighting performance monitoring, security evaluations, and reliability testing using fault injection.

Section 12: Automatic testing tools
Description 12: Review various tools developed for the automatic testing of SOA systems, including frameworks for functional and non-functional specifications, conformance testing, and visualization mechanisms.

Section 13: CONCLUSIONS
Description 13: Conclude by summarizing the SOA systems testing challenges and solutions, emphasizing the need for dynamic and automatic test case generation. Suggest future research directions and improvements in testing approaches.