{"action":"created","issue":{"url":"https://api.github.com/repos/oasp/oasp4j/issues/153","labels_url":"https://api.github.com/repos/oasp/oasp4j/issues/153/labels{/name}","comments_url":"https://api.github.com/repos/oasp/oasp4j/issues/153/comments","events_url":"https://api.github.com/repos/oasp/oasp4j/issues/153/events","html_url":"https://github.com/oasp/oasp4j/issues/153","id":53606182,"number":153,"title":"Concept for test data management","user":{"login":"Max-Goebel","id":10220824,"avatar_url":"https://avatars.githubusercontent.com/u/10220824?v=3","gravatar_id":"","url":"https://api.github.com/users/Max-Goebel","html_url":"https://github.com/Max-Goebel","followers_url":"https://api.github.com/users/Max-Goebel/followers","following_url":"https://api.github.com/users/Max-Goebel/following{/other_user}","gists_url":"https://api.github.com/users/Max-Goebel/gists{/gist_id}","starred_url":"https://api.github.com/users/Max-Goebel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Max-Goebel/subscriptions","organizations_url":"https://api.github.com/users/Max-Goebel/orgs","repos_url":"https://api.github.com/users/Max-Goebel/repos","events_url":"https://api.github.com/users/Max-Goebel/events{/privacy}","received_events_url":"https://api.github.com/users/Max-Goebel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-01-07T07:43:54Z","updated_at":"2015-01-07T09:49:27Z","closed_at":null,"body":"Currently there is no standardized concept for managing test data in OASP applications. There must be answers for the following questions:\r\n* Should test data be shared by multiple test cases, or should each test case use its own data? Maybe there are different answers for master data and for transaction data.\r\n* How is the test object reset to a defined starting state, so that test cases can be run? Possible solutions:\r\n * Each test case must create its own test data (and remove it after having been run). \r\n * There is a monolithic data creation script for all test cases (and a corresponding data removal script)\r\n * Hybrid: Data that is used by multiple test cases is created and removed by common scripts, all test case specific data is created and removed by the individual test cases\r\n* How do test cases / scripts create and remove data? \r\n * Via the service layer? \r\n * Via the logic layer?\r\n * Via data access layer?\r\n * Via SQL-Statements without using any layer?\r\n * Via importing a database dump?\r\n* In which format should test data be stored?\r\n * Interlinked objects, generated with dependency injection defined in spring xml?\r\n * Separate files (CSV, XML…)?\r\n * Hard coded in JUnit-Tests?\r\n* Is it better to use synthetic data or should productive data be use, wherever it is possible?\r\n* Should there be a central data base pool for all test levels (also test levels that are not in responsibility of the developers) or should be separate test data pools per test level?\r\n* How should test data be generated?\r\n * No generation, only manual creation?\r\n * “Throw-away-generation”. Generation scripts are not kept after data is generated\r\n * Smart, configuratable generation tools? Which tools?\r\n* How do we treat test data aging (e.g. date-fields that the system validates against the current system time)? Which wildcards are needed?\r\n* When do we update test data that is aged?\r\n * Automated data update before tests are started\r\n * On-the-fly update by each test case\r\n\r\nMaybe there are no general answers to all of these questions. At least it must be possible to give advice, in which cases which solution is the best. Furthermore, tool support is needed for managing und generating test data."},"comment":{"url":"https://api.github.com/repos/oasp/oasp4j/issues/comments/68999677","html_url":"https://github.com/oasp/oasp4j/issues/153#issuecomment-68999677","issue_url":"https://api.github.com/repos/oasp/oasp4j/issues/153","id":68999677,"user":{"login":"hohwille","id":1923119,"avatar_url":"https://avatars.githubusercontent.com/u/1923119?v=3","gravatar_id":"","url":"https://api.github.com/users/hohwille","html_url":"https://github.com/hohwille","followers_url":"https://api.github.com/users/hohwille/followers","following_url":"https://api.github.com/users/hohwille/following{/other_user}","gists_url":"https://api.github.com/users/hohwille/gists{/gist_id}","starred_url":"https://api.github.com/users/hohwille/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hohwille/subscriptions","organizations_url":"https://api.github.com/users/hohwille/orgs","repos_url":"https://api.github.com/users/hohwille/repos","events_url":"https://api.github.com/users/hohwille/events{/privacy}","received_events_url":"https://api.github.com/users/hohwille/received_events","type":"User","site_admin":false},"created_at":"2015-01-07T09:49:27Z","updated_at":"2015-01-07T09:49:27Z","body":"First of all we should separate master data from transaction data as you suggested.\r\nAs we are proposing KISS and address the 80%-90% projects it is suitable to assume that there is one set of master data that can evolve over the time (but we do not consider multi-tenancy or the like where we a totally different sets of master-data). For DDL and master-data we are using Flyway. Hence, we can consider this aspect as solved and we strongly recommend that master-data is immutable. \r\nFurther we use a concept of reserved ID ranges with low IDs for master-data and initialize hibernate sequence currently in our sample with 1000000. So all transaction-data created at runtime will have IDs >= 1000000. We can reserve a range 500000 <= x < 1000000 for testdata. \r\nThen we could have a generic script to drop all testdata or all transaction-data created by a test-case but still we have to consider some issues with referential integrity.\r\n\r\nMy few cents on test-data needs as prerequisite for multiple test-cases:\r\nMaintaining test-data in external formats related to the DDL/DB Schema (no matter if SQL, CSV or XML) is IMHO causing a lot of maintenance costs as the DDL of real-live projects evolves and test-data gets broken. A more stable way is therefore to use stable services of the application itself to create the data. If for performance reasons this might be too slow for nightly builds of integration test runs some SQL or whatever can be generated from that and committed. But if DDL changes it can be regenerated automatically.\r\n\r\nIndividual JUnits can simply hardcode simple values that they use as \"test-data\" (this is IMHO something totally different to prerequisite test-data to run test-cases on). However, if there are complex permutation tests for business rules or the like you shall consider using XLS(X) or CSV instead of hard-coding the data.\r\n\r\nAccording to KISS we should recommend to use one set of test-data that is provided in advance before integration tests are started. If a project is more complex and requires different pools of test-data then it should find a solution for itself but in most cases this only makes the world complicated and can be avoided by a KISS approach.\r\n\r\nTest data aging is also an interesting aspect. So far we have no use-case for that in our simple sample application but we might face that in the future (see issues for advanced usecases). We should first address the more urgent aspects before this one."}}