{"push_id":841076690,"size":2,"distinct_size":2,"ref":"refs/heads/zfstest-report","head":"3851348fe7319fd0744da014833dc27a6ef9e37b","before":"f5976d63a16e2fc97f7910d364411288101e897c","commits":[{"sha":"2560214873326115ead717e141b8ae2907bb5d18","author":{"email":"cdde3122786f21ced358c7dfe3d192f3c508fd50@delphix.com","name":"Prakash Surya"},"message":"Update jobs to test the \"zfstest-report\" branch","distinct":true,"url":"https://api.github.com/repos/prakashsurya/openzfs-build/commits/2560214873326115ead717e141b8ae2907bb5d18"},{"sha":"3851348fe7319fd0744da014833dc27a6ef9e37b","author":{"email":"cdde3122786f21ced358c7dfe3d192f3c508fd50@delphix.com","name":"Prakash Surya"},"message":"Add script to parse the ZFS test suite results\n\nThis patch is intended to make the results of the \"openzfs-run-zfs-test\"\neasier to consume. Currently the \"openzfs-run-zfs-test\" job will always\nreport success, even if there are failed tests within the ZFS test suite\nrun. This makes it a very involved process to determine if a given run\nhas uncovered any new failures or not.\n\nThis patch attempts to address this by adding a script that will\npost-process the results of all the individual ZFS tests, and report an\nerror if there are new failed tests that cropped up during any given\nrun. In addition, since there are a few known tests to fail, these\nspecific failures will not cause the \"openzfs-run-zfs-test\" job to fail.\nInstead, these specific failures are reported but exempt from causing\nthe Jenkins job to fail, and any unexpected results (e.g. unexpected\nfailures, or passing tests that are expected to fail) will result in the\nJenkins job to fail.\n\nAdditionally, the script that parses the results will also print a more\ncompact summary which is intended to be consumed by humans. Here's an\nexample of the scripts output:\n\n    $ python zfstest-report.py results.txt\n\n    Tests with results other than PASS that are expected:\n        FAIL cli_root/zfs_property/zfs_written_property_001_pos\n        FAIL cli_root/zfs_mount/zfs_mount_all_001_pos\n        FAIL mdb/mdb_001_pos\n        FAIL rsend/rsend_009_pos\n        FAIL zvol/zvol_swap/zvol_swap_004_pos\n        FAIL rootpool/rootpool_002_neg\n        FAIL acl/nontrivial/zfs_acl_chmod_inherit_003_pos\n        FAIL cli_root/zfs_mount/zfs_mount_009_neg\n        FAIL grow_replicas/grow_replicas_001_pos\n        FAIL rsend/rsend_008_pos\n        FAIL cli_root/zfs_snapshot/zfs_snapshot_009_pos\n        FAIL refreserv/refreserv_004_pos\n        FAIL cli_root/zpool_get/zpool_get_002_pos\n\n    Tests with results other than PASS that are unexpected:\n\n    Tests with result of PASS that are unexpected:\n\n    Percent passed: 98.53%\n    Log directory:  /var/tmp/test_results/20151025T072352","distinct":true,"url":"https://api.github.com/repos/prakashsurya/openzfs-build/commits/3851348fe7319fd0744da014833dc27a6ef9e37b"}]}