{"action":"created","issue":{"url":"https://api.github.com/repos/bitcoin/bitcoin/issues/6813","labels_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/6813/labels{/name}","comments_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/6813/comments","events_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/6813/events","html_url":"https://github.com/bitcoin/bitcoin/pull/6813","id":111065182,"number":6813,"title":"Support gathering code coverage data for RPC tests with lcov","user":{"login":"dexX7","id":5836089,"avatar_url":"https://avatars.githubusercontent.com/u/5836089?v=3","gravatar_id":"","url":"https://api.github.com/users/dexX7","html_url":"https://github.com/dexX7","followers_url":"https://api.github.com/users/dexX7/followers","following_url":"https://api.github.com/users/dexX7/following{/other_user}","gists_url":"https://api.github.com/users/dexX7/gists{/gist_id}","starred_url":"https://api.github.com/users/dexX7/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dexX7/subscriptions","organizations_url":"https://api.github.com/users/dexX7/orgs","repos_url":"https://api.github.com/users/dexX7/repos","events_url":"https://api.github.com/users/dexX7/events{/privacy}","received_events_url":"https://api.github.com/users/dexX7/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/bitcoin/bitcoin/labels/Tests","name":"Tests","color":"d4c5f9"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-10-12T22:10:03Z","updated_at":"2015-10-15T15:51:38Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/bitcoin/bitcoin/pulls/6813","html_url":"https://github.com/bitcoin/bitcoin/pull/6813","diff_url":"https://github.com/bitcoin/bitcoin/pull/6813.diff","patch_url":"https://github.com/bitcoin/bitcoin/pull/6813.patch"},"body":"Bitcoin Core has a nice way to gather code coverage data, though it only runs the unit tests and (optionally) the BitcoinJ tests.\r\n\r\nWith this update, the RPC tests (via `qa/pull-tester/rpc-tests.py`) are also executed, when gathering code coverage data with `make cov`.\r\n\r\nWhen cleaning, there were several leftovers, and only the coverage related files in `src/` were removed, while the ones in the other dirs remained. The leftovers from tests are now also removed, whereby `qa/tmp/` is related to the BitcoinJ tests, and `cache/` is related to RPC tests.\r\n\r\nBecause Python is used to run the RPC tests, it is explicitly checked, whether Python is available.\r\n\r\nThe configuration option `--enable-extended-rpc-tests` may be used to enable extended RPC tests, and the configuration option `--enable-comparison-tool-reorg-tests` may be used to enable extended tests via BitcoinJ. Note that the extended tests can take some time.\r\n\r\n---\r\n\r\nHow to:\r\n\r\nGenerating coverage data requires `lcov`, which may be installed with:\r\n```\r\nsudo apt-get install lcov\r\n```\r\n\r\nThere are a few [configuration options](http://ltp.sourceforge.net/coverage/lcov/lcovrc.5.php), but no further setup is necessary.\r\n\r\nTo include the BitcoinJ tests, get the test tool:\r\n```\r\nTOOL_URL=https://github.com/theuni/bitcoind-comparisontool/raw/master/pull-tests-8c6666f.jar\r\nTOOL_HASH=a865332b3827abcde684ab79f5f43c083b0b6a4c97ff5508c79f29fee24f11cd\r\nwget $TOOL_URL -O ./share/BitcoindComparisonTool.jar\r\necho \"$TOOL_HASH  ./share/BitcoindComparisonTool.jar\" | shasum --algorithm 256 --check\r\n```\r\nThe coverage data can then be gathered with:\r\n```\r\n./autogen.sh\r\n./configure --enable-lcov --with-comparison-tool=./share/BitcoindComparisonTool.jar\r\nmake\r\nmake cov\r\n```\r\n\r\nIt runs the tests and generates two HTML reports:\r\n\r\n - `test_bitcoin.coverage/index.html`\r\n - `total.coverage/index.html`\r\n\r\n---\r\n\r\nExample report:\r\n\r\n- http://bitwatch.co/uploads/lcov-total-5e75bae983/\r\n\r\n(only line and function coverage, no branch coverage, without extended tests)"},"comment":{"url":"https://api.github.com/repos/bitcoin/bitcoin/issues/comments/148430077","html_url":"https://github.com/bitcoin/bitcoin/pull/6813#issuecomment-148430077","issue_url":"https://api.github.com/repos/bitcoin/bitcoin/issues/6813","id":148430077,"user":{"login":"jamesob","id":73197,"avatar_url":"https://avatars.githubusercontent.com/u/73197?v=3","gravatar_id":"","url":"https://api.github.com/users/jamesob","html_url":"https://github.com/jamesob","followers_url":"https://api.github.com/users/jamesob/followers","following_url":"https://api.github.com/users/jamesob/following{/other_user}","gists_url":"https://api.github.com/users/jamesob/gists{/gist_id}","starred_url":"https://api.github.com/users/jamesob/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jamesob/subscriptions","organizations_url":"https://api.github.com/users/jamesob/orgs","repos_url":"https://api.github.com/users/jamesob/repos","events_url":"https://api.github.com/users/jamesob/events{/privacy}","received_events_url":"https://api.github.com/users/jamesob/received_events","type":"User","site_admin":false},"created_at":"2015-10-15T15:51:37Z","updated_at":"2015-10-15T15:51:37Z","body":"@dexX7 the console output itself certainly becomes less useful after all RPCs are covered, but I think the lasting value is the ability to easily fail a build if someone introduces a new RPC without writing tests that exercise it as a client."}}