{"action":"created","issue":{"url":"https://api.github.com/repos/cms-sw/cmssw/issues/10014","labels_url":"https://api.github.com/repos/cms-sw/cmssw/issues/10014/labels{/name}","comments_url":"https://api.github.com/repos/cms-sw/cmssw/issues/10014/comments","events_url":"https://api.github.com/repos/cms-sw/cmssw/issues/10014/events","html_url":"https://github.com/cms-sw/cmssw/pull/10014","id":92595753,"number":10014,"title":"Fix MultiTrackValidator efficiency summary and efficiency vs. deltaR plots","user":{"login":"makortel","id":33993,"avatar_url":"https://avatars.githubusercontent.com/u/33993?v=3","gravatar_id":"","url":"https://api.github.com/users/makortel","html_url":"https://github.com/makortel","followers_url":"https://api.github.com/users/makortel/followers","following_url":"https://api.github.com/users/makortel/following{/other_user}","gists_url":"https://api.github.com/users/makortel/gists{/gist_id}","starred_url":"https://api.github.com/users/makortel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/makortel/subscriptions","organizations_url":"https://api.github.com/users/makortel/orgs","repos_url":"https://api.github.com/users/makortel/repos","events_url":"https://api.github.com/users/makortel/events{/privacy}","received_events_url":"https://api.github.com/users/makortel/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/analysis-pending","name":"analysis-pending","color":"fbca04"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/comparison-pending","name":"comparison-pending","color":"fef2c0"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/dqm-pending","name":"dqm-pending","color":"fbca04"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/geometry-pending","name":"geometry-pending","color":"fbca04"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/orp-pending","name":"orp-pending","color":"fbca04"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/pending-signatures","name":"pending-signatures","color":"eb6420"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/reconstruction-pending","name":"reconstruction-pending","color":"fbca04"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/simulation-pending","name":"simulation-pending","color":"fbca04"},{"url":"https://api.github.com/repos/cms-sw/cmssw/labels/tests-started","name":"tests-started","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/cms-sw/cmssw/milestones/55","html_url":"https://github.com/cms-sw/cmssw/milestones/Next%20CMSSW_7_6_X","labels_url":"https://api.github.com/repos/cms-sw/cmssw/milestones/55/labels","id":1176788,"number":55,"title":"Next CMSSW_7_6_X","description":"","creator":{"login":"smuzaffar","id":4115138,"avatar_url":"https://avatars.githubusercontent.com/u/4115138?v=3","gravatar_id":"","url":"https://api.github.com/users/smuzaffar","html_url":"https://github.com/smuzaffar","followers_url":"https://api.github.com/users/smuzaffar/followers","following_url":"https://api.github.com/users/smuzaffar/following{/other_user}","gists_url":"https://api.github.com/users/smuzaffar/gists{/gist_id}","starred_url":"https://api.github.com/users/smuzaffar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/smuzaffar/subscriptions","organizations_url":"https://api.github.com/users/smuzaffar/orgs","repos_url":"https://api.github.com/users/smuzaffar/repos","events_url":"https://api.github.com/users/smuzaffar/events{/privacy}","received_events_url":"https://api.github.com/users/smuzaffar/received_events","type":"User","site_admin":false},"open_issues":70,"closed_issues":78,"state":"open","created_at":"2015-06-23T07:27:04Z","updated_at":"2015-07-02T12:25:37Z","due_on":null,"closed_at":null},"comments":4,"created_at":"2015-07-02T09:41:01Z","updated_at":"2015-07-02T12:50:07Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/cms-sw/cmssw/pulls/10014","html_url":"https://github.com/cms-sw/cmssw/pull/10014","diff_url":"https://github.com/cms-sw/cmssw/pull/10014.diff","patch_url":"https://github.com/cms-sw/cmssw/pull/10014.patch"},"body":"The recently (#9201) added efficiency summary plot (average efficiency for each input track collection) in MultiTrackValidator has a bug in putting also out-of-time TrackingParticles to the denominator. Also the computation of the DeltaR of minimum distance between any TP pair includes OOT TPs. This PR fixes both with the following developments (superseding #9956)\r\n\r\n* Add new `intimeOnly` flag to TrackingParticleSelector (with value False set to everywhere to not to change current behaviour)\r\n* Switch this flag to True for the TP selector used in MTV loop over TPs for efficiency plots to reject OOT TPs there\r\n   * The efficiency denominators themselves are fine already, as all TP selectors used in `MTVHistoProducerAlgoForTracker::fill_recoAssociated_simTrack_histos()` already have `signalOnly=True`\r\n   * This change alone would be enough to fix both of the issues with OOT TPs, but has a knock-on effect on histograms in `Tracking/Track/<iter>/simulation` which have also been filled from OOT TPs, and actually for those keeping more or less the current behaviour would be nice. Therefore, \r\n* The histograms in `Tracking/Track/<iter>/simulation` are moved to `Tracking/Track/simulation` and filled only once per MTV instance (they are anyway the same for each iteration), and now from the set of all TPs without any selection (previously only those passing the TP selector in bullet 2 were used)\r\n   * `bunchxSIM` is filled from both OOT and in-time TPs, while the others are filled only from IT TPs\r\n* In addition, the same selection as for efficiency vs. DeltaR (same as efficiency vs. eta) is applied for efficiency summary plot, but now in two flavours: one with the pT cut (pT > 0.9 GeV) and other without pT cut.\r\n\r\nTested in 750pre6, expecting changes in `effic_vs_coll`, `num_simul_coll`, `num_assoc(simToReco)_coll`, and `globalEfficiencies` under `Tracking/Track` folder (`num_*_coll` also in other MTV instances), and `effic_vs_dr`, `num_simul_dr`, and `num_assoc(simToReco)_dr` in all MTV instances (in this case there is no change in the corresponding `globalEfficiencies`, as the integrals of numerator and denominator stay the same, only the DeltaR values change).\r\n\r\nIn noPU samples the efficiency summary plots should change like (black=old, red=new)\r\n![image](https://cloud.githubusercontent.com/assets/33993/8374446/b28d6e54-1bf5-11e5-8770-7d5375aca394.png)\r\n\r\nand in pileup samples like\r\n![image](https://cloud.githubusercontent.com/assets/33993/8374430/8ce7f2c8-1bf5-11e5-97c1-7308acd3d9a7.png)\r\n\r\nIn pileup samples the number of TPs and efficiency vs. DeltaR should change like (no changes expected in noPU samples)\r\n![image](https://cloud.githubusercontent.com/assets/33993/8474440/53cb2306-20b0-11e5-890e-1c6dfd266246.png)\r\n![image](https://cloud.githubusercontent.com/assets/33993/8474451/660583a4-20b0-11e5-99f2-8ce21013ca2d.png)\r\n\r\n\r\n@rovere @VinInn "},"comment":{"url":"https://api.github.com/repos/cms-sw/cmssw/issues/comments/118022409","html_url":"https://github.com/cms-sw/cmssw/pull/10014#issuecomment-118022409","issue_url":"https://api.github.com/repos/cms-sw/cmssw/issues/10014","id":118022409,"user":{"login":"makortel","id":33993,"avatar_url":"https://avatars.githubusercontent.com/u/33993?v=3","gravatar_id":"","url":"https://api.github.com/users/makortel","html_url":"https://github.com/makortel","followers_url":"https://api.github.com/users/makortel/followers","following_url":"https://api.github.com/users/makortel/following{/other_user}","gists_url":"https://api.github.com/users/makortel/gists{/gist_id}","starred_url":"https://api.github.com/users/makortel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/makortel/subscriptions","organizations_url":"https://api.github.com/users/makortel/orgs","repos_url":"https://api.github.com/users/makortel/repos","events_url":"https://api.github.com/users/makortel/events{/privacy}","received_events_url":"https://api.github.com/users/makortel/received_events","type":"User","site_admin":false},"created_at":"2015-07-02T12:50:07Z","updated_at":"2015-07-02T12:50:07Z","body":"The test should fail with a compilation error, that I catched myself too only now (the added \"intimeOnly\" parameter was not given to TrackingParticleSelector constructor in RecoMuonValidator; I don't know why my scram didn't fail on it before)."}}