{"action":"created","issue":{"url":"https://api.github.com/repos/UCATLAS/xAODAnaHelpers/issues/287","labels_url":"https://api.github.com/repos/UCATLAS/xAODAnaHelpers/issues/287/labels{/name}","comments_url":"https://api.github.com/repos/UCATLAS/xAODAnaHelpers/issues/287/comments","events_url":"https://api.github.com/repos/UCATLAS/xAODAnaHelpers/issues/287/events","html_url":"https://github.com/UCATLAS/xAODAnaHelpers/issues/287","id":111736087,"number":287,"title":"Overlap Removal Inputs and Outputs","user":{"login":"JamesSaxon","id":12751805,"avatar_url":"https://avatars.githubusercontent.com/u/12751805?v=3","gravatar_id":"","url":"https://api.github.com/users/JamesSaxon","html_url":"https://github.com/JamesSaxon","followers_url":"https://api.github.com/users/JamesSaxon/followers","following_url":"https://api.github.com/users/JamesSaxon/following{/other_user}","gists_url":"https://api.github.com/users/JamesSaxon/gists{/gist_id}","starred_url":"https://api.github.com/users/JamesSaxon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JamesSaxon/subscriptions","organizations_url":"https://api.github.com/users/JamesSaxon/orgs","repos_url":"https://api.github.com/users/JamesSaxon/repos","events_url":"https://api.github.com/users/JamesSaxon/events{/privacy}","received_events_url":"https://api.github.com/users/JamesSaxon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-16T00:48:09Z","updated_at":"2015-10-16T04:18:08Z","closed_at":null,"body":"Hi @kratsg, @mmilesi, all\r\n\r\nPer our chat on the overlap removal, I think this is the behavior I would expect, _if one had all input algorithms defined._  I am assuming that the vector\\<string\\>'s contain \"\" entries for nominal, and can figure all this out.\r\n\r\nSimplifying to electrons and jets for now -- \r\n\r\n  - No systematics are run.  The overlap removal works normally on the nominal containers.\r\n  - A single electron OR jet systematic is defined, while the other is nominal.  The tool runs the single systematic.\r\n  - A single electron systematic AND a single jet systematic are defined.  The nominal jets are missing for the electron variations and vice versa.  The tool should complain loudly if not halt.\r\n  - A single electron systematic is defined, while the jet systematic is \"All\" (or vice versa, etc.).  It's all fine and dandy for the electrons variation -- nominal exists for the jets.  But the tool should stop or complain because it cannot do the jet variations with nominal electrons.\r\n  - Both electrons and jets have \"All\" systematics defined.  All jet and electron variations and nominal are provided.\r\n\r\nThis said, now that I understand better the intended use, I'm fine just commenting out the single line of my configuration.  Let me know what you think best.  :-)\r\n\r\nJamie"},"comment":{"url":"https://api.github.com/repos/UCATLAS/xAODAnaHelpers/issues/comments/148596190","html_url":"https://github.com/UCATLAS/xAODAnaHelpers/issues/287#issuecomment-148596190","issue_url":"https://api.github.com/repos/UCATLAS/xAODAnaHelpers/issues/287","id":148596190,"user":{"login":"mmilesi","id":12515122,"avatar_url":"https://avatars.githubusercontent.com/u/12515122?v=3","gravatar_id":"","url":"https://api.github.com/users/mmilesi","html_url":"https://github.com/mmilesi","followers_url":"https://api.github.com/users/mmilesi/followers","following_url":"https://api.github.com/users/mmilesi/following{/other_user}","gists_url":"https://api.github.com/users/mmilesi/gists{/gist_id}","starred_url":"https://api.github.com/users/mmilesi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmilesi/subscriptions","organizations_url":"https://api.github.com/users/mmilesi/orgs","repos_url":"https://api.github.com/users/mmilesi/repos","events_url":"https://api.github.com/users/mmilesi/events{/privacy}","received_events_url":"https://api.github.com/users/mmilesi/received_events","type":"User","site_admin":false},"created_at":"2015-10-16T04:18:08Z","updated_at":"2015-10-16T04:18:08Z","body":"Hi @JamesSaxon , @kratsg \r\n\r\nreplies inline:\r\n\r\n> Per our chat on the overlap removal, I think this is the behavior I would expect, if one had all input algorithms defined. I am assuming that the vector<string>'s contain \"\" entries for nominal, and can figure all this out.\r\n> \r\n> Simplifying to electrons and jets for now --\r\n> \r\n> No systematics are run. The overlap removal works normally on the nominal containers.\r\n\r\nCorrect.\r\n\r\n> A single electron OR jet systematic is defined, while the other is nominal. The tool runs the single systematic.\r\n\r\nCorrect. In general, this is the case where user runs systematics on **one object type only** (can be a single syst, e.g., `EG_SCALE_LARCALIB` - and in this case the nominal electrons will not be created at all - or `All` systematics - and in this case the nominal will be included, too), and only the nominal for all the other objects. This is in general what the workflow of the analysis should be, if I am not mistaken.\r\n\r\n> A single electron systematic AND a single jet systematic are defined. The nominal jets are missing for the electron variations and vice versa. The tool should complain loudly if not halt.\r\n\r\nAs you noticed, this will generate a crash, preceded by a lot of alarming messages. But this, based on what I said before, is right what we want (right?).\r\n\r\n> A single electron systematic is defined, while the jet systematic is \"All\" (or vice versa, etc.). It's all fine and dandy for the electrons variation -- nominal exists for the jets. But the tool should stop or complain because it cannot do the jet variations with nominal electrons.\r\n\r\nThis case is pretty much as the one above: this is not how things should be done.\r\n\r\n> Both electrons and jets have \"All\" systematics defined. All jet and electron variations and nominal are provided.\r\n\r\nI haven't tried this case yet, but I assume it will not give problems, although again, it's not how the workflow is intended to be.\r\n\r\n> This said, now that I understand better the intended use, I'm fine just commenting out the single line of my configuration. Let me know what you think best. :-)\r\n\r\nI will show you what one would get from xAH when trying to run a single syst (EG_SCALE_LARCALIB) **only** for the electrons.\r\n\r\nIn a typical situation where the order of call would be:\r\n\r\n```\r\nJetCalibrator\r\nElectronCalibrator\r\nJetSelector\r\nElectronSelector\r\nOverlapRemover\r\n...\r\n```\r\n\r\nThis is the relevant configuration for:\r\n\r\n1. ElectronCalibrator\r\n\r\n```\r\nm_inputAlgoSystNames = \"\"\r\nm_outputAlgoSystNames = \"ElectronCalibrator_Syst\"\r\n```\r\n\r\n2. ElectronSelector\r\n\r\n```\r\nm_inputAlgoSystNames = \"ElectronCalibrator_Syst\"\r\nm_outputAlgoSystNames = \"ElectronSelector_Syst\"\r\n```\r\n3. OverlapRemover\r\n\r\n```\r\nm_inputAlgoElectrons = \"ElectronSelector_Syst\"\r\nm_outputAlgoElectrons = \"ElectronsOR_Syst\"\r\n\r\nm_inputAlgoJets = \"\"\r\nm_outputAlgoJets = \"AntiKt4EMTopoJetsOR_Syst\"\r\n\r\nm_inputAlgoMuons = \"\"\r\nm_outputAlgoMuons = \"MuonOR_Syst\"\r\n\r\n```\r\n\r\n(notice that the input algos for jets and muons are empty!!)\r\n\r\nBy doing like this, the following is the set of object containers the user will end up having in the TStore, and that will be available to the xAH/user algorithm downstream:\r\n\r\n```\r\nxAOD::TStore::print       INFO    Contents of transient store:\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: AntiKt4EMTopoJets_Calib\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Jet_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x14561650\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: AntiKt4EMTopoJets_CalibShallowCopy\r\nxAOD::TStore::print       INFO        Type: DataVector<xAOD::Jet_v1>\r\nxAOD::TStore::print       INFO        Pointer: 0x145893d0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: AntiKt4EMTopoJets_CalibShallowCopyAux.\r\nxAOD::TStore::print       INFO        Type: xAOD::ShallowAuxContainer\r\nxAOD::TStore::print       INFO        Pointer: 0x1448d6c0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: AntiKt4EMTopoJets_OREG_SCALE_LARCALIB__1down\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Jet_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x16a40ae0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: AntiKt4EMTopoJets_OREG_SCALE_LARCALIB__1up\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Jet_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x16f406d0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: AntiKt4EMTopoJets_Selected\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Jet_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x13e6ed70\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: ElectronCalibrator_Syst\r\nxAOD::TStore::print       INFO        Type: vector<string>\r\nxAOD::TStore::print       INFO        Pointer: 0x163458d0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: ElectronSelector_Syst\r\nxAOD::TStore::print       INFO        Type: vector<string>\r\nxAOD::TStore::print       INFO        Pointer: 0x17513da0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: ElectronsOR_Syst\t\t\t\t\r\nxAOD::TStore::print       INFO        Type: vector<string>\r\nxAOD::TStore::print       INFO        Pointer: 0x14997660\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_CalibEG_SCALE_LARCALIB__1down\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Electron_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x1492a020\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_CalibEG_SCALE_LARCALIB__1up\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Electron_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x16a45a70\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_CalibShallowCopyAux.EG_SCALE_LARCALIB__1down\r\nxAOD::TStore::print       INFO        Type: xAOD::ShallowAuxContainer\r\nxAOD::TStore::print       INFO        Pointer: 0x17576ce0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_CalibShallowCopyAux.EG_SCALE_LARCALIB__1up\r\nxAOD::TStore::print       INFO        Type: xAOD::ShallowAuxContainer\r\nxAOD::TStore::print       INFO        Pointer: 0x13964820\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_CalibShallowCopyEG_SCALE_LARCALIB__1down\r\nxAOD::TStore::print       INFO        Type: DataVector<xAOD::Electron_v1>\r\nxAOD::TStore::print       INFO        Pointer: 0x17578c00\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_CalibShallowCopyEG_SCALE_LARCALIB__1up\r\nxAOD::TStore::print       INFO        Type: DataVector<xAOD::Electron_v1>\r\nxAOD::TStore::print       INFO        Pointer: 0x16f40440\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_OREG_SCALE_LARCALIB__1down\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Electron_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x16b083a0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_OREG_SCALE_LARCALIB__1up\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Electron_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x13e75900\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_SelectedEG_SCALE_LARCALIB__1down\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Electron_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x13961b30\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Electrons_SelectedEG_SCALE_LARCALIB__1up\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Electron_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x16a40d10\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: JetCalibrator_Syst\r\nxAOD::TStore::print       INFO        Type: vector<string>\r\nxAOD::TStore::print       INFO        Pointer: 0x1383f0b0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: MuonCalibrator_Syst\r\nxAOD::TStore::print       INFO        Type: vector<string>\r\nxAOD::TStore::print       INFO        Pointer: 0x15fd47d0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: MuonSelector_Syst\r\nxAOD::TStore::print       INFO        Type: vector<string>\r\nxAOD::TStore::print       INFO        Pointer: 0x164076e0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Muons_Calib\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Muon_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x17069c40\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Muons_CalibShallowCopy\r\nxAOD::TStore::print       INFO        Type: DataVector<xAOD::Muon_v1>\r\nxAOD::TStore::print       INFO        Pointer: 0x159258c0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Muons_CalibShallowCopyAux.\r\nxAOD::TStore::print       INFO        Type: xAOD::ShallowAuxContainer\r\nxAOD::TStore::print       INFO        Pointer: 0x16b067f0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: Yes\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Muons_OREG_SCALE_LARCALIB__1down\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Muon_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x16a409e0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Muons_OREG_SCALE_LARCALIB__1up\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Muon_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x13e75a00\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\nxAOD::TStore::print       INFO      Name: Muons_Selected\r\nxAOD::TStore::print       INFO        Type: ConstDataVector<DataVector<xAOD::Muon_v1> >\r\nxAOD::TStore::print       INFO        Pointer: 0x173801d0\r\nxAOD::TStore::print       INFO        IsOwner: Yes\r\nxAOD::TStore::print       INFO        HasDictionary: No\r\nxAOD::TStore::print       INFO    -----------------------------------\r\n```\r\n\r\nAs you can see, the containers after the Overlap Removal (`*_OR*`) will have a name identifier with the systematic name appended to it. So, any algorithm downstream should be made able to pick them up the correct way,  by reading the systematics names from, e.g.,  `vector<string> ElectronOR_Syst`, which are stored in TStore as well. Of course, one could simply configure the downstream algos, e.g. the tree manager, using the **exact** name of the desired container.  \r\n\r\nTo my knowledge, at this stage of xAH implementation, nobody have really tried to think about how to dump all this systematically varied containers to a persistent output , i.e., a `TTree` or a mini-xAOD. For the tree case, it won't be too difficult to modify `TreeAlgo`, or any tree manager class derived from it that has access to the TStore, to let it read the different containers and dump separate trees for each syst. Maybe @kratsg can chime in on this.\r\n\r\nI would really appreciate any comments, ideas, suggestions to make this the most flexible and easy for the user to handle!\r\n\r\nCheers\r\n\r\nMarco \r\n"}}