{"action":"created","issue":{"url":"https://api.github.com/repos/CMS-SUS-XPAG/GenLHEfiles/issues/8","labels_url":"https://api.github.com/repos/CMS-SUS-XPAG/GenLHEfiles/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/CMS-SUS-XPAG/GenLHEfiles/issues/8/comments","events_url":"https://api.github.com/repos/CMS-SUS-XPAG/GenLHEfiles/issues/8/events","html_url":"https://github.com/CMS-SUS-XPAG/GenLHEfiles/pull/8","id":69605206,"number":8,"title":"Reorganize scripts, add LPC Condor functionality","user":{"login":"kpedro88","id":4672808,"avatar_url":"https://avatars.githubusercontent.com/u/4672808?v=3","gravatar_id":"","url":"https://api.github.com/users/kpedro88","html_url":"https://github.com/kpedro88","followers_url":"https://api.github.com/users/kpedro88/followers","following_url":"https://api.github.com/users/kpedro88/following{/other_user}","gists_url":"https://api.github.com/users/kpedro88/gists{/gist_id}","starred_url":"https://api.github.com/users/kpedro88/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kpedro88/subscriptions","organizations_url":"https://api.github.com/users/kpedro88/orgs","repos_url":"https://api.github.com/users/kpedro88/repos","events_url":"https://api.github.com/users/kpedro88/events{/privacy}","received_events_url":"https://api.github.com/users/kpedro88/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2015-04-20T14:52:48Z","updated_at":"2015-06-19T19:45:19Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/CMS-SUS-XPAG/GenLHEfiles/pulls/8","html_url":"https://github.com/CMS-SUS-XPAG/GenLHEfiles/pull/8","diff_url":"https://github.com/CMS-SUS-XPAG/GenLHEfiles/pull/8.diff","patch_url":"https://github.com/CMS-SUS-XPAG/GenLHEfiles/pull/8.patch"},"body":"SUSY_generation.sh split into three parts:\r\n* installGenerators.sh to do the initial installation of MadGraph\r\n* setupGenEnv.(c)sh to set the MadGraph environment variables\r\n* SUSY_generation.sh to run MadGraph\r\n\r\nrun_scan.py modified:\r\n* uses jobExecCondor.jdl or jobExecLXbatch.sh as template files to create jobs for Condor and LXbatch, respectively\r\n* some arguments added and functions modified to support LPC Condor\r\n\r\ncreate_update_header_config.py split into two parts:\r\n* makeMassDict.py which has the mass dictionary generation functions\r\n* create_update_header_config.py which imports makeMassDict.py and has examples of how to use the functions and create a configuration\r\n\r\nother things added for Condor:\r\n* CACHEDIR.TAG in folders that store logs and output from jobs (to run on LPC Condor, the CMSSW folder is tarballed and sent to the worker node; cached directories are excluded from the tarball to reduce the file size)\r\n* processUndecayed.sh, an example script to run Step 2 correctly with LPC EOS (using xrootd)\r\n\r\nA separate folder \"genstep\" was added for the private generation step to determine the qcut values:\r\n* the macro plotdjr.C is now located there\r\n* sample CMSSW configuration and Condor submission scripts added to this folder\r\n\r\nThe readme file has been updated to include all of these changes. I tested the code with LXbatch using bsub (to make sure the original functionality was retained) and it appears to work. (The commit history for this branch is a little messy, but the end result is nice.)\r\n\r\nAttn: @ferencek @nstrobbe "},"comment":{"url":"https://api.github.com/repos/CMS-SUS-XPAG/GenLHEfiles/issues/comments/113621440","html_url":"https://github.com/CMS-SUS-XPAG/GenLHEfiles/pull/8#issuecomment-113621440","issue_url":"https://api.github.com/repos/CMS-SUS-XPAG/GenLHEfiles/issues/8","id":113621440,"user":{"login":"kpedro88","id":4672808,"avatar_url":"https://avatars.githubusercontent.com/u/4672808?v=3","gravatar_id":"","url":"https://api.github.com/users/kpedro88","html_url":"https://github.com/kpedro88","followers_url":"https://api.github.com/users/kpedro88/followers","following_url":"https://api.github.com/users/kpedro88/following{/other_user}","gists_url":"https://api.github.com/users/kpedro88/gists{/gist_id}","starred_url":"https://api.github.com/users/kpedro88/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kpedro88/subscriptions","organizations_url":"https://api.github.com/users/kpedro88/orgs","repos_url":"https://api.github.com/users/kpedro88/repos","events_url":"https://api.github.com/users/kpedro88/events{/privacy}","received_events_url":"https://api.github.com/users/kpedro88/received_events","type":"User","site_admin":false},"created_at":"2015-06-19T19:45:19Z","updated_at":"2015-06-19T19:45:19Z","body":"Hi Dinko,\r\n\r\n>Ah, OK, I missed that part. I think it would be better to put it earlier right after all run_scan.py options are listed. In general, however, I think it would also be useful to be able to get file copied to a local folder, not just EOS.\r\n\r\nI moved that line in the readme so it will be more prominent.\r\n\r\nLPC Condor worker nodes do not have permission to write directly to /uscms/home or /uscms_data. The only way to transfer Condor output to local folders is using the Condor parameters:\r\n```\r\nShould_Transfer_Files = YES\r\nWhenToTransferOutput = ON_EXIT\r\n```\r\nThese are already turned on in the template JDL file to make sure the output, error, and log files get sent back when the job finishes.\r\n\r\nI could add a flag in the job script to use this option for the LHE file if the \"output\" parameter is not set, but I would prefer not to do this. Condor automatically copies the files to the local directory from which the job was submitted, which is \"scripts\" in this case. (I tried to use the Condor parameter \"output_destination\" to redirect the output to the \"lhe\" directory, but had no success. I don't think this works with the LPC setup.)\r\n\r\n>I guess my confusion was with what happens if I specify 5 when installing MadGraph and then later use --ncores=8. Somehow I would expect this to be a runtime parameter, not an installation one.\r\n\r\nIt was in the \"installation\" part of the old SUSY_generation.sh script, so I left it in installGenerators.sh when I reorganized things. The ncores parameter is used to configure MadGraph in some way. Maybe this configuration could be done at runtime instead of during installation, but I don't know for sure. @nstrobbe, do you know?\r\n\r\n>However, I am now facing problems with the update_header.py script. First, it does not seem to support ready directly from EOS folders. Second, when I copy the LHE files into the local lhe/ folder and run update_header.py, I get...\r\n\r\nThis was a bug, which I have fixed.\r\n\r\n>Also, the naming convention for the output LHE files seems to have changed. This now means that the updated scripts are probably no longer compatible with previously produced undecayed files. I would keep this part fully backward compatible.\r\n\r\nI edited the python scripts so that the naming convention is backward-compatible."}}