{"action":"created","issue":{"url":"https://api.github.com/repos/dmwm/WMCore/issues/6294","labels_url":"https://api.github.com/repos/dmwm/WMCore/issues/6294/labels{/name}","comments_url":"https://api.github.com/repos/dmwm/WMCore/issues/6294/comments","events_url":"https://api.github.com/repos/dmwm/WMCore/issues/6294/events","html_url":"https://github.com/dmwm/WMCore/pull/6294","id":109007122,"number":6294,"title":"initial patch for eos stageout (has error do not merge)","user":{"login":"ticoann","id":1247993,"avatar_url":"https://avatars.githubusercontent.com/u/1247993?v=3","gravatar_id":"","url":"https://api.github.com/users/ticoann","html_url":"https://github.com/ticoann","followers_url":"https://api.github.com/users/ticoann/followers","following_url":"https://api.github.com/users/ticoann/following{/other_user}","gists_url":"https://api.github.com/users/ticoann/gists{/gist_id}","starred_url":"https://api.github.com/users/ticoann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ticoann/subscriptions","organizations_url":"https://api.github.com/users/ticoann/orgs","repos_url":"https://api.github.com/users/ticoann/repos","events_url":"https://api.github.com/users/ticoann/events{/privacy}","received_events_url":"https://api.github.com/users/ticoann/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"ticoann","id":1247993,"avatar_url":"https://avatars.githubusercontent.com/u/1247993?v=3","gravatar_id":"","url":"https://api.github.com/users/ticoann","html_url":"https://github.com/ticoann","followers_url":"https://api.github.com/users/ticoann/followers","following_url":"https://api.github.com/users/ticoann/following{/other_user}","gists_url":"https://api.github.com/users/ticoann/gists{/gist_id}","starred_url":"https://api.github.com/users/ticoann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ticoann/subscriptions","organizations_url":"https://api.github.com/users/ticoann/orgs","repos_url":"https://api.github.com/users/ticoann/repos","events_url":"https://api.github.com/users/ticoann/events{/privacy}","received_events_url":"https://api.github.com/users/ticoann/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/dmwm/WMCore/milestones/116","html_url":"https://github.com/dmwm/WMCore/milestones/WMAgent1509","labels_url":"https://api.github.com/repos/dmwm/WMCore/milestones/116/labels","id":1245706,"number":116,"title":"WMAgent1509","description":"","creator":{"login":"ticoann","id":1247993,"avatar_url":"https://avatars.githubusercontent.com/u/1247993?v=3","gravatar_id":"","url":"https://api.github.com/users/ticoann","html_url":"https://github.com/ticoann","followers_url":"https://api.github.com/users/ticoann/followers","following_url":"https://api.github.com/users/ticoann/following{/other_user}","gists_url":"https://api.github.com/users/ticoann/gists{/gist_id}","starred_url":"https://api.github.com/users/ticoann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ticoann/subscriptions","organizations_url":"https://api.github.com/users/ticoann/orgs","repos_url":"https://api.github.com/users/ticoann/repos","events_url":"https://api.github.com/users/ticoann/events{/privacy}","received_events_url":"https://api.github.com/users/ticoann/received_events","type":"User","site_admin":false},"open_issues":12,"closed_issues":21,"state":"open","created_at":"2015-08-07T15:47:07Z","updated_at":"2015-09-30T19:21:05Z","due_on":"2015-09-30T05:00:00Z","closed_at":null},"comments":18,"created_at":"2015-09-30T03:23:18Z","updated_at":"2015-10-01T13:09:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/dmwm/WMCore/pulls/6294","html_url":"https://github.com/dmwm/WMCore/pull/6294","diff_url":"https://github.com/dmwm/WMCore/pull/6294.diff","patch_url":"https://github.com/dmwm/WMCore/pull/6294.patch"},"body":"Error message\r\n```\r\nMSG: Unable to stage out log archive to EOS: \r\nStageOutFailure\r\nMessage: Failure for fallback stage out:\r\nStageOutError\r\nMessage: Command exited non-zero\r\n        ErrorCode : 60311\r\n        ModuleName : WMCore.Storage.StageOutError\r\n        MethodName : __init__\r\n        ErrorType : GeneralStageOutFailure\r\n        ClassInstance : None\r\n        FileName : /pool/condor/dir_1319717/glide_jTKtqa/execute/dir_6580/job/WMCore.zip/WMCore/Storage/StageOutError.py\r\n        ClassName : None\r\n        Command : #!/bin/sh\r\nsource /etc/profile.d/grid-env.sh; lcg-cp -b -D srmv2 --vo cms --srm-timeout 2400 --sendreceive-timeout 2400 --connect-timeout 300 --verbose  srm://srm-cms.cern.ch:8443/srm/managerv2?SFN=/castor/cern.ch/cms/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar  srm://srm-eoscms.cern.ch:8443/srm/v2/server?SFN=/eos/cms/store/logs/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar\r\n\r\n            EXIT_STATUS=$?\r\n            echo \"lcg-cp exit status: $EXIT_STATUS\"\r\n            if [[ $EXIT_STATUS != 0 ]]; then\r\n               echo \"Non-zero lcg-cp Exit status!!!\"\r\n               echo \"Cleaning up failed file:\"\r\n                source /etc/profile.d/grid-env.sh;  lcg-del -b -l -D srmv2 --vo cms srm://srm-eoscms.cern.ch:8443/srm/v2/server?SFN=/eos/cms/store/logs/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar\r\n               exit 60311\r\n            fi\r\n\r\n            FILE_SIZE=`stat -c %s srm://srm-cms.cern.ch:8443/srm/managerv2?SFN=/castor/cern.ch/cms/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar `\r\necho \"Local File Size is: $FILE_SIZE\"\r\n\r\n        for ((a=1; a <= 10 ; a++))\r\n        do\r\n           LCG_OUTPUT=`lcg-ls -l -b -D srmv2 srm://srm-eoscms.cern.ch:8443/srm/v2/server?SFN=/eos/cms/store/logs/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar 2>/dev/null`\r\n           SRM_SIZE=`echo \"$LCG_OUTPUT\" | awk 'NR==1{print $5}'`\r\n           SRM_CHECKSUM=`echo \"$LCG_OUTPUT\" | sed -nr 's/^.*\\s([a-f0-9]{8})\\s*\\([aA][dD][lL][eE][rR]32\\)\\s*$/\\1/p'`\r\n           echo \"Remote Size is $SRM_SIZE\"\r\n           echo \"Remote Checksum is $SRM_CHECKSUM\"\r\n           if [[ $SRM_SIZE > 0 ]]; then\r\n              if [[ $SRM_SIZE == $FILE_SIZE ]]; then\r\n                 exit 0\r\n              else\r\n                 echo \"Error: Size Mismatch between local and SE\"\r\n                 echo \"Cleaning up failed file:\"\r\n                 source /etc/profile.d/grid-env.sh;  lcg-del -b -l -D srmv2 --vo cms srm://srm-eoscms.cern.ch:8443/srm/v2/server?SFN=/eos/cms/store/logs/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar\r\n                 exit 60311\r\n              fi\r\n           else\r\n              sleep 2\r\n           fi\r\n        done\r\n        echo \"Cleaning up failed file:\"\r\n        source /etc/profile.d/grid-env.sh;  lcg-del -b -l -D srmv2 --vo cms srm://srm-eoscms.cern.ch:8443/srm/v2/server?SFN=/eos/cms/store/logs/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar\r\n        exit 60311\r\n\r\n\r\n        LineNumber :32\r\n        ErrorNr : 0\r\n        ExitCode : 151\r\n\r\nTraceback:\r\n  File \"/pool/condor/dir_1319717/glide_jTKtqa/execute/dir_6580/job/WMCore.zip/WMCore/Storage/StageOutImpl.py\", line 196, in __call__\r\n    self.executeCommand(command)\r\n\r\n  File \"/pool/condor/dir_1319717/glide_jTKtqa/execute/dir_6580/job/WMCore.zip/WMCore/Storage/StageOutImpl.py\", line 65, in executeCommand\r\n    raise StageOutError(msg, Command = command, ExitCode = exitCode)\r\n\r\n\r\n        TargetPFN : srm://srm-eoscms.cern.ch:8443/srm/v2/server?SFN=/eos/cms/store/logs/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar\r\n        ErrorCode : 60311\r\n        ModuleName : WMCore.Storage.StageOutError\r\n        MethodName : __init__\r\n        LFN : /store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar\r\n        ClassInstance : None\r\n        FileName : /pool/condor/dir_1319717/glide_jTKtqa/execute/dir_6580/job/WMCore.zip/WMCore/Storage/StageOutError.py\r\n        ClassName : None\r\n        Command : srmv2-lcg\r\n        LineNumber : 32\r\n        InputPFN : srm://srm-cms.cern.ch:8443/srm/managerv2?SFN=/castor/cern.ch/cms/store/logs/prod/2015/09/WMAgent/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435/sryu_MonteCarlo_LHE_mc_lumi_validation_150929_175102_2435-LogCollect-b69cfa5dbd-1-logs.tar\r\n        ErrorNr : 0\r\n        ErrorType : GeneralStageOutFailure\r\n\r\nTraceback:\r\n  File \"/pool/condor/dir_1319717/glide_jTKtqa/execute/dir_6580/job/WMCore.zip/WMCore/Storage/StageOutMgr.py\", line 260, in fallbackStageOut\r\n    impl(fbParams['command'], localPfn, pfn, fbParams.get(\"option\", None), checksums)\r\n\r\n  File \"/pool/condor/dir_1319717/glide_jTKtqa/execute/dir_6580/job/WMCore.zip/WMCore/Storage/StageOutImpl.py\", line 214, in __call__\r\n    raise ex\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/dmwm/WMCore/issues/comments/144721863","html_url":"https://github.com/dmwm/WMCore/pull/6294#issuecomment-144721863","issue_url":"https://api.github.com/repos/dmwm/WMCore/issues/6294","id":144721863,"user":{"login":"amaltaro","id":1798520,"avatar_url":"https://avatars.githubusercontent.com/u/1798520?v=3","gravatar_id":"","url":"https://api.github.com/users/amaltaro","html_url":"https://github.com/amaltaro","followers_url":"https://api.github.com/users/amaltaro/followers","following_url":"https://api.github.com/users/amaltaro/following{/other_user}","gists_url":"https://api.github.com/users/amaltaro/gists{/gist_id}","starred_url":"https://api.github.com/users/amaltaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amaltaro/subscriptions","organizations_url":"https://api.github.com/users/amaltaro/orgs","repos_url":"https://api.github.com/users/amaltaro/repos","events_url":"https://api.github.com/users/amaltaro/events{/privacy}","received_events_url":"https://api.github.com/users/amaltaro/received_events","type":"User","site_admin":false},"created_at":"2015-10-01T13:09:44Z","updated_at":"2015-10-01T13:09:44Z","body":"Yeah, that was the only missing piece, thanks Nicolo.\r\n@ticoann here is a working version of your patch https://github.com/amaltaro/WMCore/commit/8cc73029bfe03f5d875c248f9153cc35f376c49c\r\ncan you merge it and update this PR please?\r\n\r\nNicolo, one more thing, could you please also map Dave Mason's cert to cmsprd account? If something bad happen to me, he would be the backup man :-)"}}