{"action":"created","issue":{"url":"https://api.github.com/repos/dmwm/CRABServer/issues/5101","labels_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5101/labels{/name}","comments_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5101/comments","events_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5101/events","html_url":"https://github.com/dmwm/CRABServer/issues/5101","id":120419041,"number":5101,"title":"Local fallback stageout issues","user":{"login":"juztas","id":646582,"avatar_url":"https://avatars.githubusercontent.com/u/646582?v=3","gravatar_id":"","url":"https://api.github.com/users/juztas","html_url":"https://github.com/juztas","followers_url":"https://api.github.com/users/juztas/followers","following_url":"https://api.github.com/users/juztas/following{/other_user}","gists_url":"https://api.github.com/users/juztas/gists{/gist_id}","starred_url":"https://api.github.com/users/juztas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/juztas/subscriptions","organizations_url":"https://api.github.com/users/juztas/orgs","repos_url":"https://api.github.com/users/juztas/repos","events_url":"https://api.github.com/users/juztas/events{/privacy}","received_events_url":"https://api.github.com/users/juztas/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/dmwm/CRABServer/labels/bug","name":"bug","color":"eb6420"}],"state":"closed","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/dmwm/CRABServer/milestones/75","html_url":"https://github.com/dmwm/CRABServer/milestones/CRAB3%20December%202015","labels_url":"https://api.github.com/repos/dmwm/CRABServer/milestones/75/labels","id":1251881,"number":75,"title":"CRAB3 December 2015","description":"","creator":{"login":"mmascher","id":1586246,"avatar_url":"https://avatars.githubusercontent.com/u/1586246?v=3","gravatar_id":"","url":"https://api.github.com/users/mmascher","html_url":"https://github.com/mmascher","followers_url":"https://api.github.com/users/mmascher/followers","following_url":"https://api.github.com/users/mmascher/following{/other_user}","gists_url":"https://api.github.com/users/mmascher/gists{/gist_id}","starred_url":"https://api.github.com/users/mmascher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmascher/subscriptions","organizations_url":"https://api.github.com/users/mmascher/orgs","repos_url":"https://api.github.com/users/mmascher/repos","events_url":"https://api.github.com/users/mmascher/events{/privacy}","received_events_url":"https://api.github.com/users/mmascher/received_events","type":"User","site_admin":false},"open_issues":1,"closed_issues":18,"state":"open","created_at":"2015-08-12T14:52:53Z","updated_at":"2015-12-11T14:34:03Z","due_on":null,"closed_at":null},"comments":7,"created_at":"2015-12-04T15:05:03Z","updated_at":"2015-12-14T15:07:46Z","closed_at":"2015-12-10T10:37:20Z","body":"Andrew pinged me about his problems. So in summary: Job finishes on T2_US_UCSD, CRAB tries to do local stageout and fails with permission denied, after that comes fallback stageout, which can point to any other site (but usually nearest one) in this case Caltech and fallback stageout succeeds. CRAB uploads doc to couch in which it says that output is at UCSD. And of course FTS fails to transfer file as it does not exist.\r\n\r\ndo we want to do fallback stageout? \r\n    a) if yes, site reporting to ASO should be fixed.\r\n    b) If not, CRAB should say that to WMCore not to do fallback stageout and in case of local stageout failures do remote. \r\n\r\nFor example log file:\r\nhttp://submit-4.t2.ucsd.edu/CSstoragePath/54/uscms3895/151204_113644:anlevin_crab_fr_muon_ntuples_v10000/job_out.138.0.txt"},"comment":{"url":"https://api.github.com/repos/dmwm/CRABServer/issues/comments/164461053","html_url":"https://github.com/dmwm/CRABServer/issues/5101#issuecomment-164461053","issue_url":"https://api.github.com/repos/dmwm/CRABServer/issues/5101","id":164461053,"user":{"login":"mmascher","id":1586246,"avatar_url":"https://avatars.githubusercontent.com/u/1586246?v=3","gravatar_id":"","url":"https://api.github.com/users/mmascher","html_url":"https://github.com/mmascher","followers_url":"https://api.github.com/users/mmascher/followers","following_url":"https://api.github.com/users/mmascher/following{/other_user}","gists_url":"https://api.github.com/users/mmascher/gists{/gist_id}","starred_url":"https://api.github.com/users/mmascher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmascher/subscriptions","organizations_url":"https://api.github.com/users/mmascher/orgs","repos_url":"https://api.github.com/users/mmascher/repos","events_url":"https://api.github.com/users/mmascher/events{/privacy}","received_events_url":"https://api.github.com/users/mmascher/received_events","type":"User","site_admin":false},"created_at":"2015-12-14T15:07:46Z","updated_at":"2015-12-14T15:07:46Z","body":"I am using https://github.com/mmascher/WMCore/tree/3.3.1511 which is based on 1.0.9.pre4. It's on my (immediate) todo list to update to a new WMCore :)\r\n\r\nMaybe it's better to apply it on top of my WMCore this time. I'll prepare and RPM for tomorrow morning"}}