{"action":"created","issue":{"url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/issues/83","labels_url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/issues/83/labels{/name}","comments_url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/issues/83/comments","events_url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/issues/83/events","html_url":"https://github.com/armenzg/mozilla_ci_tools/issues/83","id":59812814,"number":83,"title":"Failing to find a job","user":{"login":"armenzg","id":44410,"avatar_url":"https://avatars.githubusercontent.com/u/44410?v=3","gravatar_id":"","url":"https://api.github.com/users/armenzg","html_url":"https://github.com/armenzg","followers_url":"https://api.github.com/users/armenzg/followers","following_url":"https://api.github.com/users/armenzg/following{/other_user}","gists_url":"https://api.github.com/users/armenzg/gists{/gist_id}","starred_url":"https://api.github.com/users/armenzg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/armenzg/subscriptions","organizations_url":"https://api.github.com/users/armenzg/orgs","repos_url":"https://api.github.com/users/armenzg/repos","events_url":"https://api.github.com/users/armenzg/events{/privacy}","received_events_url":"https://api.github.com/users/armenzg/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"armenzg","id":44410,"avatar_url":"https://avatars.githubusercontent.com/u/44410?v=3","gravatar_id":"","url":"https://api.github.com/users/armenzg","html_url":"https://github.com/armenzg","followers_url":"https://api.github.com/users/armenzg/followers","following_url":"https://api.github.com/users/armenzg/following{/other_user}","gists_url":"https://api.github.com/users/armenzg/gists{/gist_id}","starred_url":"https://api.github.com/users/armenzg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/armenzg/subscriptions","organizations_url":"https://api.github.com/users/armenzg/orgs","repos_url":"https://api.github.com/users/armenzg/repos","events_url":"https://api.github.com/users/armenzg/events{/privacy}","received_events_url":"https://api.github.com/users/armenzg/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/milestones/1","html_url":"https://github.com/armenzg/mozilla_ci_tools/milestones/Make%20current%20scripts%20acceptable%20for%20sheriffs%20backfilling%20needs","labels_url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/milestones/1/labels","id":1002496,"number":1,"title":"Make current scripts acceptable for sheriffs backfilling needs","description":"We're tying to make sheriffs' lives better by allowing them to backfil when a bad job appears.\r\nI'm aiming to have this done before March 11th.","creator":{"login":"armenzg","id":44410,"avatar_url":"https://avatars.githubusercontent.com/u/44410?v=3","gravatar_id":"","url":"https://api.github.com/users/armenzg","html_url":"https://github.com/armenzg","followers_url":"https://api.github.com/users/armenzg/followers","following_url":"https://api.github.com/users/armenzg/following{/other_user}","gists_url":"https://api.github.com/users/armenzg/gists{/gist_id}","starred_url":"https://api.github.com/users/armenzg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/armenzg/subscriptions","organizations_url":"https://api.github.com/users/armenzg/orgs","repos_url":"https://api.github.com/users/armenzg/repos","events_url":"https://api.github.com/users/armenzg/events{/privacy}","received_events_url":"https://api.github.com/users/armenzg/received_events","type":"User","site_admin":false},"open_issues":4,"closed_issues":0,"state":"open","created_at":"2015-03-02T18:49:19Z","updated_at":"2015-03-04T15:25:24Z","due_on":"2015-03-11T04:00:00Z","closed_at":null},"comments":3,"created_at":"2015-03-04T15:25:24Z","updated_at":"2015-03-04T20:31:10Z","closed_at":null,"body":"I will work on it.\r\n\r\npython trigger_range.py -b 'Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o' --delta 4 --times 4 --rev f1134d7e0dab --debug --dry-run\r\n\r\n03/04/2015 10:21:16 INFO:\t === ac621df2174a ===\r\n03/04/2015 10:21:16 INFO:\t We want to have 4 job(s) of Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o on revision ac621df2174a\r\n03/04/2015 10:21:16 DEBUG:\t Determine if the revision is valid for buildapi.\r\n03/04/2015 10:21:16 DEBUG:\t Determine repository associated to mozilla-inbound\r\n03/04/2015 10:21:16 DEBUG:\t Loading /home/armenzg/repos/ci_tools/scripts/repositories.txt\r\n03/04/2015 10:21:16 DEBUG:\t About to fetch https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?changeset=ac621df2174a&tipsonly=1&full=1\r\n03/04/2015 10:21:16 INFO:\t Starting new HTTPS connection (1): hg.mozilla.org\r\n03/04/2015 10:21:17 DEBUG:\t \"GET /integration/mozilla-inbound/json-pushes?changeset=ac621df2174a&tipsonly=1&full=1 HTTP/1.1\" 200 303\r\n03/04/2015 10:21:17 DEBUG:\t Requesting the info with full=1 can yield too much unecessary output to debug anything properly\r\n03/04/2015 10:21:17 DEBUG:\t About to fetch https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-inbound/rev/ac621df2174a?format=json\r\n03/04/2015 10:21:17 INFO:\t Starting new HTTPS connection (1): secure.pub.build.mozilla.org\r\n03/04/2015 10:21:17 DEBUG:\t \"GET /buildapi/self-serve/mozilla-inbound/rev/ac621df2174a?format=json HTTP/1.1\" 200 37460\r\n03/04/2015 10:21:18 DEBUG:\t Find jobs matching 'Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o'\r\n03/04/2015 10:21:18 DEBUG:\t We have found 1 job(s) of 'Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o'.\r\n03/04/2015 10:21:18 DEBUG:\t We found 0 pending jobs, 0 running jobs and 1 successful_jobs.\r\n03/04/2015 10:21:18 INFO:\t We have found 1 potential job(s) matching 'Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o' on ac621df2174a. We need to trigger more.\r\n03/04/2015 10:21:18 DEBUG:\t Loading /home/armenzg/repos/ci_tools/scripts/repositories.txt\r\n03/04/2015 10:21:18 INFO:\t We want to trigger 'Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o' on revision 'ac621df2174a' a total of 3 time(s).\r\n03/04/2015 10:21:18 DEBUG:\t Determine if the revision is valid for buildapi.\r\n03/04/2015 10:21:18 DEBUG:\t Determine repository associated to mozilla-inbound\r\n03/04/2015 10:21:18 DEBUG:\t Loading /home/armenzg/repos/ci_tools/scripts/repositories.txt\r\n03/04/2015 10:21:18 DEBUG:\t About to fetch https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?changeset=ac621df2174a&tipsonly=1&full=1\r\n03/04/2015 10:21:18 INFO:\t Starting new HTTPS connection (1): hg.mozilla.org\r\n03/04/2015 10:21:18 DEBUG:\t \"GET /integration/mozilla-inbound/json-pushes?changeset=ac621df2174a&tipsonly=1&full=1 HTTP/1.1\" 200 303\r\n03/04/2015 10:21:18 DEBUG:\t Requesting the info with full=1 can yield too much unecessary output to debug anything properly\r\n03/04/2015 10:21:18 INFO:\t Starting new HTTPS connection (1): secure.pub.build.mozilla.org\r\n03/04/2015 10:21:24 DEBUG:\t \"HEAD /builddata/reports/allthethings.json HTTP/1.1\" 200 0\r\n03/04/2015 10:21:24 DEBUG:\t Buildername Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o is valid.\r\n03/04/2015 10:21:24 INFO:\t Starting new HTTPS connection (1): secure.pub.build.mozilla.org\r\n03/04/2015 10:21:24 DEBUG:\t \"HEAD /builddata/reports/allthethings.json HTTP/1.1\" 200 0\r\n03/04/2015 10:21:25 DEBUG:\t Buildername Linux x86-64 mozilla-inbound build is valid.\r\n03/04/2015 10:21:25 DEBUG:\t Determine if the revision is valid for buildapi.\r\n03/04/2015 10:21:25 DEBUG:\t Determine repository associated to mozilla-inbound\r\n03/04/2015 10:21:25 DEBUG:\t Loading /home/armenzg/repos/ci_tools/scripts/repositories.txt\r\n03/04/2015 10:21:25 DEBUG:\t About to fetch https://hg.mozilla.org/integration/mozilla-inbound/json-pushes?changeset=ac621df2174a&tipsonly=1&full=1\r\n03/04/2015 10:21:25 INFO:\t Starting new HTTPS connection (1): hg.mozilla.org\r\n03/04/2015 10:21:25 DEBUG:\t \"GET /integration/mozilla-inbound/json-pushes?changeset=ac621df2174a&tipsonly=1&full=1 HTTP/1.1\" 200 303\r\n03/04/2015 10:21:25 DEBUG:\t Requesting the info with full=1 can yield too much unecessary output to debug anything properly\r\n03/04/2015 10:21:25 DEBUG:\t About to fetch https://secure.pub.build.mozilla.org/buildapi/self-serve/mozilla-inbound/rev/ac621df2174a?format=json\r\n03/04/2015 10:21:25 INFO:\t Starting new HTTPS connection (1): secure.pub.build.mozilla.org\r\n03/04/2015 10:21:26 DEBUG:\t \"GET /buildapi/self-serve/mozilla-inbound/rev/ac621df2174a?format=json HTTP/1.1\" 200 37550\r\n03/04/2015 10:21:26 DEBUG:\t Find jobs matching 'Linux x86-64 mozilla-inbound build'\r\n03/04/2015 10:21:26 DEBUG:\t We have found 1 job(s) of 'Linux x86-64 mozilla-inbound build'.\r\n03/04/2015 10:21:26 DEBUG:\t List of matching jobs:\r\n03/04/2015 10:21:26 DEBUG:\t {u'build_id': 64479653, u'status': 0, u'branch': u'integration/mozilla-inbound', u'buildername': u'Linux x86-64 mozilla-inbound build', u'claimed_by_name': u'buildbot-master70.bb.releng.use1.mozilla.com:/builds/buildbot/build1/master', u'buildnumber': 257, u'starttime': 1425408398, u'requests': [{u'complete_at': 1425409676, u'complete': 1, u'buildername': u'Linux x86-64 mozilla-inbound build', u'claimed_at': 1425409477, u'priority': 0, u'submittime': 1425408345, u'reason': u'scheduler', u'branch': u'integration/mozilla-inbound', u'request_id': 63335359, u'revision': u'ac621df2174afebd1530a2b1f731927ab996031b'}], u'endtime': 1425409674, u'revision': u'ac621df2174afebd1530a2b1f731927ab996031b'}\r\n03/04/2015 10:21:26 DEBUG:\t We found a successful job. We don't search anymore.\r\n03/04/2015 10:21:26 INFO:\t There is a _build_ job that has completed successfully.\r\n03/04/2015 10:21:26 DEBUG:\t {u'build_id': 64479653, u'status': 0, u'branch': u'integration/mozilla-inbound', u'buildername': u'Linux x86-64 mozilla-inbound build', u'claimed_by_name': u'buildbot-master70.bb.releng.use1.mozilla.com:/builds/buildbot/build1/master', u'buildnumber': 257, u'starttime': 1425408398, u'requests': [{u'complete_at': 1425409676, u'complete': 1, u'buildername': u'Linux x86-64 mozilla-inbound build', u'claimed_at': 1425409477, u'priority': 0, u'submittime': 1425408345, u'reason': u'scheduler', u'branch': u'integration/mozilla-inbound', u'request_id': 63335359, u'revision': u'ac621df2174afebd1530a2b1f731927ab996031b'}], u'endtime': 1425409674, u'revision': u'ac621df2174afebd1530a2b1f731927ab996031b'}\r\n03/04/2015 10:21:26 DEBUG:\t Job identified with complete_at value: 1425409676 run on 2015-03-03 UTC.\r\n03/04/2015 10:21:26 DEBUG:\t The job completed at Tue, 03 Mar 2015 19:07:56 +0000 (UTC) (20 hours ago).\r\n03/04/2015 10:21:31 DEBUG:\t We are going to look for 63335359 in builds-2015-03-03.js.\r\n03/04/2015 10:21:31 ERROR:\t We have not found the job. If you see this problem please grep in builds-2015-03-03.js for 63335359 and run again with --debug and --dry-run.\r\nTraceback (most recent call last):\r\n  File \"trigger_range.py\", line 164, in <module>\r\n    dry_run=options.dry_run\r\n  File \"/home/armenzg/repos/ci_tools/mozci/mozci.py\", line 378, in trigger_range\r\n    dry_run=dry_run)\r\n  File \"/home/armenzg/repos/ci_tools/mozci/mozci.py\", line 300, in trigger_job\r\n    buildername,\r\n  File \"/home/armenzg/repos/ci_tools/mozci/mozci.py\", line 105, in _determine_trigger_objective\r\n    files = _find_files(successful_job)\r\n  File \"/home/armenzg/repos/ci_tools/mozci/mozci.py\", line 142, in _find_files\r\n    job_status = buildjson.query_job_data(complete_at, request_id)\r\n  File \"/home/armenzg/repos/ci_tools/mozci/sources/buildjson.py\", line 168, in query_job_data\r\n    job = _find_job(request_id, builds, filename)\r\n  File \"/home/armenzg/repos/ci_tools/mozci/sources/buildjson.py\", line 76, in _find_job\r\n    \"in %s for %d and run again with --debug and --dry-run.\" % (filename, request_id)\r\nException: We have not found the job. If you see this problem please grep in builds-2015-03-03.js for 63335359 and run again with --debug and --dry-run"},"comment":{"url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/issues/comments/77240671","html_url":"https://github.com/armenzg/mozilla_ci_tools/issues/83#issuecomment-77240671","issue_url":"https://api.github.com/repos/armenzg/mozilla_ci_tools/issues/83","id":77240671,"user":{"login":"armenzg","id":44410,"avatar_url":"https://avatars.githubusercontent.com/u/44410?v=3","gravatar_id":"","url":"https://api.github.com/users/armenzg","html_url":"https://github.com/armenzg","followers_url":"https://api.github.com/users/armenzg/followers","following_url":"https://api.github.com/users/armenzg/following{/other_user}","gists_url":"https://api.github.com/users/armenzg/gists{/gist_id}","starred_url":"https://api.github.com/users/armenzg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/armenzg/subscriptions","organizations_url":"https://api.github.com/users/armenzg/orgs","repos_url":"https://api.github.com/users/armenzg/repos","events_url":"https://api.github.com/users/armenzg/events{/privacy}","received_events_url":"https://api.github.com/users/armenzg/received_events","type":"User","site_admin":false},"created_at":"2015-03-04T20:31:10Z","updated_at":"2015-03-04T20:31:10Z","body":"I managed to fix this issue on \"master\".\r\nWe were using job[\"request_ids\"] instead of job[\"properties\"][\"request_ids\"]:\r\nhttps://github.com/armenzg/mozilla_ci_tools/commit/87c32cb126f0ccf4f606c45c7461f84cff881204\r\n\r\nI have also added a script under misc/ to help us understand this better in the near future.\r\n\r\nBoth of these commands work:\r\npython scripts/trigger_range.py -b 'Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o' --times 4 --rev ac621df2174a --debug --dry-run\r\npython scripts/trigger_range.py -b 'Ubuntu HW 12.04 x64 mozilla-inbound talos tp5o' --times 5 --rev ac621df2174a --debug --dry-run"}}