{"action":"created","issue":{"url":"https://api.github.com/repos/certtools/intelmq/issues/216","labels_url":"https://api.github.com/repos/certtools/intelmq/issues/216/labels{/name}","comments_url":"https://api.github.com/repos/certtools/intelmq/issues/216/comments","events_url":"https://api.github.com/repos/certtools/intelmq/issues/216/events","html_url":"https://github.com/certtools/intelmq/pull/216","id":85771259,"number":216,"title":"Feature, Header in URL fetcher bot.","user":{"login":"Th4nat0s","id":1728186,"avatar_url":"https://avatars.githubusercontent.com/u/1728186?v=3","gravatar_id":"","url":"https://api.github.com/users/Th4nat0s","html_url":"https://github.com/Th4nat0s","followers_url":"https://api.github.com/users/Th4nat0s/followers","following_url":"https://api.github.com/users/Th4nat0s/following{/other_user}","gists_url":"https://api.github.com/users/Th4nat0s/gists{/gist_id}","starred_url":"https://api.github.com/users/Th4nat0s/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Th4nat0s/subscriptions","organizations_url":"https://api.github.com/users/Th4nat0s/orgs","repos_url":"https://api.github.com/users/Th4nat0s/repos","events_url":"https://api.github.com/users/Th4nat0s/events{/privacy}","received_events_url":"https://api.github.com/users/Th4nat0s/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2015-06-06T15:36:26Z","updated_at":"2015-08-12T07:06:03Z","closed_at":"2015-08-11T17:10:25Z","pull_request":{"url":"https://api.github.com/repos/certtools/intelmq/pulls/216","html_url":"https://github.com/certtools/intelmq/pull/216","diff_url":"https://github.com/certtools/intelmq/pull/216.diff","patch_url":"https://github.com/certtools/intelmq/pull/216.patch"},"body":"Sometimes you got authentication key required in headers (http Authorization for basic, or key for misp by example) not only a key in the url.\r\n\r\nThis quick fix allow the definition of header per feed. The field is not mandatory to be defined.\r\n\r\nUsage:\r\n ```\r\n \"phishtank-collector\": {\r\n    \"error_procedure\": \"retry\",\r\n    \"error_retry_delay\": 30,\r\n    \"error_log_message\": true,\r\n    \"error_dump_message\": true,\r\n    \"url\": \"http://data.phishtank.com/data/online-valid.csv\",\r\n    \"header\": \"x-test: helloworld\",\r\n    \"rate_limit\": 3600\r\n  },\r\n\r\n```\r\nAnd it's added to http request.\r\n\r\n```\r\n0030  39 08 82 c3 00 00 47 45 54 20 2f 64 61 74 61 2f   9.....GET /data/\r\n0040  6f 6e 6c 69 6e 65 2d 76 61 6c 69 64 2e 63 73 76   online-valid.csv\r\n0050  20 48 54 54 50 2f 31 2e 31 0d 0a 41 63 63 65 70    HTTP/1.1..Accep\r\n0060  74 2d 45 6e 63 6f 64 69 6e 67 3a 20 69 64 65 6e   t-Encoding: iden\r\n0070  74 69 74 79 0d 0a 48 6f 73 74 3a 20 64 61 74 61   tity..Host: data\r\n0080  2e 70 68 69 73 68 74 61 6e 6b 2e 63 6f 6d 0d 0a   .phishtank.com..\r\n0090  43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 63 6c 6f 73   Connection: clos\r\n00a0  65 0d 0a 58 2d 54 65 73 74 3a 20 68 65 6c 6c 6f   e..X-Test: hello\r\n00b0  77 6f 72 64 72 64 0d 0a 0d 0a                     wordrd....\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/certtools/intelmq/issues/comments/130193808","html_url":"https://github.com/certtools/intelmq/pull/216#issuecomment-130193808","issue_url":"https://api.github.com/repos/certtools/intelmq/issues/216","id":130193808,"user":{"login":"sebix","id":199050,"avatar_url":"https://avatars.githubusercontent.com/u/199050?v=3","gravatar_id":"","url":"https://api.github.com/users/sebix","html_url":"https://github.com/sebix","followers_url":"https://api.github.com/users/sebix/followers","following_url":"https://api.github.com/users/sebix/following{/other_user}","gists_url":"https://api.github.com/users/sebix/gists{/gist_id}","starred_url":"https://api.github.com/users/sebix/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sebix/subscriptions","organizations_url":"https://api.github.com/users/sebix/orgs","repos_url":"https://api.github.com/users/sebix/repos","events_url":"https://api.github.com/users/sebix/events{/privacy}","received_events_url":"https://api.github.com/users/sebix/received_events","type":"User","site_admin":false},"created_at":"2015-08-12T07:06:03Z","updated_at":"2015-08-12T07:06:03Z","body":"This PR needs to be corrected since it includes 998 commits and the target branch does not exist anymore, but it is not resolved as we want this addition, I think?\r\n\r\n@Th4nat0s: Please don't pull (fetch and merge) from remote, use rebase instead."}}