{"action":"created","issue":{"url":"https://api.github.com/repos/Norconex/collector-http/issues/144","labels_url":"https://api.github.com/repos/Norconex/collector-http/issues/144/labels{/name}","comments_url":"https://api.github.com/repos/Norconex/collector-http/issues/144/comments","events_url":"https://api.github.com/repos/Norconex/collector-http/issues/144/events","html_url":"https://github.com/Norconex/collector-http/issues/144","id":107346357,"number":144,"title":"http / https redirection","user":{"login":"csaezl","id":10865115,"avatar_url":"https://avatars.githubusercontent.com/u/10865115?v=3","gravatar_id":"","url":"https://api.github.com/users/csaezl","html_url":"https://github.com/csaezl","followers_url":"https://api.github.com/users/csaezl/followers","following_url":"https://api.github.com/users/csaezl/following{/other_user}","gists_url":"https://api.github.com/users/csaezl/gists{/gist_id}","starred_url":"https://api.github.com/users/csaezl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/csaezl/subscriptions","organizations_url":"https://api.github.com/users/csaezl/orgs","repos_url":"https://api.github.com/users/csaezl/repos","events_url":"https://api.github.com/users/csaezl/events{/privacy}","received_events_url":"https://api.github.com/users/csaezl/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Norconex/collector-http/labels/feature-request","name":"feature-request","color":"0052cc"},{"url":"https://api.github.com/repos/Norconex/collector-http/labels/fixed/done","name":"fixed/done","color":"bfe5bf"}],"state":"open","locked":false,"assignee":{"login":"essiembre","id":2326274,"avatar_url":"https://avatars.githubusercontent.com/u/2326274?v=3","gravatar_id":"","url":"https://api.github.com/users/essiembre","html_url":"https://github.com/essiembre","followers_url":"https://api.github.com/users/essiembre/followers","following_url":"https://api.github.com/users/essiembre/following{/other_user}","gists_url":"https://api.github.com/users/essiembre/gists{/gist_id}","starred_url":"https://api.github.com/users/essiembre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/essiembre/subscriptions","organizations_url":"https://api.github.com/users/essiembre/orgs","repos_url":"https://api.github.com/users/essiembre/repos","events_url":"https://api.github.com/users/essiembre/events{/privacy}","received_events_url":"https://api.github.com/users/essiembre/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/Norconex/collector-http/milestones/9","html_url":"https://github.com/Norconex/collector-http/milestones/2.3.0","labels_url":"https://api.github.com/repos/Norconex/collector-http/milestones/9/labels","id":1246265,"number":9,"title":"2.3.0","description":"Feature release.","creator":{"login":"essiembre","id":2326274,"avatar_url":"https://avatars.githubusercontent.com/u/2326274?v=3","gravatar_id":"","url":"https://api.github.com/users/essiembre","html_url":"https://github.com/essiembre","followers_url":"https://api.github.com/users/essiembre/followers","following_url":"https://api.github.com/users/essiembre/following{/other_user}","gists_url":"https://api.github.com/users/essiembre/gists{/gist_id}","starred_url":"https://api.github.com/users/essiembre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/essiembre/subscriptions","organizations_url":"https://api.github.com/users/essiembre/orgs","repos_url":"https://api.github.com/users/essiembre/repos","events_url":"https://api.github.com/users/essiembre/events{/privacy}","received_events_url":"https://api.github.com/users/essiembre/received_events","type":"User","site_admin":false},"open_issues":3,"closed_issues":7,"state":"open","created_at":"2015-08-08T02:31:07Z","updated_at":"2015-09-28T15:39:36Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2015-09-19T17:33:15Z","updated_at":"2015-09-29T15:35:08Z","closed_at":null,"body":"Crawling ```http://adelantextremadura.wordpress.com/``` with ```<extractor ignoreExternalLinks=\"true\" />``` when the redirection is based on http/https, the original URL is rejected. Could it be possible to manage them as the same protocol?. If not, it would be mandatory to know what to use, http/https, in advance.\r\n```\r\nMC (crawler): 2015-09-19 18:54:48 INFO -           CRAWLER_STARTED\r\nMC (crawler): 2015-09-19 18:54:48 INFO - MC (crawler): Crawling references...\r\nMC (crawler): 2015-09-19 18:54:49 INFO - Ignoring external redirect: http://adelantextremadura.wordpress.com/ -> https://adelantextremadura.wordpress.com/\r\nMC (crawler): 2015-09-19 18:54:49 INFO -       REJECTED_BAD_STATUS: http://adelantextremadura.wordpress.com/\r\nMC (crawler): 2015-09-19 18:54:50 INFO - MC (crawler): Crawler finishing: committing documents.\r\nMC (crawler): 2015-09-19 18:54:50 INFO - MC (crawler): 1 reference(s) processed.\r\nMC (crawler): 2015-09-19 18:54:50 INFO -          CRAWLER_FINISHED\r\n\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/Norconex/collector-http/issues/comments/144096129","html_url":"https://github.com/Norconex/collector-http/issues/144#issuecomment-144096129","issue_url":"https://api.github.com/repos/Norconex/collector-http/issues/144","id":144096129,"user":{"login":"csaezl","id":10865115,"avatar_url":"https://avatars.githubusercontent.com/u/10865115?v=3","gravatar_id":"","url":"https://api.github.com/users/csaezl","html_url":"https://github.com/csaezl","followers_url":"https://api.github.com/users/csaezl/followers","following_url":"https://api.github.com/users/csaezl/following{/other_user}","gists_url":"https://api.github.com/users/csaezl/gists{/gist_id}","starred_url":"https://api.github.com/users/csaezl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/csaezl/subscriptions","organizations_url":"https://api.github.com/users/csaezl/orgs","repos_url":"https://api.github.com/users/csaezl/repos","events_url":"https://api.github.com/users/csaezl/events{/privacy}","received_events_url":"https://api.github.com/users/csaezl/received_events","type":"User","site_admin":false},"created_at":"2015-09-29T15:35:08Z","updated_at":"2015-09-29T15:35:08Z","body":"\r\nI tested ```http://adelantextremadura.wordpress.com/``` with:\r\n```\r\n      <startURLs stayOnDomain=\"true\" stayOnPort=\"true\" stayOnProtocol=\"true\">\r\n\t\t         <url>http://adelantextremadura.wordpress.com/</url>\r\n```\r\n\r\nand got similar result:\r\n```\r\n           CRAWLER_STARTED\r\n MC (crawler): Crawling references...\r\n       REJECTED_REDIRECTED: http://adelantextremadura.wordpress.com/\r\n           REJECTED_FILTER: https://adelantextremadura.wordpress.com/\r\n MC (crawler): Crawler finishing: committing documents.\r\n MC (crawler): 1 reference(s) processed.\r\n          CRAWLER_FINISHED\r\n```"}}