{"action":"created","issue":{"url":"https://api.github.com/repos/Norconex/collector-http/issues/119","labels_url":"https://api.github.com/repos/Norconex/collector-http/issues/119/labels{/name}","comments_url":"https://api.github.com/repos/Norconex/collector-http/issues/119/comments","events_url":"https://api.github.com/repos/Norconex/collector-http/issues/119/events","html_url":"https://github.com/Norconex/collector-http/issues/119","id":88464336,"number":119,"title":"Javascript generated URLs","user":{"login":"AntonioAmore","id":7804074,"avatar_url":"https://avatars.githubusercontent.com/u/7804074?v=3","gravatar_id":"","url":"https://api.github.com/users/AntonioAmore","html_url":"https://github.com/AntonioAmore","followers_url":"https://api.github.com/users/AntonioAmore/followers","following_url":"https://api.github.com/users/AntonioAmore/following{/other_user}","gists_url":"https://api.github.com/users/AntonioAmore/gists{/gist_id}","starred_url":"https://api.github.com/users/AntonioAmore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AntonioAmore/subscriptions","organizations_url":"https://api.github.com/users/AntonioAmore/orgs","repos_url":"https://api.github.com/users/AntonioAmore/repos","events_url":"https://api.github.com/users/AntonioAmore/events{/privacy}","received_events_url":"https://api.github.com/users/AntonioAmore/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Norconex/collector-http/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/Norconex/collector-http/milestones/7","html_url":"https://github.com/Norconex/collector-http/milestones/2.2.0","labels_url":"https://api.github.com/repos/Norconex/collector-http/milestones/7/labels","id":1062185,"number":7,"title":"2.2.0","description":"Feature release.","creator":{"login":"essiembre","id":2326274,"avatar_url":"https://avatars.githubusercontent.com/u/2326274?v=3","gravatar_id":"","url":"https://api.github.com/users/essiembre","html_url":"https://github.com/essiembre","followers_url":"https://api.github.com/users/essiembre/followers","following_url":"https://api.github.com/users/essiembre/following{/other_user}","gists_url":"https://api.github.com/users/essiembre/gists{/gist_id}","starred_url":"https://api.github.com/users/essiembre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/essiembre/subscriptions","organizations_url":"https://api.github.com/users/essiembre/orgs","repos_url":"https://api.github.com/users/essiembre/repos","events_url":"https://api.github.com/users/essiembre/events{/privacy}","received_events_url":"https://api.github.com/users/essiembre/received_events","type":"User","site_admin":false},"open_issues":13,"closed_issues":0,"state":"open","created_at":"2015-04-10T16:00:57Z","updated_at":"2015-06-17T18:08:59Z","due_on":null,"closed_at":null},"comments":3,"created_at":"2015-06-15T15:44:57Z","updated_at":"2015-06-19T02:49:34Z","closed_at":null,"body":"I tried to crawl a site and get following error in log:\r\n```\r\nsite: 2015-06-10 21:38:12 DEBUG - ACCEPTED document reference. Reference=http://www.site.com/Projects/c2c/channel/images/'+L140413[1+Math.round(Math.random()*(1-1))]+' Filter=com.norconex.collector.core.filter.impl.ExtensionReferenceFilter@3ec300f1[onMatch=EXCLUDE,extensions=jpg,gif,png,ico,swf,css,js,pdf,doc,xls,ppt,txt,odt,zip,rar,gz,swf,xlsx,docx,pptx,mp3,wav,mid,caseSensitive=false]\r\nsite: 2015-06-10 21:38:12 ERROR - site: Could not process document: http://www.site.com/dccom/0-5-7171-1-1810165-1-0-0-0-0-0-9296-0-0-0-0-0-0-0-0.html (Invalid URL syntax: http://www.site.com/Projects/c2c/channel/images/'+L140413[1+Math.round(Math.random()*(1-1))]+')\r\ncom.norconex.commons.lang.url.URLException: Invalid URL syntax: http://www.site.com/Projects/c2c/channel/images/'+L140413[1+Math.round(Math.random()*(1-1))]+'\r\n\tat com.norconex.commons.lang.url.URLNormalizer.<init>(URLNormalizer.java:181)\r\n\tat com.norconex.collector.http.url.impl.GenericURLNormalizer.normalizeURL(GenericURLNormalizer.java:159)\r\n\tat com.norconex.collector.http.pipeline.queue.HttpQueuePipeline$URLNormalizerStage.executeStage(HttpQueuePipeline.java:133)\r\n\tat com.norconex.collector.http.pipeline.queue.AbstractQueueStage.execute(AbstractQueueStage.java:31)\r\n\tat com.norconex.collector.http.pipeline.queue.AbstractQueueStage.execute(AbstractQueueStage.java:24)\r\n\tat com.norconex.commons.lang.pipeline.Pipeline.execute(Pipeline.java:90)\r\n\tat com.norconex.collector.http.pipeline.importer.LinkExtractorStage.executeStage(LinkExtractorStage.java:94)\r\n\tat com.norconex.collector.http.pipeline.importer.AbstractImporterStage.execute(AbstractImporterStage.java:31)\r\n\tat com.norconex.collector.http.pipeline.importer.AbstractImporterStage.execute(AbstractImporterStage.java:24)\r\n\tat com.norconex.commons.lang.pipeline.Pipeline.execute(Pipeline.java:90)\r\n\tat com.norconex.collector.http.crawler.HttpCrawler.executeImporterPipeline(HttpCrawler.java:213)\r\n\tat com.norconex.collector.core.crawler.AbstractCrawler.processNextQueuedCrawlData(AbstractCrawler.java:473)\r\n\tat com.norconex.collector.core.crawler.AbstractCrawler.processNextReference(AbstractCrawler.java:373)\r\n\tat com.norconex.collector.core.crawler.AbstractCrawler$ProcessURLsRunnable.run(AbstractCrawler.java:631)\r\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\r\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\r\n\tat java.lang.Thread.run(Thread.java:745)\r\nCaused by: java.net.URISyntaxException: Illegal character in path at index 60: http://www.site.com/Projects/c2c/channel/images/'+L140413[1+Math.round(Math.random()*(1-1))]+'\r\n\tat java.net.URI$Parser.fail(URI.java:2848)\r\n\tat java.net.URI$Parser.checkChars(URI.java:3021)\r\n\tat java.net.URI$Parser.parseHierarchical(URI.java:3105)\r\n\tat java.net.URI$Parser.parse(URI.java:3053)\r\n\tat java.net.URI.<init>(URI.java:588)\r\n\tat com.norconex.commons.lang.url.URLNormalizer.<init>(URLNormalizer.java:179)\r\n\t... 16 more\r\n```\r\n\r\nAfter reading the page's HTML I found the following fragment there:\r\n```\r\n<SCRIPT language=javascript><!--- \r\n document.write('<img src=\"/Projects/c2c/channel/images/'+L140413[1+Math.round(Math.random()*(1-1))]+'\"  border=\"0\">');\r\n// --->\r\n...\r\n```\r\n\r\nSeems it were causing the exception, and looks it is the reason there is no pages imported while crawl. \r\n\r\nIs there a way to avoid such links, or don't break importing if something happening like that?\r\n"},"comment":{"url":"https://api.github.com/repos/Norconex/collector-http/issues/comments/113349141","html_url":"https://github.com/Norconex/collector-http/issues/119#issuecomment-113349141","issue_url":"https://api.github.com/repos/Norconex/collector-http/issues/119","id":113349141,"user":{"login":"essiembre","id":2326274,"avatar_url":"https://avatars.githubusercontent.com/u/2326274?v=3","gravatar_id":"","url":"https://api.github.com/users/essiembre","html_url":"https://github.com/essiembre","followers_url":"https://api.github.com/users/essiembre/followers","following_url":"https://api.github.com/users/essiembre/following{/other_user}","gists_url":"https://api.github.com/users/essiembre/gists{/gist_id}","starred_url":"https://api.github.com/users/essiembre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/essiembre/subscriptions","organizations_url":"https://api.github.com/users/essiembre/orgs","repos_url":"https://api.github.com/users/essiembre/repos","events_url":"https://api.github.com/users/essiembre/events{/privacy}","received_events_url":"https://api.github.com/users/essiembre/received_events","type":"User","site_admin":false},"created_at":"2015-06-19T02:49:34Z","updated_at":"2015-06-19T02:49:34Z","body":"The fix is in the latest [snapshot release](http://www.norconex.com/collectors/collector-http/download)."}}