{"action":"created","issue":{"url":"https://api.github.com/repos/datasets/registry/issues/94","labels_url":"https://api.github.com/repos/datasets/registry/issues/94/labels{/name}","comments_url":"https://api.github.com/repos/datasets/registry/issues/94/comments","events_url":"https://api.github.com/repos/datasets/registry/issues/94/events","html_url":"https://github.com/datasets/registry/pull/94","id":72300806,"number":94,"title":"Add validate script","user":{"login":"Deiz","id":84350,"avatar_url":"https://avatars.githubusercontent.com/u/84350?v=3","gravatar_id":"","url":"https://api.github.com/users/Deiz","html_url":"https://github.com/Deiz","followers_url":"https://api.github.com/users/Deiz/followers","following_url":"https://api.github.com/users/Deiz/following{/other_user}","gists_url":"https://api.github.com/users/Deiz/gists{/gist_id}","starred_url":"https://api.github.com/users/Deiz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Deiz/subscriptions","organizations_url":"https://api.github.com/users/Deiz/orgs","repos_url":"https://api.github.com/users/Deiz/repos","events_url":"https://api.github.com/users/Deiz/events{/privacy}","received_events_url":"https://api.github.com/users/Deiz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-04-30T22:36:30Z","updated_at":"2015-05-01T10:01:23Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/datasets/registry/pulls/94","html_url":"https://github.com/datasets/registry/pull/94","diff_url":"https://github.com/datasets/registry/pull/94.diff","patch_url":"https://github.com/datasets/registry/pull/94.patch"},"body":"For use in conjunction with the lists in this repo, or any other list\r\nof URLs.\r\n\r\nThe utility iterates through URLs, passes them to the validator API and\r\nreturns whether they passed or failed. For failures, a link to the\r\nvalidator is provided by default, to see why the failure occurred.\r\n\r\nFor interest's sake I ran the script against the list @pdehaye posted on #92, and 145 of the 216 successfully validated. I think the updated scraper in conjunction with (semi-)automatic validation would make it much easier to maintain catalog-list.txt."},"comment":{"url":"https://api.github.com/repos/datasets/registry/issues/comments/98088234","html_url":"https://github.com/datasets/registry/pull/94#issuecomment-98088234","issue_url":"https://api.github.com/repos/datasets/registry/issues/94","id":98088234,"user":{"login":"Deiz","id":84350,"avatar_url":"https://avatars.githubusercontent.com/u/84350?v=3","gravatar_id":"","url":"https://api.github.com/users/Deiz","html_url":"https://github.com/Deiz","followers_url":"https://api.github.com/users/Deiz/followers","following_url":"https://api.github.com/users/Deiz/following{/other_user}","gists_url":"https://api.github.com/users/Deiz/gists{/gist_id}","starred_url":"https://api.github.com/users/Deiz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Deiz/subscriptions","organizations_url":"https://api.github.com/users/Deiz/orgs","repos_url":"https://api.github.com/users/Deiz/repos","events_url":"https://api.github.com/users/Deiz/events{/privacy}","received_events_url":"https://api.github.com/users/Deiz/received_events","type":"User","site_admin":false},"created_at":"2015-05-01T10:01:23Z","updated_at":"2015-05-01T10:01:23Z","body":"Right now I've got `--quiet` printing only invalid URLs, and I'm wondering if I should replace that with explicit `--valid` and `--invalid` options to avoid needing to do something like this when only interested in valid URLs:\r\n`python scripts/validate.py --quiet catalog-list.txt | sort catalog-list.txt - | uniq -u`"}}