{"action":"created","issue":{"url":"https://api.github.com/repos/pyvideo/steve/issues/24","labels_url":"https://api.github.com/repos/pyvideo/steve/issues/24/labels{/name}","comments_url":"https://api.github.com/repos/pyvideo/steve/issues/24/comments","events_url":"https://api.github.com/repos/pyvideo/steve/issues/24/events","html_url":"https://github.com/pyvideo/steve/issues/24","id":73165849,"number":24,"title":"write new scraper","user":{"login":"willkg","id":820826,"avatar_url":"https://avatars.githubusercontent.com/u/820826?v=3","gravatar_id":"","url":"https://api.github.com/users/willkg","html_url":"https://github.com/willkg","followers_url":"https://api.github.com/users/willkg/followers","following_url":"https://api.github.com/users/willkg/following{/other_user}","gists_url":"https://api.github.com/users/willkg/gists{/gist_id}","starred_url":"https://api.github.com/users/willkg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/willkg/subscriptions","organizations_url":"https://api.github.com/users/willkg/orgs","repos_url":"https://api.github.com/users/willkg/repos","events_url":"https://api.github.com/users/willkg/events{/privacy}","received_events_url":"https://api.github.com/users/willkg/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-05-05T00:29:22Z","updated_at":"2015-06-15T14:12:31Z","closed_at":null,"body":"@codersquid started some scraping code to replace vidscraper.\r\n\r\nhttps://gist.github.com/codersquid/8c1aa67f29c97ad0572f\r\n\r\nWe should base the new scraper on that. If we make it pluggable, that'll make it easier to use/extend later, too."},"comment":{"url":"https://api.github.com/repos/pyvideo/steve/issues/comments/112083159","html_url":"https://github.com/pyvideo/steve/issues/24#issuecomment-112083159","issue_url":"https://api.github.com/repos/pyvideo/steve/issues/24","id":112083159,"user":{"login":"willkg","id":820826,"avatar_url":"https://avatars.githubusercontent.com/u/820826?v=3","gravatar_id":"","url":"https://api.github.com/users/willkg","html_url":"https://github.com/willkg","followers_url":"https://api.github.com/users/willkg/followers","following_url":"https://api.github.com/users/willkg/following{/other_user}","gists_url":"https://api.github.com/users/willkg/gists{/gist_id}","starred_url":"https://api.github.com/users/willkg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/willkg/subscriptions","organizations_url":"https://api.github.com/users/willkg/orgs","repos_url":"https://api.github.com/users/willkg/repos","events_url":"https://api.github.com/users/willkg/events{/privacy}","received_events_url":"https://api.github.com/users/willkg/received_events","type":"User","site_admin":false},"created_at":"2015-06-15T14:12:30Z","updated_at":"2015-06-15T14:12:30Z","body":"\"summary\" is really important since it shows up in the video lists and in some cases it's really the only thing that tells you what the video is about. Generally, I've seen people putting short descriptions on YouTube which work well as summaries in pyvideo. I think the only huge exception to that is the data conferences. It should be pretty easy to write a script to move them post-fetch.\r\n\r\nI'm game for switching this, but it seems like it's a better default in regards to how much work is required for post processing.\r\n\r\nFor YouTube tags, I suggest writing a post-fetch script that wipes out useless ones. Occasionally we've seen useful ones and if the scraper doesn't capture them, then there's no way to do it by hand. It's easier to remove things post-fetch than add things that didn't exist.\r\n\r\nDo you still want to change this? If so, writing up a PR is probably the easiest thing to do. Otherwise I'll get to it when I get to it."}}