{"action":"created","issue":{"url":"https://api.github.com/repos/darkrho/scrapy-redis/issues/39","labels_url":"https://api.github.com/repos/darkrho/scrapy-redis/issues/39/labels{/name}","comments_url":"https://api.github.com/repos/darkrho/scrapy-redis/issues/39/comments","events_url":"https://api.github.com/repos/darkrho/scrapy-redis/issues/39/events","html_url":"https://github.com/darkrho/scrapy-redis/issues/39","id":106497434,"number":39,"title":"分布式爬虫爬取多个网站的新闻","user":{"login":"zhangjianbinJAVA","id":11731212,"avatar_url":"https://avatars.githubusercontent.com/u/11731212?v=3","gravatar_id":"","url":"https://api.github.com/users/zhangjianbinJAVA","html_url":"https://github.com/zhangjianbinJAVA","followers_url":"https://api.github.com/users/zhangjianbinJAVA/followers","following_url":"https://api.github.com/users/zhangjianbinJAVA/following{/other_user}","gists_url":"https://api.github.com/users/zhangjianbinJAVA/gists{/gist_id}","starred_url":"https://api.github.com/users/zhangjianbinJAVA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zhangjianbinJAVA/subscriptions","organizations_url":"https://api.github.com/users/zhangjianbinJAVA/orgs","repos_url":"https://api.github.com/users/zhangjianbinJAVA/repos","events_url":"https://api.github.com/users/zhangjianbinJAVA/events{/privacy}","received_events_url":"https://api.github.com/users/zhangjianbinJAVA/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-09-15T07:24:07Z","updated_at":"2015-09-15T07:55:32Z","closed_at":null,"body":"你好!,怎么实现多个爬虫分布式爬取多个网站的新闻文章呢,是不是每个新闻网站的建一个爬虫呢,谢谢."},"comment":{"url":"https://api.github.com/repos/darkrho/scrapy-redis/issues/comments/140312651","html_url":"https://github.com/darkrho/scrapy-redis/issues/39#issuecomment-140312651","issue_url":"https://api.github.com/repos/darkrho/scrapy-redis/issues/39","id":140312651,"user":{"login":"euangelion666","id":5959737,"avatar_url":"https://avatars.githubusercontent.com/u/5959737?v=3","gravatar_id":"","url":"https://api.github.com/users/euangelion666","html_url":"https://github.com/euangelion666","followers_url":"https://api.github.com/users/euangelion666/followers","following_url":"https://api.github.com/users/euangelion666/following{/other_user}","gists_url":"https://api.github.com/users/euangelion666/gists{/gist_id}","starred_url":"https://api.github.com/users/euangelion666/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/euangelion666/subscriptions","organizations_url":"https://api.github.com/users/euangelion666/orgs","repos_url":"https://api.github.com/users/euangelion666/repos","events_url":"https://api.github.com/users/euangelion666/events{/privacy}","received_events_url":"https://api.github.com/users/euangelion666/received_events","type":"User","site_admin":false},"created_at":"2015-09-15T07:55:32Z","updated_at":"2015-09-15T07:55:32Z","body":"我基本是这样做的，如果你比较厉害的话我觉得可以想办法识别不同的网站来源，这样或许可以一个爬虫，用不同的函数分析内容"}}