{"action":"created","issue":{"url":"https://api.github.com/repos/realpython/book2-exercises/issues/55","labels_url":"https://api.github.com/repos/realpython/book2-exercises/issues/55/labels{/name}","comments_url":"https://api.github.com/repos/realpython/book2-exercises/issues/55/comments","events_url":"https://api.github.com/repos/realpython/book2-exercises/issues/55/events","html_url":"https://github.com/realpython/book2-exercises/issues/55","id":106563193,"number":55,"title":"About web-crawling in hackernews","user":{"login":"ni8mr","id":6823100,"avatar_url":"https://avatars.githubusercontent.com/u/6823100?v=3","gravatar_id":"","url":"https://api.github.com/users/ni8mr","html_url":"https://github.com/ni8mr","followers_url":"https://api.github.com/users/ni8mr/followers","following_url":"https://api.github.com/users/ni8mr/following{/other_user}","gists_url":"https://api.github.com/users/ni8mr/gists{/gist_id}","starred_url":"https://api.github.com/users/ni8mr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ni8mr/subscriptions","organizations_url":"https://api.github.com/users/ni8mr/orgs","repos_url":"https://api.github.com/users/ni8mr/repos","events_url":"https://api.github.com/users/ni8mr/events{/privacy}","received_events_url":"https://api.github.com/users/ni8mr/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-09-15T14:02:51Z","updated_at":"2015-09-16T12:50:55Z","closed_at":"2015-09-16T12:50:55Z","body":"I have been trying to crawl on *hackernews* according to the code provided. I have faced some problem. I have reconstructed the code in *spider.py* like this-\r\n\r\nFrom\r\n\r\n    from scrapy.spider import BaseSpider\r\n    ...............\r\n    class MySpider(BaseSpider):\r\n    .........\r\n\r\nTo\r\n\r\n    from scrapy.spiders import Spider\r\n    ..................\r\n    class MySpider(Spider):\r\n\r\n and than it has worked. Is this a problem? Should i make a pull request?\r\n\r\nAnother question about the output of the crawler- Here is the output:\r\n\r\n    2015-09-15 19:30:04 [scrapy] INFO: Scrapy 1.0.3 started (bot: hackernews)\r\n    2015-09-15 19:30:04 [scrapy] INFO: Optional features available: ssl, http11\r\n    2015-09-15 19:30:04 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE':'hackernews.spiders', 'SPIDER_MODULES': ['hackernews.spiders'],'BOT_NAME': 'hackernews'}\r\n    2015-09-15 19:30:04 [scrapy] INFO: Enabled extensions: CloseSpider, TelnetConsole, LogStats, CoreStats, SpiderState\r\n    2015-09-15 19:30:04 [scrapy] INFO: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, MetaRefreshMiddleware, HttpCompressionMiddleware, RedirectMiddleware, CookiesMiddleware,ChunkedTransferMiddleware, DownloaderStats\r\n    2015-09-15 19:30:04 [scrapy] INFO: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware\r\n    2015-09-15 19:30:04 [scrapy] INFO: Enabled item pipelines:\r\n    2015-09-15 19:30:04 [scrapy] INFO: Spider opened\r\n    2015-09-15 19:30:04 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)\r\n    2015-09-15 19:30:04 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023\r\n    2015-09-15 19:30:04 [scrapy] INFO: Closing spider (finished)\r\n    2015-09-15 19:30:04 [scrapy] INFO: Dumping Scrapy stats:\r\n    {'finish_reason': 'finished','finish_time': datetime.datetime(2015, 9, 15, 13, 30, 4, 895000), 'log_count/DEBUG': 1,'log_count/INFO': 7,'start_time': datetime.datetime(2015, 9, 15, 13, 30, 4, 886000)}\r\n    2015-09-15 19:30:04 [scrapy] INFO: Spider closed (finished)\r\n\r\n\r\nIs the output ok? I can't understand the meaning of the output. Shouldn't we see some *title* and *text*? \r\n"},"comment":{"url":"https://api.github.com/repos/realpython/book2-exercises/issues/comments/140732361","html_url":"https://github.com/realpython/book2-exercises/issues/55#issuecomment-140732361","issue_url":"https://api.github.com/repos/realpython/book2-exercises/issues/55","id":140732361,"user":{"login":"mjhea0","id":2018167,"avatar_url":"https://avatars.githubusercontent.com/u/2018167?v=3","gravatar_id":"","url":"https://api.github.com/users/mjhea0","html_url":"https://github.com/mjhea0","followers_url":"https://api.github.com/users/mjhea0/followers","following_url":"https://api.github.com/users/mjhea0/following{/other_user}","gists_url":"https://api.github.com/users/mjhea0/gists{/gist_id}","starred_url":"https://api.github.com/users/mjhea0/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mjhea0/subscriptions","organizations_url":"https://api.github.com/users/mjhea0/orgs","repos_url":"https://api.github.com/users/mjhea0/repos","events_url":"https://api.github.com/users/mjhea0/events{/privacy}","received_events_url":"https://api.github.com/users/mjhea0/received_events","type":"User","site_admin":false},"created_at":"2015-09-16T12:50:55Z","updated_at":"2015-09-16T12:50:55Z","body":"I'm a different version - `pip install Scrapy==0.16.4`. Need to update. Thanks!"}}