{"action":"created","issue":{"url":"https://api.github.com/repos/codelucas/newspaper/issues/128","labels_url":"https://api.github.com/repos/codelucas/newspaper/issues/128/labels{/name}","comments_url":"https://api.github.com/repos/codelucas/newspaper/issues/128/comments","events_url":"https://api.github.com/repos/codelucas/newspaper/issues/128/events","html_url":"https://github.com/codelucas/newspaper/issues/128","id":60496694,"number":128,"title":"Incorrect article extraction","user":{"login":"attodorov","id":300580,"avatar_url":"https://avatars.githubusercontent.com/u/300580?v=3","gravatar_id":"","url":"https://api.github.com/users/attodorov","html_url":"https://github.com/attodorov","followers_url":"https://api.github.com/users/attodorov/followers","following_url":"https://api.github.com/users/attodorov/following{/other_user}","gists_url":"https://api.github.com/users/attodorov/gists{/gist_id}","starred_url":"https://api.github.com/users/attodorov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/attodorov/subscriptions","organizations_url":"https://api.github.com/users/attodorov/orgs","repos_url":"https://api.github.com/users/attodorov/repos","events_url":"https://api.github.com/users/attodorov/events{/privacy}","received_events_url":"https://api.github.com/users/attodorov/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-03-10T13:10:38Z","updated_at":"2015-03-14T22:07:32Z","closed_at":null,"body":"Hey,\r\n\r\nI really like newspaper - nice job ! It has one major flaw, though - on a lot of sites, like wsj.com, reuters.com, etc. - it doesn't exract the article text precisely, but also includes left/right menus such as \"Top most read articles\" and similar things. If you use a library like the one Instapaper uses, you can extract the actual article text without any extra clutter. Here is one example (written in Java) that works pretty well:\r\n\r\nhttps://code.google.com/p/boilerpipe/\r\n\r\nThanks\r\n--Angel"},"comment":{"url":"https://api.github.com/repos/codelucas/newspaper/issues/comments/80741512","html_url":"https://github.com/codelucas/newspaper/issues/128#issuecomment-80741512","issue_url":"https://api.github.com/repos/codelucas/newspaper/issues/128","id":80741512,"user":{"login":"codelucas","id":2288709,"avatar_url":"https://avatars.githubusercontent.com/u/2288709?v=3","gravatar_id":"","url":"https://api.github.com/users/codelucas","html_url":"https://github.com/codelucas","followers_url":"https://api.github.com/users/codelucas/followers","following_url":"https://api.github.com/users/codelucas/following{/other_user}","gists_url":"https://api.github.com/users/codelucas/gists{/gist_id}","starred_url":"https://api.github.com/users/codelucas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/codelucas/subscriptions","organizations_url":"https://api.github.com/users/codelucas/orgs","repos_url":"https://api.github.com/users/codelucas/repos","events_url":"https://api.github.com/users/codelucas/events{/privacy}","received_events_url":"https://api.github.com/users/codelucas/received_events","type":"User","site_admin":false},"created_at":"2015-03-14T22:07:32Z","updated_at":"2015-03-14T22:07:32Z","body":"Give a few URL examples where newspaper fails, we'll debug it\r\n\r\nAlso, the algorithm newspaper uses is based off of Gravity Lab's goose extractor. There was a competition in extraction performance and goose's algorithm did as well or better in most areas. Maybe for the examples you are looking at newspaper fails but for others maybe it's flipped."}}