{"action":"created","issue":{"url":"https://api.github.com/repos/SeSaMe-NUS/Trendspedia2/issues/13","labels_url":"https://api.github.com/repos/SeSaMe-NUS/Trendspedia2/issues/13/labels{/name}","comments_url":"https://api.github.com/repos/SeSaMe-NUS/Trendspedia2/issues/13/comments","events_url":"https://api.github.com/repos/SeSaMe-NUS/Trendspedia2/issues/13/events","html_url":"https://github.com/SeSaMe-NUS/Trendspedia2/pull/13","id":78447662,"number":13,"title":"Fixed Hot URLs not showing up","user":{"login":"allwinpb","id":4397532,"avatar_url":"https://avatars.githubusercontent.com/u/4397532?v=3","gravatar_id":"","url":"https://api.github.com/users/allwinpb","html_url":"https://github.com/allwinpb","followers_url":"https://api.github.com/users/allwinpb/followers","following_url":"https://api.github.com/users/allwinpb/following{/other_user}","gists_url":"https://api.github.com/users/allwinpb/gists{/gist_id}","starred_url":"https://api.github.com/users/allwinpb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/allwinpb/subscriptions","organizations_url":"https://api.github.com/users/allwinpb/orgs","repos_url":"https://api.github.com/users/allwinpb/repos","events_url":"https://api.github.com/users/allwinpb/events{/privacy}","received_events_url":"https://api.github.com/users/allwinpb/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-05-20T09:26:04Z","updated_at":"2015-05-21T08:22:12Z","closed_at":"2015-05-20T11:00:55Z","pull_request":{"url":"https://api.github.com/repos/SeSaMe-NUS/Trendspedia2/pulls/13","html_url":"https://github.com/SeSaMe-NUS/Trendspedia2/pull/13","diff_url":"https://github.com/SeSaMe-NUS/Trendspedia2/pull/13.diff","patch_url":"https://github.com/SeSaMe-NUS/Trendspedia2/pull/13.patch"},"body":"The issue there was that certain urls in the top 500 hot urls contained unicode characters not representable in ascii, and hence the str() calls always failed in the /hotMaterials path, giving out a 500 error. I've removed the str calls and refactored the code up a bit.\r\n\r\nThe `json_util` line is to correctly parse python datetime objects into JSON. Please refer to this link for details: http://stackoverflow.com/questions/11875770/how-to-overcome-datetime-datetime-not-json-serializable-in-python\r\n\r\nAlso removed an errant `return None` in twitter/views.py which was used for development but somehow got on the repo.\r\n\r\nI've updated DEPLOY.md with a better way to task celery workers such that celery spawns and destroys worker threads depending on load.\r\n\r\nI've added a crawled property to the Hot document to denote whether the URL has been crawled and summarized.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/SeSaMe-NUS/Trendspedia2/issues/comments/104175991","html_url":"https://github.com/SeSaMe-NUS/Trendspedia2/pull/13#issuecomment-104175991","issue_url":"https://api.github.com/repos/SeSaMe-NUS/Trendspedia2/issues/13","id":104175991,"user":{"login":"shubhamgoyal","id":774891,"avatar_url":"https://avatars.githubusercontent.com/u/774891?v=3","gravatar_id":"","url":"https://api.github.com/users/shubhamgoyal","html_url":"https://github.com/shubhamgoyal","followers_url":"https://api.github.com/users/shubhamgoyal/followers","following_url":"https://api.github.com/users/shubhamgoyal/following{/other_user}","gists_url":"https://api.github.com/users/shubhamgoyal/gists{/gist_id}","starred_url":"https://api.github.com/users/shubhamgoyal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shubhamgoyal/subscriptions","organizations_url":"https://api.github.com/users/shubhamgoyal/orgs","repos_url":"https://api.github.com/users/shubhamgoyal/repos","events_url":"https://api.github.com/users/shubhamgoyal/events{/privacy}","received_events_url":"https://api.github.com/users/shubhamgoyal/received_events","type":"User","site_admin":false},"created_at":"2015-05-21T08:22:12Z","updated_at":"2015-05-21T08:22:12Z","body":"@kangism I don't think deleting the URLs is a solution. Even if we delete URLs now, backlog will still happen in the future because we will only increase our crawling rate from now on as we get more tokens and improve background crawling. I will work with @allwinpb to find a solution and then update you."}}