{"action":"created","issue":{"url":"https://api.github.com/repos/opencivicdata/pupa/issues/169","labels_url":"https://api.github.com/repos/opencivicdata/pupa/issues/169/labels{/name}","comments_url":"https://api.github.com/repos/opencivicdata/pupa/issues/169/comments","events_url":"https://api.github.com/repos/opencivicdata/pupa/issues/169/events","html_url":"https://github.com/opencivicdata/pupa/issues/169","id":69909524,"number":169,"title":"Smart updates","user":{"login":"fgregg","id":536941,"avatar_url":"https://avatars.githubusercontent.com/u/536941?v=3","gravatar_id":"","url":"https://api.github.com/users/fgregg","html_url":"https://github.com/fgregg","followers_url":"https://api.github.com/users/fgregg/followers","following_url":"https://api.github.com/users/fgregg/following{/other_user}","gists_url":"https://api.github.com/users/fgregg/gists{/gist_id}","starred_url":"https://api.github.com/users/fgregg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fgregg/subscriptions","organizations_url":"https://api.github.com/users/fgregg/orgs","repos_url":"https://api.github.com/users/fgregg/repos","events_url":"https://api.github.com/users/fgregg/events{/privacy}","received_events_url":"https://api.github.com/users/fgregg/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-04-21T17:45:17Z","updated_at":"2015-04-21T17:47:15Z","closed_at":null,"body":"Chicago has a *lot* of legislation long legislative sessions (4 years). This means that it currently takes about 48 hours to scrape the site every week. \r\n\r\nThere are some strategies for a smarter scraping strategy but they all require that the scraper know, to some degree, what's already in the database. \r\n\r\nRight now the scraper doesn't know anything about the database, and that seems like it has been a good and sane thing, so how should we proceed.\r\n\r\nOne idea is for the scraper to maybe hit the OCD API? Is that a looser coupling? Thoughts please!"},"comment":{"url":"https://api.github.com/repos/opencivicdata/pupa/issues/comments/94885689","html_url":"https://github.com/opencivicdata/pupa/issues/169#issuecomment-94885689","issue_url":"https://api.github.com/repos/opencivicdata/pupa/issues/169","id":94885689,"user":{"login":"paultag","id":132447,"avatar_url":"https://avatars.githubusercontent.com/u/132447?v=3","gravatar_id":"","url":"https://api.github.com/users/paultag","html_url":"https://github.com/paultag","followers_url":"https://api.github.com/users/paultag/followers","following_url":"https://api.github.com/users/paultag/following{/other_user}","gists_url":"https://api.github.com/users/paultag/gists{/gist_id}","starred_url":"https://api.github.com/users/paultag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paultag/subscriptions","organizations_url":"https://api.github.com/users/paultag/orgs","repos_url":"https://api.github.com/users/paultag/repos","events_url":"https://api.github.com/users/paultag/events{/privacy}","received_events_url":"https://api.github.com/users/paultag/received_events","type":"User","site_admin":false},"created_at":"2015-04-21T17:47:15Z","updated_at":"2015-04-21T17:47:15Z","body":"We've been calling this 'incremental' updates internally.\r\n\r\nCC'ing @boblannon "}}