{"action":"closed","issue":{"url":"https://api.github.com/repos/Murodese/pynab/issues/148","labels_url":"https://api.github.com/repos/Murodese/pynab/issues/148/labels{/name}","comments_url":"https://api.github.com/repos/Murodese/pynab/issues/148/comments","events_url":"https://api.github.com/repos/Murodese/pynab/issues/148/events","html_url":"https://github.com/Murodese/pynab/issues/148","id":55616806,"number":148,"title":"Finding the date of old posts for backfills breaks sometimes","user":{"login":"Murodese","id":86297,"avatar_url":"https://avatars.githubusercontent.com/u/86297?v=3","gravatar_id":"","url":"https://api.github.com/users/Murodese","html_url":"https://github.com/Murodese","followers_url":"https://api.github.com/users/Murodese/followers","following_url":"https://api.github.com/users/Murodese/following{/other_user}","gists_url":"https://api.github.com/users/Murodese/gists{/gist_id}","starred_url":"https://api.github.com/users/Murodese/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Murodese/subscriptions","organizations_url":"https://api.github.com/users/Murodese/orgs","repos_url":"https://api.github.com/users/Murodese/repos","events_url":"https://api.github.com/users/Murodese/events{/privacy}","received_events_url":"https://api.github.com/users/Murodese/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Murodese/pynab/labels/bug","name":"bug","color":"fc2929"}],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2015-01-27T13:23:12Z","updated_at":"2015-02-11T13:56:58Z","closed_at":"2015-02-11T13:56:58Z","body":"Sometimes it'll get stuck endlessly, because articles have been deleted and so on. I had a really good idea for finding an article by date quickly, but it doesn't work properly:\r\n\r\n```\r\n_, count, first, last, _ = self.connection.group(group_name)\r\n\r\n        # get first, last and target dates\r\n        candidate_post = None\r\n        target_date = datetime.datetime.now(pytz.utc) - datetime.timedelta(days)\r\n        bottom_date = first_date = self.post_date(group_name, first+1000)\r\n        top_date = last_date = self.post_date(group_name, last)\r\n        bottom = first\r\n        top = last\r\n\r\n        # iterative, obviously\r\n        while True:\r\n            # do something like a binary search\r\n            # find the percentage-point of target date between first and last dates\r\n            # ie. start |-------T---| end = ~70%\r\n            # so we'd find the post number ~70% through the message count\r\n            target = target_date - bottom_date\r\n            total = top_date - bottom_date\r\n            perc = target.total_seconds() / total.total_seconds()\r\n\r\n            while True:\r\n                candidate_post = int(abs(bottom + ((top - bottom) * perc)))\r\n                print('candidate: {}% {}'.format(perc, candidate_post))\r\n                candidate_date = self.post_date(group_name, candidate_post)\r\n                if candidate_date:\r\n                    break\r\n                else:\r\n                    print('not found')\r\n                    addition = (random.choice([-1, 1]) / 100) * perc\r\n                    if perc + addition > 1.0:\r\n                        perc -= addition\r\n                    elif perc - addition < 0.0:\r\n                        perc += addition\r\n                    else:\r\n                        perc += addition\r\n\r\n            print('new candidate date: {}'.format(candidate_date))\r\n\r\n            # one day leeway\r\n            if abs(target_date - candidate_date) < datetime.timedelta(days=1):\r\n                break\r\n\r\n            if perc >= 0.5:\r\n                bottom = candidate_post\r\n                bottom_date = candidate_date\r\n            else:\r\n                top = candidate_post\r\n                top_date = candidate_date\r\n\r\n        return candidate_post\r\n```\r\n\r\nThe problem is that some NNTP servers are busted, ie. alt.binaries.teevee's first article number is 2, which is December 2009, while article ~7million or so is October 2009. Need to think of a way around that."}}