{"action":"created","issue":{"url":"https://api.github.com/repos/gimoya/FloraWebScrapeR/issues/2","labels_url":"https://api.github.com/repos/gimoya/FloraWebScrapeR/issues/2/labels{/name}","comments_url":"https://api.github.com/repos/gimoya/FloraWebScrapeR/issues/2/comments","events_url":"https://api.github.com/repos/gimoya/FloraWebScrapeR/issues/2/events","html_url":"https://github.com/gimoya/FloraWebScrapeR/issues/2","id":56770952,"number":2,"title":"Suggestion for a time delay between scraping iterations","user":{"login":"roberdorb","id":10003836,"avatar_url":"https://avatars.githubusercontent.com/u/10003836?v=3","gravatar_id":"","url":"https://api.github.com/users/roberdorb","html_url":"https://github.com/roberdorb","followers_url":"https://api.github.com/users/roberdorb/followers","following_url":"https://api.github.com/users/roberdorb/following{/other_user}","gists_url":"https://api.github.com/users/roberdorb/gists{/gist_id}","starred_url":"https://api.github.com/users/roberdorb/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/roberdorb/subscriptions","organizations_url":"https://api.github.com/users/roberdorb/orgs","repos_url":"https://api.github.com/users/roberdorb/repos","events_url":"https://api.github.com/users/roberdorb/events{/privacy}","received_events_url":"https://api.github.com/users/roberdorb/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-02-06T05:08:17Z","updated_at":"2015-02-06T08:07:40Z","closed_at":null,"body":"It may appear to be a style question to some, but for me it is more of a small necessity; the integration of a time delay in an automated online request loop, be it a scraper or a request-series over an application programming interface (API), e.g. google API, or anything else.\r\n\r\nThis idea is based on the care for scalability of the script. If just a few people use the script, it should be no big deal for the webserver, but I really do not know what the result would be, if many website requests were fired off in high frequency :-) through many scripts. Weighing the benefits I gain from lovely scraping scripts like this, it may really pay off, and be even seen as a kind of gesture of respect towards the website (floraweb, OK it is a filthy governmental website, so maybe .... but then again, I think they really present a wide collection of not so bad botanical information, I myself am surprised ... ) .\r\n\r\nComing back to what I started with, it really is basically a style question, but from my point of view, should not be, but more than that a matter of responsability for the script, to ensure scalability towards high-usage-compatibility.\r\n\r\nI would, if I may continue on such maybe touchy issue, consider something like a 1 second gap to be a really OK time for a delay between the requests.\r\n\r\nTherefore adding a *Sys.sleep(1)* within the *main_function* could be pondered upon, and I think, as stated before, really does not hurt. More accurately; to really not spend more time than 1 second for one request would be to get *startTime <- Sys.time()* in the beginning of *main_function* and get *endTime <- Sys.time()* at the end again, build the difference *diffTime <- (endTime - startTime)[[1]]* and then work with that to determine whether a sleep time could be considered, and if so, how much. This could, all in all, look something like this;\r\n\r\n*main_function <- function(x) {*\r\n*startTime <- Sys.time()*\r\n...\r\n...\r\n*endTime <- Sys.time()*\r\n*diffTime <- (endTime - startTime)[[1]]*\r\n*if(diffTime < 1){Sys.sleep(1-diffTime)}*\r\n*}*\r\n\r\nPlease, let me know what you think. I am not sure, if this is just a question of preference, or matters to the functionality of the code - therefore I was hesitant to actually post this suggestion.\r\n\r\nGreetings"},"comment":{"url":"https://api.github.com/repos/gimoya/FloraWebScrapeR/issues/comments/73200380","html_url":"https://github.com/gimoya/FloraWebScrapeR/issues/2#issuecomment-73200380","issue_url":"https://api.github.com/repos/gimoya/FloraWebScrapeR/issues/2","id":73200380,"user":{"login":"gimoya","id":1248198,"avatar_url":"https://avatars.githubusercontent.com/u/1248198?v=3","gravatar_id":"","url":"https://api.github.com/users/gimoya","html_url":"https://github.com/gimoya","followers_url":"https://api.github.com/users/gimoya/followers","following_url":"https://api.github.com/users/gimoya/following{/other_user}","gists_url":"https://api.github.com/users/gimoya/gists{/gist_id}","starred_url":"https://api.github.com/users/gimoya/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gimoya/subscriptions","organizations_url":"https://api.github.com/users/gimoya/orgs","repos_url":"https://api.github.com/users/gimoya/repos","events_url":"https://api.github.com/users/gimoya/events{/privacy}","received_events_url":"https://api.github.com/users/gimoya/received_events","type":"User","site_admin":false},"created_at":"2015-02-06T08:07:40Z","updated_at":"2015-02-06T08:07:40Z","body":"I appreciate yout input! And, yes you're absolutely right with what you're saying! Please make a pull request and I will merge it!"}}