{"action":"created","issue":{"url":"https://api.github.com/repos/vufind-org/vufind/issues/414","labels_url":"https://api.github.com/repos/vufind-org/vufind/issues/414/labels{/name}","comments_url":"https://api.github.com/repos/vufind-org/vufind/issues/414/comments","events_url":"https://api.github.com/repos/vufind-org/vufind/issues/414/events","html_url":"https://github.com/vufind-org/vufind/pull/414","id":95133305,"number":414,"title":"Config setting to limit deep pagination for results","user":{"login":"lahmann","id":7081217,"avatar_url":"https://avatars.githubusercontent.com/u/7081217?v=3","gravatar_id":"","url":"https://api.github.com/users/lahmann","html_url":"https://github.com/lahmann","followers_url":"https://api.github.com/users/lahmann/followers","following_url":"https://api.github.com/users/lahmann/following{/other_user}","gists_url":"https://api.github.com/users/lahmann/gists{/gist_id}","starred_url":"https://api.github.com/users/lahmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lahmann/subscriptions","organizations_url":"https://api.github.com/users/lahmann/orgs","repos_url":"https://api.github.com/users/lahmann/repos","events_url":"https://api.github.com/users/lahmann/events{/privacy}","received_events_url":"https://api.github.com/users/lahmann/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-07-15T08:21:37Z","updated_at":"2015-07-15T12:46:18Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/vufind-org/vufind/pulls/414","html_url":"https://github.com/vufind-org/vufind/pull/414","diff_url":"https://github.com/vufind-org/vufind/pull/414.diff","patch_url":"https://github.com/vufind-org/vufind/pull/414.patch"},"body":"Demian, this is a fairly simple implementation of a setting in `searches.ini` to limit maximum pages for results in order to avoid high loads on Solr due to deep pagination. Although this might be handled differently with [Solr 4.7+](https://issues.apache.org/jira/browse/SOLR-5463) this implementation can be helpful for shared-index setups. Especially if the catalogues are being crawled this can cause very high loads plus it's doubtable that users are likely to decent deeper than 20 pages of results. E.g. [Google does this too](https://www.google.com/?gfe_rd=cr&ei=AxemVdiEAsaK8QfJ6bCgCg&gws_rd=cr#q=vufind&start=1000), so it is at least a common practise."},"comment":{"url":"https://api.github.com/repos/vufind-org/vufind/issues/comments/121604611","html_url":"https://github.com/vufind-org/vufind/pull/414#issuecomment-121604611","issue_url":"https://api.github.com/repos/vufind-org/vufind/issues/414","id":121604611,"user":{"login":"demiankatz","id":309069,"avatar_url":"https://avatars.githubusercontent.com/u/309069?v=3","gravatar_id":"","url":"https://api.github.com/users/demiankatz","html_url":"https://github.com/demiankatz","followers_url":"https://api.github.com/users/demiankatz/followers","following_url":"https://api.github.com/users/demiankatz/following{/other_user}","gists_url":"https://api.github.com/users/demiankatz/gists{/gist_id}","starred_url":"https://api.github.com/users/demiankatz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/demiankatz/subscriptions","organizations_url":"https://api.github.com/users/demiankatz/orgs","repos_url":"https://api.github.com/users/demiankatz/repos","events_url":"https://api.github.com/users/demiankatz/events{/privacy}","received_events_url":"https://api.github.com/users/demiankatz/received_events","type":"User","site_admin":false},"created_at":"2015-07-15T12:46:18Z","updated_at":"2015-07-15T12:46:18Z","body":"André,\r\n\r\nWe actually already have a feature like this for Summon because the API sets a hard limit on number of results. This may be implemented for some other backends as well (though I can't remember off the top of my head). You can see the setting here:\r\n\r\nhttps://github.com/vufind-org/vufind/blob/master/config/vufind/Summon.ini#L77\r\n\r\nBecause of the specifics of the Summon API, we were forced to limit to a total maximum number of results rather than a total number of pages (since page size can be configured to be variable, and we don't want to allow users to get deeper into the set simply by making the page size bigger). I think this is probably a best practice across the board because it's more consistent when page size is variable.\r\n\r\nRather than implementing two different mechanisms, I think it would probably make more sense to make the Solr limit consistent with the Summon one (and any others that already exist). Perhaps this could either be bumped into the base class or turned into a Trait to reduce redundancy.\r\n\r\nWhat do you think? (And apologies if there are details in your code that are not covered by the Summon implementation I mentioned -- I haven't looked at it in detail, since I figured it was better to have this discussion before reviewing anything)."}}