{"action":"created","issue":{"url":"https://api.github.com/repos/ukwa/shine/issues/10","labels_url":"https://api.github.com/repos/ukwa/shine/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/ukwa/shine/issues/10/comments","events_url":"https://api.github.com/repos/ukwa/shine/issues/10/events","html_url":"https://github.com/ukwa/shine/issues/10","id":37286978,"number":10,"title":"Export of list of results","user":{"login":"peterwebster","id":5520710,"avatar_url":"https://avatars.githubusercontent.com/u/5520710?v=3","gravatar_id":"","url":"https://api.github.com/users/peterwebster","html_url":"https://github.com/peterwebster","followers_url":"https://api.github.com/users/peterwebster/followers","following_url":"https://api.github.com/users/peterwebster/following{/other_user}","gists_url":"https://api.github.com/users/peterwebster/gists{/gist_id}","starred_url":"https://api.github.com/users/peterwebster/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/peterwebster/subscriptions","organizations_url":"https://api.github.com/users/peterwebster/orgs","repos_url":"https://api.github.com/users/peterwebster/repos","events_url":"https://api.github.com/users/peterwebster/events{/privacy}","received_events_url":"https://api.github.com/users/peterwebster/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/ukwa/shine/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":{"login":"peterwebster","id":5520710,"avatar_url":"https://avatars.githubusercontent.com/u/5520710?v=3","gravatar_id":"","url":"https://api.github.com/users/peterwebster","html_url":"https://github.com/peterwebster","followers_url":"https://api.github.com/users/peterwebster/followers","following_url":"https://api.github.com/users/peterwebster/following{/other_user}","gists_url":"https://api.github.com/users/peterwebster/gists{/gist_id}","starred_url":"https://api.github.com/users/peterwebster/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/peterwebster/subscriptions","organizations_url":"https://api.github.com/users/peterwebster/orgs","repos_url":"https://api.github.com/users/peterwebster/repos","events_url":"https://api.github.com/users/peterwebster/events{/privacy}","received_events_url":"https://api.github.com/users/peterwebster/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/ukwa/shine/milestones/1","html_url":"https://github.com/ukwa/shine/milestones/Big%20Data%20Stage%201","labels_url":"https://api.github.com/repos/ukwa/shine/milestones/1/labels","id":712394,"number":1,"title":"Big Data Stage 1","description":"","creator":{"login":"anjackson","id":87843,"avatar_url":"https://avatars.githubusercontent.com/u/87843?v=3","gravatar_id":"","url":"https://api.github.com/users/anjackson","html_url":"https://github.com/anjackson","followers_url":"https://api.github.com/users/anjackson/followers","following_url":"https://api.github.com/users/anjackson/following{/other_user}","gists_url":"https://api.github.com/users/anjackson/gists{/gist_id}","starred_url":"https://api.github.com/users/anjackson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anjackson/subscriptions","organizations_url":"https://api.github.com/users/anjackson/orgs","repos_url":"https://api.github.com/users/anjackson/repos","events_url":"https://api.github.com/users/anjackson/events{/privacy}","received_events_url":"https://api.github.com/users/anjackson/received_events","type":"User","site_admin":false},"open_issues":8,"closed_issues":11,"state":"open","created_at":"2014-07-07T13:20:48Z","updated_at":"2014-10-10T14:28:18Z","due_on":null,"closed_at":null},"comments":32,"created_at":"2014-07-07T16:43:29Z","updated_at":"2015-03-16T19:07:18Z","closed_at":null,"body":"For any query, users need to be able to export as CSV the list of results, one line per result.\r\n\r\nExact composition to be define, but I imagine two options:  Full and Brief\r\n\r\nFull = all metadata associated with a resource, including facet values [which may not be straightforward if there are multiple values.]\r\n\r\nBrief = title, crawl date, URL"},"comment":{"url":"https://api.github.com/repos/ukwa/shine/issues/comments/81876822","html_url":"https://github.com/ukwa/shine/issues/10#issuecomment-81876822","issue_url":"https://api.github.com/repos/ukwa/shine/issues/10","id":81876822,"user":{"login":"tokee","id":199454,"avatar_url":"https://avatars.githubusercontent.com/u/199454?v=3","gravatar_id":"","url":"https://api.github.com/users/tokee","html_url":"https://github.com/tokee","followers_url":"https://api.github.com/users/tokee/followers","following_url":"https://api.github.com/users/tokee/following{/other_user}","gists_url":"https://api.github.com/users/tokee/gists{/gist_id}","starred_url":"https://api.github.com/users/tokee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tokee/subscriptions","organizations_url":"https://api.github.com/users/tokee/orgs","repos_url":"https://api.github.com/users/tokee/repos","events_url":"https://api.github.com/users/tokee/events{/privacy}","received_events_url":"https://api.github.com/users/tokee/received_events","type":"User","site_admin":false},"created_at":"2015-03-16T19:07:18Z","updated_at":"2015-03-16T19:07:18Z","body":"Internally in Solr, direct paging to large page numbers is discouraged as the search time grows fast (there's an internal priority queue of size page*pagesize). A quick look in Shine.java indicates that this is done when doing a CSV-export.\r\n\r\nFor export, the \"old\" way is to use cursorMark (https://cwiki.apache.org/confluence/display/solr/Pagination+of+Results) and the new way is to use streaming export (https://cwiki.apache.org/confluence/display/solr/Exporting+Result+Sets). However, the new way requires Solr 4.10 and that the export fields are DocValues, so pagination with cursorMark seem to be the conservative choise."}}