{"action":"created","issue":{"url":"https://api.github.com/repos/lenskit/lenskit/issues/821","labels_url":"https://api.github.com/repos/lenskit/lenskit/issues/821/labels{/name}","comments_url":"https://api.github.com/repos/lenskit/lenskit/issues/821/comments","events_url":"https://api.github.com/repos/lenskit/lenskit/issues/821/events","html_url":"https://github.com/lenskit/lenskit/issues/821","id":116366116,"number":821,"title":"Provide way to pre-load data to remove loading time from training time","user":{"login":"mdekstrand","id":186612,"avatar_url":"https://avatars.githubusercontent.com/u/186612?v=3","gravatar_id":"","url":"https://api.github.com/users/mdekstrand","html_url":"https://github.com/mdekstrand","followers_url":"https://api.github.com/users/mdekstrand/followers","following_url":"https://api.github.com/users/mdekstrand/following{/other_user}","gists_url":"https://api.github.com/users/mdekstrand/gists{/gist_id}","starred_url":"https://api.github.com/users/mdekstrand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdekstrand/subscriptions","organizations_url":"https://api.github.com/users/mdekstrand/orgs","repos_url":"https://api.github.com/users/mdekstrand/repos","events_url":"https://api.github.com/users/mdekstrand/events{/privacy}","received_events_url":"https://api.github.com/users/mdekstrand/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/lenskit/lenskit/labels/Evaluator","name":"Evaluator","color":"5319e7"}],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/lenskit/lenskit/milestones/23","html_url":"https://github.com/lenskit/lenskit/milestones/3.0","labels_url":"https://api.github.com/repos/lenskit/lenskit/milestones/23/labels","id":433278,"number":23,"title":"3.0","description":"LensKit 3.0, upcoming major change + long-term stable version.","creator":null,"open_issues":64,"closed_issues":76,"state":"open","created_at":"2013-09-21T00:54:48Z","updated_at":"2015-11-11T18:40:22Z","due_on":"2015-11-30T06:00:00Z","closed_at":null},"comments":9,"created_at":"2015-11-11T16:08:14Z","updated_at":"2015-11-11T18:45:08Z","closed_at":null,"body":"We should have a way to tell the evaluator to pre-fetch data so that the time reading data from disk is not considered in the training time of the algorithm. This is a problem because the first algorithm to read the data will have the extra training time induced by a cold cache.\r\n\r\nFor both the binary rating DAO and the text DAO, we could do a pre-scan of the ratings before training the model to warm up the OS caches/page tables.\r\n\r\nThis should be part of a broader 'timing mode' that turns off component caching so we can have accurate times."},"comment":{"url":"https://api.github.com/repos/lenskit/lenskit/issues/comments/155873809","html_url":"https://github.com/lenskit/lenskit/issues/821#issuecomment-155873809","issue_url":"https://api.github.com/repos/lenskit/lenskit/issues/821","id":155873809,"user":{"login":"mdekstrand","id":186612,"avatar_url":"https://avatars.githubusercontent.com/u/186612?v=3","gravatar_id":"","url":"https://api.github.com/users/mdekstrand","html_url":"https://github.com/mdekstrand","followers_url":"https://api.github.com/users/mdekstrand/followers","following_url":"https://api.github.com/users/mdekstrand/following{/other_user}","gists_url":"https://api.github.com/users/mdekstrand/gists{/gist_id}","starred_url":"https://api.github.com/users/mdekstrand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mdekstrand/subscriptions","organizations_url":"https://api.github.com/users/mdekstrand/orgs","repos_url":"https://api.github.com/users/mdekstrand/repos","events_url":"https://api.github.com/users/mdekstrand/events{/privacy}","received_events_url":"https://api.github.com/users/mdekstrand/received_events","type":"User","site_admin":false},"created_at":"2015-11-11T18:45:08Z","updated_at":"2015-11-11T18:45:08Z","body":"I have raised #822 to track the overall work on supporting reliable timing, so we can use this ticket to specifically focus on the data pre-fetching support."}}