{"action":"created","issue":{"url":"https://api.github.com/repos/nilearn/nilearn/issues/832","labels_url":"https://api.github.com/repos/nilearn/nilearn/issues/832/labels{/name}","comments_url":"https://api.github.com/repos/nilearn/nilearn/issues/832/comments","events_url":"https://api.github.com/repos/nilearn/nilearn/issues/832/events","html_url":"https://github.com/nilearn/nilearn/pull/832","id":115607968,"number":832,"title":"WIP: NeuroVault downloader","user":{"login":"bcipolli","id":4072455,"avatar_url":"https://avatars.githubusercontent.com/u/4072455?v=3","gravatar_id":"","url":"https://api.github.com/users/bcipolli","html_url":"https://github.com/bcipolli","followers_url":"https://api.github.com/users/bcipolli/followers","following_url":"https://api.github.com/users/bcipolli/following{/other_user}","gists_url":"https://api.github.com/users/bcipolli/gists{/gist_id}","starred_url":"https://api.github.com/users/bcipolli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bcipolli/subscriptions","organizations_url":"https://api.github.com/users/bcipolli/orgs","repos_url":"https://api.github.com/users/bcipolli/repos","events_url":"https://api.github.com/users/bcipolli/events{/privacy}","received_events_url":"https://api.github.com/users/bcipolli/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-11-06T23:00:16Z","updated_at":"2015-11-10T17:05:00Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/nilearn/nilearn/pulls/832","html_url":"https://github.com/nilearn/nilearn/pull/832","diff_url":"https://github.com/nilearn/nilearn/pull/832.diff","patch_url":"https://github.com/nilearn/nilearn/pull/832.patch"},"body":"This is an enhancement for #640. Not quite done, but far enough along to talk about the API I've implemented.\r\n\r\nTo do:\r\n* [x] Implement an API for downloading selected images from http://neurovault.org/\r\n* [x] Create dataset documentation\r\n* [x] Create a basic example for downloading a subset of files.\r\n* [ ] Create a more complex ICA-based example.\r\n* [ ] Create tests\r\n* [ ] Remove dependency on `requests` - use `urllib`\r\n* [ ] Read from disk when offline.\r\n* [ ] Detect outdated cached collection or image metadata or data.\r\n\r\nNeurovault contains images in collections. Collections can be filtered via a few querystring arguments. Images to download and return should be limited by filters on collection and image metadata.\r\n\r\nTo do this, I've included `collection_filters` and `image_filters` arguments.\r\n* If `isinstance(collection_filters, dict)`, the filters are used as querystring argument filters.\r\n* If `isinstance(*_filters, collections.Iterable)` (i.e. a list), the filters are assumed to be functions that receive a collection or image metadata, and returns `True` if the collection's images should be queried, or if the image should be downloaded.\r\n\r\nThe function returns a `Bunch` of three:\r\n* `collections`: a dict (key: collection_id) of collection metadata for matched collections.\r\n* `images`: a list of image metadata (including local file paths) for matched & downloaded images.\r\n* `func_files`: a list of paths to downloaded files. Could be removed, but I kept it as many other functions return a list of paths to images.\r\n\r\nThe example I've begun with is a `stop-go` task meta-analysis, ported from:\r\nhttps://github.com/NeuroVault/neurovault_analysis/blob/master/Stop%20vs.%20Go%20metaanalysis.ipynb\r\n\r\nNote that I avoided trying to convert things to a `pandas` dataframe. The previous examples downloaded all images and filtered using `pandas`; however, all `nilearn` dataset fetchers filter before returning to the user. It was simple enough to allow lambda functions as filters to make very powerful pre-fetch filtering available to the user, so I went that route.\r\n\r\nLooking for feedback on the overall API design to start. @chrisfilo and anyone else around! "},"comment":{"url":"https://api.github.com/repos/nilearn/nilearn/issues/comments/155488432","html_url":"https://github.com/nilearn/nilearn/pull/832#issuecomment-155488432","issue_url":"https://api.github.com/repos/nilearn/nilearn/issues/832","id":155488432,"user":{"login":"bcipolli","id":4072455,"avatar_url":"https://avatars.githubusercontent.com/u/4072455?v=3","gravatar_id":"","url":"https://api.github.com/users/bcipolli","html_url":"https://github.com/bcipolli","followers_url":"https://api.github.com/users/bcipolli/followers","following_url":"https://api.github.com/users/bcipolli/following{/other_user}","gists_url":"https://api.github.com/users/bcipolli/gists{/gist_id}","starred_url":"https://api.github.com/users/bcipolli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bcipolli/subscriptions","organizations_url":"https://api.github.com/users/bcipolli/orgs","repos_url":"https://api.github.com/users/bcipolli/repos","events_url":"https://api.github.com/users/bcipolli/events{/privacy}","received_events_url":"https://api.github.com/users/bcipolli/received_events","type":"User","site_admin":false},"created_at":"2015-11-10T17:05:00Z","updated_at":"2015-11-10T17:05:00Z","body":"@AlexandreAbraham in order to remove the `requests` dependency, I needed to make changes to `_fetch_files` (#838 and #839; add overwrite option, modify output). Not sure this is a net positive; the code was simpler all around when using `requests`. Glad to get feedback!"}}