{"action":"created","issue":{"url":"https://api.github.com/repos/nidata/nidata/issues/17","labels_url":"https://api.github.com/repos/nidata/nidata/issues/17/labels{/name}","comments_url":"https://api.github.com/repos/nidata/nidata/issues/17/comments","events_url":"https://api.github.com/repos/nidata/nidata/issues/17/events","html_url":"https://github.com/nidata/nidata/pull/17","id":93835864,"number":17,"title":"Add AmazonS3Fetcher, HcpDataset","user":{"login":"bcipolli","id":4072455,"avatar_url":"https://avatars.githubusercontent.com/u/4072455?v=3","gravatar_id":"","url":"https://api.github.com/users/bcipolli","html_url":"https://github.com/bcipolli","followers_url":"https://api.github.com/users/bcipolli/followers","following_url":"https://api.github.com/users/bcipolli/following{/other_user}","gists_url":"https://api.github.com/users/bcipolli/gists{/gist_id}","starred_url":"https://api.github.com/users/bcipolli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bcipolli/subscriptions","organizations_url":"https://api.github.com/users/bcipolli/orgs","repos_url":"https://api.github.com/users/bcipolli/repos","events_url":"https://api.github.com/users/bcipolli/events{/privacy}","received_events_url":"https://api.github.com/users/bcipolli/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-07-08T16:37:24Z","updated_at":"2015-07-13T23:32:25Z","closed_at":"2015-07-09T23:19:05Z","pull_request":{"url":"https://api.github.com/repos/nidata/nidata/pulls/17","html_url":"https://github.com/nidata/nidata/pull/17","diff_url":"https://github.com/nidata/nidata/pull/17.diff","patch_url":"https://github.com/nidata/nidata/pull/17.patch"},"body":"New object structure:\r\n\r\n* `Fetchers` - encapsulate how to get data with a given protocol (s3, http, XNAT, etc) and website (Amazon, OpenFMRI, CRCNS, etc).\r\n* `Datasets` - encapsulate individual research studies, with a simple function interface for fetching data on a per-subject basis. Returns a standardized output format.\r\n\r\nThis PR creates and `AmazonS3Fetcher`, and uses it to fetch `HCP` data. Currently, only implemented a bit of diffusion fetching as a proof of concept.\r\n\r\nExample usage:\r\n```python\r\nfrom nidata.multimodal import HcpDataset\r\nprint HcpDataset(profile_name='hcp').fetch(n_subjects=1, data_types=['diff'])\r\n```\r\n@arokem , this is a sloppy attempt to push your IPython notebook into an object wrapper."},"comment":{"url":"https://api.github.com/repos/nidata/nidata/issues/comments/121090445","html_url":"https://github.com/nidata/nidata/pull/17#issuecomment-121090445","issue_url":"https://api.github.com/repos/nidata/nidata/issues/17","id":121090445,"user":{"login":"bcipolli","id":4072455,"avatar_url":"https://avatars.githubusercontent.com/u/4072455?v=3","gravatar_id":"","url":"https://api.github.com/users/bcipolli","html_url":"https://github.com/bcipolli","followers_url":"https://api.github.com/users/bcipolli/followers","following_url":"https://api.github.com/users/bcipolli/following{/other_user}","gists_url":"https://api.github.com/users/bcipolli/gists{/gist_id}","starred_url":"https://api.github.com/users/bcipolli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bcipolli/subscriptions","organizations_url":"https://api.github.com/users/bcipolli/orgs","repos_url":"https://api.github.com/users/bcipolli/repos","events_url":"https://api.github.com/users/bcipolli/events{/privacy}","received_events_url":"https://api.github.com/users/bcipolli/received_events","type":"User","site_admin":false},"created_at":"2015-07-13T23:32:25Z","updated_at":"2015-07-13T23:32:25Z","body":"@arokem I'm actually already going to deprecate the use of `boto` already, as it doesn't have resume capabilities. HTTP fetchers do, and public amazon S3 buckets can be accessed through `https`.\r\n\r\nI've got an HCP fetcher through http initially working today, though the paths don't match those via the S3 bucket. Will be sending an email to the user list to get further info.\r\n\r\nAll that to say, testing of `http` fetchers is already implemented (via mock).\r\n\r\nHow's all that sound?"}}