{"action":"created","issue":{"url":"https://api.github.com/repos/NVIDIA/caffe/issues/15","labels_url":"https://api.github.com/repos/NVIDIA/caffe/issues/15/labels{/name}","comments_url":"https://api.github.com/repos/NVIDIA/caffe/issues/15/comments","events_url":"https://api.github.com/repos/NVIDIA/caffe/issues/15/events","html_url":"https://github.com/NVIDIA/caffe/issues/15","id":99375606,"number":15,"title":"Multi GPU on AWS g2.8x large","user":{"login":"lolz0r","id":4228021,"avatar_url":"https://avatars.githubusercontent.com/u/4228021?v=3","gravatar_id":"","url":"https://api.github.com/users/lolz0r","html_url":"https://github.com/lolz0r","followers_url":"https://api.github.com/users/lolz0r/followers","following_url":"https://api.github.com/users/lolz0r/following{/other_user}","gists_url":"https://api.github.com/users/lolz0r/gists{/gist_id}","starred_url":"https://api.github.com/users/lolz0r/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lolz0r/subscriptions","organizations_url":"https://api.github.com/users/lolz0r/orgs","repos_url":"https://api.github.com/users/lolz0r/repos","events_url":"https://api.github.com/users/lolz0r/events{/privacy}","received_events_url":"https://api.github.com/users/lolz0r/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2015-08-06T07:29:43Z","updated_at":"2015-08-06T20:45:51Z","closed_at":null,"body":"Hello,\r\nI am unable to run this fork in multi GPU mode when using a AWS g2.8x large instance. \r\n\r\nWhen executing the actual training with the flag ' -gpu all' I see errors relating to CUDA peer communication. After doing some research on this issue it appears that CUDA peer communication is disabled on AWS g2.8x large. \r\n\r\nIs it possible/practical to disable CUDA peer communication so that we can utilize multiGPU training on AWS?\r\n\r\nThank you for any help!"},"comment":{"url":"https://api.github.com/repos/NVIDIA/caffe/issues/comments/128502982","html_url":"https://github.com/NVIDIA/caffe/issues/15#issuecomment-128502982","issue_url":"https://api.github.com/repos/NVIDIA/caffe/issues/15","id":128502982,"user":{"login":"lukeyeager","id":687269,"avatar_url":"https://avatars.githubusercontent.com/u/687269?v=3","gravatar_id":"","url":"https://api.github.com/users/lukeyeager","html_url":"https://github.com/lukeyeager","followers_url":"https://api.github.com/users/lukeyeager/followers","following_url":"https://api.github.com/users/lukeyeager/following{/other_user}","gists_url":"https://api.github.com/users/lukeyeager/gists{/gist_id}","starred_url":"https://api.github.com/users/lukeyeager/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lukeyeager/subscriptions","organizations_url":"https://api.github.com/users/lukeyeager/orgs","repos_url":"https://api.github.com/users/lukeyeager/repos","events_url":"https://api.github.com/users/lukeyeager/events{/privacy}","received_events_url":"https://api.github.com/users/lukeyeager/received_events","type":"User","site_admin":false},"created_at":"2015-08-06T20:45:51Z","updated_at":"2015-08-06T20:45:51Z","body":"Go ahead and try the latest changes to the `caffe-0.12` branch. The latest version in that branch is `v0.12.2`. There are some other bugfix commits which may address your problem. See here:\r\n\r\nhttps://github.com/NVIDIA/caffe/commits/caffe-0.12\r\n\r\nIf you're still seeing the same issue, this is definitely a bug. But I've run `v0.12.2` on a g2.8xlarge instance several times without any issue."}}