{"action":"created","issue":{"url":"https://api.github.com/repos/BVLC/caffe/issues/2940","labels_url":"https://api.github.com/repos/BVLC/caffe/issues/2940/labels{/name}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/2940/comments","events_url":"https://api.github.com/repos/BVLC/caffe/issues/2940/events","html_url":"https://github.com/BVLC/caffe/issues/2940","id":101491558,"number":2940,"title":"Asynchronous Elastic Averaging SGD","user":{"login":"cNikolaou","id":3640109,"avatar_url":"https://avatars.githubusercontent.com/u/3640109?v=3","gravatar_id":"","url":"https://api.github.com/users/cNikolaou","html_url":"https://github.com/cNikolaou","followers_url":"https://api.github.com/users/cNikolaou/followers","following_url":"https://api.github.com/users/cNikolaou/following{/other_user}","gists_url":"https://api.github.com/users/cNikolaou/gists{/gist_id}","starred_url":"https://api.github.com/users/cNikolaou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cNikolaou/subscriptions","organizations_url":"https://api.github.com/users/cNikolaou/orgs","repos_url":"https://api.github.com/users/cNikolaou/repos","events_url":"https://api.github.com/users/cNikolaou/events{/privacy}","received_events_url":"https://api.github.com/users/cNikolaou/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2015-08-17T19:19:36Z","updated_at":"2015-08-18T14:53:13Z","closed_at":null,"body":"I think that it might be a good improvement if there is an implementation of the Asynchronous Elastic Averaging SGD algorithm, as discussed in the [Deep Learning with Elastic Averaging SGD](http://arxiv.org/pdf/1412.6651v7.pdf), contributed to the Caffe.\r\n\r\nI would like to work on that and I am making this issue to ask the Caffe team about how should I work on that to make something that might be contributed to Caffe later on. I am still reviewing the last addition of the Multi-GPU Data Parallelism (#2903) and I would like to know which parts of the new code can be reused to implement asynchronous parallelism. \r\n\r\nIt would be good if you are interested in something like that and it will be beneficial (for anyone who might want to work on that) if you can help by providing some milestones and how to develop the new algorithm based on the current Caffe's code. I have never contributed code to a community project before so it will be good if anyone can help me to start out."},"comment":{"url":"https://api.github.com/repos/BVLC/caffe/issues/comments/132238675","html_url":"https://github.com/BVLC/caffe/issues/2940#issuecomment-132238675","issue_url":"https://api.github.com/repos/BVLC/caffe/issues/2940","id":132238675,"user":{"login":"thatguymike","id":3503919,"avatar_url":"https://avatars.githubusercontent.com/u/3503919?v=3","gravatar_id":"","url":"https://api.github.com/users/thatguymike","html_url":"https://github.com/thatguymike","followers_url":"https://api.github.com/users/thatguymike/followers","following_url":"https://api.github.com/users/thatguymike/following{/other_user}","gists_url":"https://api.github.com/users/thatguymike/gists{/gist_id}","starred_url":"https://api.github.com/users/thatguymike/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thatguymike/subscriptions","organizations_url":"https://api.github.com/users/thatguymike/orgs","repos_url":"https://api.github.com/users/thatguymike/repos","events_url":"https://api.github.com/users/thatguymike/events{/privacy}","received_events_url":"https://api.github.com/users/thatguymike/received_events","type":"User","site_admin":false},"created_at":"2015-08-18T14:53:13Z","updated_at":"2015-08-18T14:53:13Z","body":"Hence one of the reasons I suggested the first solution copy through host\nmemory.  Also, there are single node variants in the paper we might also\nwant to look at and likely start with the synchronous version which will be\nmuch easier to debug.\n\nOne of the issues with the async version is we will have to look at the\ndata reader again as each gpu/node becomes independent.  IO systems prefer\nlinear reads and less bouncing around.  We hit this with the original async\nsgd code.\n\nSent from mobile\nOn Aug 18, 2015 4:10 AM, \"bhack\" <notifications@github.com> wrote:\n\n> @thatguymike <https://github.com/thatguymike> I know that basically only\n> Nvidia it is supported in Caffe and at Nvidia you are dedicating resource\n> to caffe but please consider also @naibaf7 <https://github.com/naibaf7>\n> effort. I still think that we cannot support P2P on all the Opencl devices\n> so I will consider both version.\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/BVLC/caffe/issues/2940#issuecomment-132173760>.\n>\n"}}