{"action":"created","issue":{"url":"https://api.github.com/repos/cBio/cbio-cluster/issues/220","labels_url":"https://api.github.com/repos/cBio/cbio-cluster/issues/220/labels{/name}","comments_url":"https://api.github.com/repos/cBio/cbio-cluster/issues/220/comments","events_url":"https://api.github.com/repos/cBio/cbio-cluster/issues/220/events","html_url":"https://github.com/cBio/cbio-cluster/issues/220","id":59853891,"number":220,"title":"gpu requests","user":{"login":"karalets","id":1401135,"avatar_url":"https://avatars.githubusercontent.com/u/1401135?v=3","gravatar_id":"","url":"https://api.github.com/users/karalets","html_url":"https://github.com/karalets","followers_url":"https://api.github.com/users/karalets/followers","following_url":"https://api.github.com/users/karalets/following{/other_user}","gists_url":"https://api.github.com/users/karalets/gists{/gist_id}","starred_url":"https://api.github.com/users/karalets/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/karalets/subscriptions","organizations_url":"https://api.github.com/users/karalets/orgs","repos_url":"https://api.github.com/users/karalets/repos","events_url":"https://api.github.com/users/karalets/events{/privacy}","received_events_url":"https://api.github.com/users/karalets/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2015-03-04T20:07:41Z","updated_at":"2015-03-04T20:19:02Z","closed_at":null,"body":"So, I am sure I have seen discussions related to this from Fall last year, but I did not quite understand what I need to do to fix it.\r\n\r\nMy problem is the following:\r\nIf I request a gpu using qlogin -l gpus=1 I do indeed get an online session on a node that has gpus with lots of unused resources (load and memory).\r\nUp to this point everything is great. The nvidia script reveals though that they are in use.\r\n\r\nSo, theano (yes, that evil) refuses to start a job on the gpu citing a memory error. By now I know this is not a memory error, it simply means the device/gpu is not available because someone else is running a job on it and it is blocked from access.\r\n\r\nMy question boils down to this simple one: how can I request an (online) session on a node with a gpu that is free for me to use and not blocked by someone? I assume I can try it by just resubmitting scripts until I get lucky, but this is highly unsatisfactory.\r\n\r\nAny solutions in sight?\r\n\r\nBest,\r\n\r\nTheo\r\n"},"comment":{"url":"https://api.github.com/repos/cBio/cbio-cluster/issues/comments/77238270","html_url":"https://github.com/cBio/cbio-cluster/issues/220#issuecomment-77238270","issue_url":"https://api.github.com/repos/cBio/cbio-cluster/issues/220","id":77238270,"user":{"login":"KjongLehmann","id":2054148,"avatar_url":"https://avatars.githubusercontent.com/u/2054148?v=3","gravatar_id":"","url":"https://api.github.com/users/KjongLehmann","html_url":"https://github.com/KjongLehmann","followers_url":"https://api.github.com/users/KjongLehmann/followers","following_url":"https://api.github.com/users/KjongLehmann/following{/other_user}","gists_url":"https://api.github.com/users/KjongLehmann/gists{/gist_id}","starred_url":"https://api.github.com/users/KjongLehmann/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/KjongLehmann/subscriptions","organizations_url":"https://api.github.com/users/KjongLehmann/orgs","repos_url":"https://api.github.com/users/KjongLehmann/repos","events_url":"https://api.github.com/users/KjongLehmann/events{/privacy}","received_events_url":"https://api.github.com/users/KjongLehmann/received_events","type":"User","site_admin":false},"created_at":"2015-03-04T20:19:02Z","updated_at":"2015-03-04T20:19:02Z","body":"Andre wrote it but I believe that it supports all Torque arguments.  Just need to specify -q gpu and you should be in business.\n\nOn Mar 4, 2015, at 3:15 PM, tatarsky <notifications@github.com> wrote:\n\n> While this qlogin assign items to the gpu queue if asked? Because thats why he's not getting the gpu reserved....see pgrinaways comment.\n> \n> —\n> Reply to this email directly or view it on GitHub.\n> \n\n"}}