{"action":"created","issue":{"url":"https://api.github.com/repos/flux-framework/flux-core/issues/357","labels_url":"https://api.github.com/repos/flux-framework/flux-core/issues/357/labels{/name}","comments_url":"https://api.github.com/repos/flux-framework/flux-core/issues/357/comments","events_url":"https://api.github.com/repos/flux-framework/flux-core/issues/357/events","html_url":"https://github.com/flux-framework/flux-core/issues/357","id":102671600,"number":357,"title":"PMI unstable","user":{"login":"trws","id":660149,"avatar_url":"https://avatars.githubusercontent.com/u/660149?v=3","gravatar_id":"","url":"https://api.github.com/users/trws","html_url":"https://github.com/trws","followers_url":"https://api.github.com/users/trws/followers","following_url":"https://api.github.com/users/trws/following{/other_user}","gists_url":"https://api.github.com/users/trws/gists{/gist_id}","starred_url":"https://api.github.com/users/trws/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trws/subscriptions","organizations_url":"https://api.github.com/users/trws/orgs","repos_url":"https://api.github.com/users/trws/repos","events_url":"https://api.github.com/users/trws/events{/privacy}","received_events_url":"https://api.github.com/users/trws/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":{"url":"https://api.github.com/repos/flux-framework/flux-core/milestones/2","html_url":"https://github.com/flux-framework/flux-core/milestones/Flux%20in-job%20scheduling%20tool%20-%20minimal","labels_url":"https://api.github.com/repos/flux-framework/flux-core/milestones/2/labels","id":1086398,"number":2,"title":"Flux in-job scheduling tool - minimal","description":"This aggregates the issues necessary to get flux to the point that it can be used as a scheduler inside an allocation from an outer resource manager such as SLURM or Torque.  It should be the first published test release for flux as a hierarchical resource manager for a single user on a relatively small scale.","creator":{"login":"trws","id":660149,"avatar_url":"https://avatars.githubusercontent.com/u/660149?v=3","gravatar_id":"","url":"https://api.github.com/users/trws","html_url":"https://github.com/trws","followers_url":"https://api.github.com/users/trws/followers","following_url":"https://api.github.com/users/trws/following{/other_user}","gists_url":"https://api.github.com/users/trws/gists{/gist_id}","starred_url":"https://api.github.com/users/trws/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/trws/subscriptions","organizations_url":"https://api.github.com/users/trws/orgs","repos_url":"https://api.github.com/users/trws/repos","events_url":"https://api.github.com/users/trws/events{/privacy}","received_events_url":"https://api.github.com/users/trws/received_events","type":"User","site_admin":false},"open_issues":11,"closed_issues":7,"state":"open","created_at":"2015-04-27T22:59:02Z","updated_at":"2015-08-23T23:31:09Z","due_on":null,"closed_at":null},"comments":18,"created_at":"2015-08-23T23:31:09Z","updated_at":"2015-08-25T04:33:41Z","closed_at":null,"body":"Segfaults, exits, barrier failures and hangs all pop up using the PMI interface for jobs of over about 4 tasks.  Sometimes it will manage as many as 10, 40 never seems to work.  Tested under both wreckrun frontend and capacitor, both break the same way.\r\n\r\nSome processes get an error 4, meaning key not found, on a key read.  Others get an error on `kvs_get_dir`, and still others enter a barrier and never leave.  Have not yet tracked down the source of this issue, but this is a nasty one."},"comment":{"url":"https://api.github.com/repos/flux-framework/flux-core/issues/comments/134471016","html_url":"https://github.com/flux-framework/flux-core/issues/357#issuecomment-134471016","issue_url":"https://api.github.com/repos/flux-framework/flux-core/issues/357","id":134471016,"user":{"login":"grondo","id":741970,"avatar_url":"https://avatars.githubusercontent.com/u/741970?v=3","gravatar_id":"","url":"https://api.github.com/users/grondo","html_url":"https://github.com/grondo","followers_url":"https://api.github.com/users/grondo/followers","following_url":"https://api.github.com/users/grondo/following{/other_user}","gists_url":"https://api.github.com/users/grondo/gists{/gist_id}","starred_url":"https://api.github.com/users/grondo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/grondo/subscriptions","organizations_url":"https://api.github.com/users/grondo/orgs","repos_url":"https://api.github.com/users/grondo/repos","events_url":"https://api.github.com/users/grondo/events{/privacy}","received_events_url":"https://api.github.com/users/grondo/received_events","type":"User","site_admin":false},"created_at":"2015-08-25T04:33:41Z","updated_at":"2015-08-25T04:33:41Z","body":"Ok, I'll take a look first thing in the morning. Output from wreckrun might also help, when doing the \"fake\" resource assignment wreckrun prints something like:\r\n```\r\n\u0001\u0001$ src/cmd/flux wreckrun -N8 -n16 printenv FLUX_LWJ_NTASKS                                                                                                 \r\nwreckrun: 0.001s: Sending LWJ request for 16 tasks (cmdline \"printenv FLUX_LWJ_NTASKS\")\r\nwreckrun: 0.008s: Registered jobid 3\r\nwreckrun: Allocating 16 tasks across 8 available nodes..\r\nwreckrun: tasks per node: node[0-7]: 2\r\n```\r\n\r\nThe last line of which shows how the `alloc_tasks_hack` in flux-wreckrun tries to distribute tasks. (I noticed right away that this doesn't work at all if total tasks isn't evenly divisible into the number of ranks...)"}}