{"action":"created","issue":{"url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/706","labels_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/706/labels{/name}","comments_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/706/comments","events_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/706/events","html_url":"https://github.com/cooperative-computing-lab/cctools/issues/706","id":68146930,"number":706,"title":"Fixing Work Queue Foreman","user":{"login":"nhazekam","id":5231159,"avatar_url":"https://avatars.githubusercontent.com/u/5231159?v=3","gravatar_id":"","url":"https://api.github.com/users/nhazekam","html_url":"https://github.com/nhazekam","followers_url":"https://api.github.com/users/nhazekam/followers","following_url":"https://api.github.com/users/nhazekam/following{/other_user}","gists_url":"https://api.github.com/users/nhazekam/gists{/gist_id}","starred_url":"https://api.github.com/users/nhazekam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nhazekam/subscriptions","organizations_url":"https://api.github.com/users/nhazekam/orgs","repos_url":"https://api.github.com/users/nhazekam/repos","events_url":"https://api.github.com/users/nhazekam/events{/privacy}","received_events_url":"https://api.github.com/users/nhazekam/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cooperative-computing-lab/cctools/labels/enhancement","name":"enhancement","color":"84b6eb"},{"url":"https://api.github.com/repos/cooperative-computing-lab/cctools/labels/Work%20Queue","name":"Work Queue","color":"eb6420"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-04-13T17:15:23Z","updated_at":"2015-05-11T14:47:49Z","closed_at":null,"body":"As per the discussion with @btovar and @dthain, this ticket should outline different areas of work to better model the interactions of the master<->foreman<->workers. The goal is to allow existing functionality to work with the foreman structure, and allow additional functionality to utilize foreman as it is added. To fix the foreman here are key areas to understand and model.\r\n\r\n- Communication between master, foreman, and workers on task information\r\n  - Currently, the master send task information out to foreman and workers, but the only information that is retrieved is the done status and the resulting files.\r\n  - The end goal would allow information to move back and forth from master and worker through chains of foreman without losing important information about execution due to the encapsulating nature of foreman task management.\r\n  - Initially, this may be addressed by passing a doc about task execution back to the master, using the same structure as exists for sending task information to workers.\r\n\r\n- Management of policy attributes in the master\r\n  - Currently, policies created and managed by the master are not communicated or enforced by the foreman. This causes some of the functionality to either not work function differently from expectations when foreman are used.\r\n  - The end goal would be a standard way to communicated policies to foreman. This should work for chains of foreman and be clear how Work Queue refreshes these policies so behavior is understood.\r\n\r\n- Revise structure to model internal structure as closely to API calls where appropriate\r\n  - As we revise and add functionality to the master and foreman, to maintain clarity the API functions should be clearly related to the structures.\r\n  - Though this is not an issue currently, this is included to help model the above work.\r\n\r\nLet me know if I missed something substantial or if the direction is not quite right. "},"comment":{"url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/comments/100931254","html_url":"https://github.com/cooperative-computing-lab/cctools/issues/706#issuecomment-100931254","issue_url":"https://api.github.com/repos/cooperative-computing-lab/cctools/issues/706","id":100931254,"user":{"login":"nhazekam","id":5231159,"avatar_url":"https://avatars.githubusercontent.com/u/5231159?v=3","gravatar_id":"","url":"https://api.github.com/users/nhazekam","html_url":"https://github.com/nhazekam","followers_url":"https://api.github.com/users/nhazekam/followers","following_url":"https://api.github.com/users/nhazekam/following{/other_user}","gists_url":"https://api.github.com/users/nhazekam/gists{/gist_id}","starred_url":"https://api.github.com/users/nhazekam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nhazekam/subscriptions","organizations_url":"https://api.github.com/users/nhazekam/orgs","repos_url":"https://api.github.com/users/nhazekam/repos","events_url":"https://api.github.com/users/nhazekam/events{/privacy}","received_events_url":"https://api.github.com/users/nhazekam/received_events","type":"User","site_admin":false},"created_at":"2015-05-11T14:47:49Z","updated_at":"2015-05-11T14:47:49Z","body":"For specific examples of how foreman breaks existing functionality, there are two that come to mind.\r\n\r\n1. Blacklist - Blacklisting workers if currently managed at the master. If there is a worker misbehaving or not executing properly, the master can blacklist the worker. However, if the worker is used by the foreman, the master does not know the name of the worker, and therefore can only blacklist the foreman losing all of its workers.\r\n\r\n2. Fast Abort - To utilize fast abort currently both the master and the foreman must have fast abort turned on. However, in this case the statistics used to determine an aborted task are calculated separately at the master and all foreman, so the behavior is not as expected.\r\n\r\n@btovar Please comment on other cases where foreman breaks existing functionality."}}