{"action":"created","issue":{"url":"https://api.github.com/repos/apache/helix/issues/30","labels_url":"https://api.github.com/repos/apache/helix/issues/30/labels{/name}","comments_url":"https://api.github.com/repos/apache/helix/issues/30/comments","events_url":"https://api.github.com/repos/apache/helix/issues/30/events","html_url":"https://github.com/apache/helix/pull/30","id":89635058,"number":30,"title":"[HELIX-601] Allow work flow to schedule dependency jobs in parallel","user":{"login":"jicongrui","id":8269773,"avatar_url":"https://avatars.githubusercontent.com/u/8269773?v=3","gravatar_id":"","url":"https://api.github.com/users/jicongrui","html_url":"https://github.com/jicongrui","followers_url":"https://api.github.com/users/jicongrui/followers","following_url":"https://api.github.com/users/jicongrui/following{/other_user}","gists_url":"https://api.github.com/users/jicongrui/gists{/gist_id}","starred_url":"https://api.github.com/users/jicongrui/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jicongrui/subscriptions","organizations_url":"https://api.github.com/users/jicongrui/orgs","repos_url":"https://api.github.com/users/jicongrui/repos","events_url":"https://api.github.com/users/jicongrui/events{/privacy}","received_events_url":"https://api.github.com/users/jicongrui/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2015-06-19T18:56:17Z","updated_at":"2015-06-22T17:09:41Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/helix/pulls/30","html_url":"https://github.com/apache/helix/pull/30","diff_url":"https://github.com/apache/helix/pull/30.diff","patch_url":"https://github.com/apache/helix/pull/30.patch"},"body":"Currently, Helix won't schedule dependency jobs in a same work flow. For example, if Job2 depends on Job1, Job2 won't be scheduled until every partition of Job1 is completed.\r\nHowever, if some participant is very slow, then all dependency jobs is waiting for that single participant.\r\nHelix should be able to schedule multiple jobs according to a parameter.\r\nA.C.\r\n1. Introduce parallel count parameter in work flow and job queue.\r\n2. Dependency jobs can be scheduled according to the parameter (Now the parameter is always 1, so no parallel)\r\n3. If Job2 depends on Job1, Job1 is scheduled before Job2.\r\n4. No parallel jobs on the same instance. If a instance is running Job1, it won't run Job2 until Job1 is finished."},"comment":{"url":"https://api.github.com/repos/apache/helix/issues/comments/114181050","html_url":"https://github.com/apache/helix/pull/30#issuecomment-114181050","issue_url":"https://api.github.com/repos/apache/helix/issues/30","id":114181050,"user":{"login":"jicongrui","id":8269773,"avatar_url":"https://avatars.githubusercontent.com/u/8269773?v=3","gravatar_id":"","url":"https://api.github.com/users/jicongrui","html_url":"https://github.com/jicongrui","followers_url":"https://api.github.com/users/jicongrui/followers","following_url":"https://api.github.com/users/jicongrui/following{/other_user}","gists_url":"https://api.github.com/users/jicongrui/gists{/gist_id}","starred_url":"https://api.github.com/users/jicongrui/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jicongrui/subscriptions","organizations_url":"https://api.github.com/users/jicongrui/orgs","repos_url":"https://api.github.com/users/jicongrui/repos","events_url":"https://api.github.com/users/jicongrui/events{/privacy}","received_events_url":"https://api.github.com/users/jicongrui/received_events","type":"User","site_admin":false},"created_at":"2015-06-22T17:09:41Z","updated_at":"2015-06-22T17:09:41Z","body":"This is kind of hacky, but helix has no better way to handle it.\r\nThe request is something between totally out of order (workflow, no dependency) and totally order (jobDag, job2 can't run after job1).\r\nThe request hope job1 is scheduled before job2, and job2 can be scheduled even if some participants get stuck on job1"}}