{"action":"created","issue":{"url":"https://api.github.com/repos/codalab/codalab/issues/816","labels_url":"https://api.github.com/repos/codalab/codalab/issues/816/labels{/name}","comments_url":"https://api.github.com/repos/codalab/codalab/issues/816/comments","events_url":"https://api.github.com/repos/codalab/codalab/issues/816/events","html_url":"https://github.com/codalab/codalab/issues/816","id":54158997,"number":816,"title":"Multiple tracks","user":{"login":"madclam","id":7594063,"avatar_url":"https://avatars.githubusercontent.com/u/7594063?v=3","gravatar_id":"","url":"https://api.github.com/users/madclam","html_url":"https://github.com/madclam","followers_url":"https://api.github.com/users/madclam/followers","following_url":"https://api.github.com/users/madclam/following{/other_user}","gists_url":"https://api.github.com/users/madclam/gists{/gist_id}","starred_url":"https://api.github.com/users/madclam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/madclam/subscriptions","organizations_url":"https://api.github.com/users/madclam/orgs","repos_url":"https://api.github.com/users/madclam/repos","events_url":"https://api.github.com/users/madclam/events{/privacy}","received_events_url":"https://api.github.com/users/madclam/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/codalab/codalab/labels/Competition","name":"Competition","color":"009800"}],"state":"open","locked":false,"assignee":{"login":"ckcollab","id":2185159,"avatar_url":"https://avatars.githubusercontent.com/u/2185159?v=3","gravatar_id":"","url":"https://api.github.com/users/ckcollab","html_url":"https://github.com/ckcollab","followers_url":"https://api.github.com/users/ckcollab/followers","following_url":"https://api.github.com/users/ckcollab/following{/other_user}","gists_url":"https://api.github.com/users/ckcollab/gists{/gist_id}","starred_url":"https://api.github.com/users/ckcollab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ckcollab/subscriptions","organizations_url":"https://api.github.com/users/ckcollab/orgs","repos_url":"https://api.github.com/users/ckcollab/repos","events_url":"https://api.github.com/users/ckcollab/events{/privacy}","received_events_url":"https://api.github.com/users/ckcollab/received_events","type":"User","site_admin":false},"milestone":{"url":"https://api.github.com/repos/codalab/codalab/milestones/26","labels_url":"https://api.github.com/repos/codalab/codalab/milestones/26/labels","id":890406,"number":26,"title":"Sprint 6: Competitions - January 2015","description":"Sprint 6 Competition issues during the month of January 2015","creator":{"login":"ErickWa","id":4361560,"avatar_url":"https://avatars.githubusercontent.com/u/4361560?v=3","gravatar_id":"","url":"https://api.github.com/users/ErickWa","html_url":"https://github.com/ErickWa","followers_url":"https://api.github.com/users/ErickWa/followers","following_url":"https://api.github.com/users/ErickWa/following{/other_user}","gists_url":"https://api.github.com/users/ErickWa/gists{/gist_id}","starred_url":"https://api.github.com/users/ErickWa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ErickWa/subscriptions","organizations_url":"https://api.github.com/users/ErickWa/orgs","repos_url":"https://api.github.com/users/ErickWa/repos","events_url":"https://api.github.com/users/ErickWa/events{/privacy}","received_events_url":"https://api.github.com/users/ErickWa/received_events","type":"User","site_admin":false},"open_issues":9,"closed_issues":9,"state":"open","created_at":"2014-12-03T18:38:52Z","updated_at":"2015-01-23T00:43:15Z","due_on":"2015-01-31T08:00:00Z","closed_at":null},"comments":2,"created_at":"2015-01-13T07:28:42Z","updated_at":"2015-01-24T10:58:30Z","closed_at":null,"body":"There is a limitation of Codalab presently that forces users to submit 1 entry for all the tasks when we have multiple datasets/tasks in a given phase. For example, in each phase of the AutoML challenge we have 5 tasks. Therefore, the computational time limit is globally computed on the 5 tasks. \r\n\r\nWhat we really meant to have is a time limit (which is different) for each task. At present the global limit is the sum of the individual task limits and it is up to the participants to manage their time. But we cannot check how they do their time management. Therefore, it is possible for them to re-allocate time differently.\r\n\r\nWe talked in the past of having parallel tracks within a phase. One solution would be to launch multiple processes with the metadata file when you make a submission as a participant.\r\n\r\nFor instance, now a submission must include in the metadata file:\r\ncommand: python $program/run.py $input $output\r\n\r\nCould we have:\r\ncommand: python $program/run.py $input/set1 $output/set1\r\ncommand: python $program/run.py $input/set2 $output/set2\r\ncommand: python $program/run.py $input/set3 $output/set3\r\ncommand: python $program/run.py $input/set4 $output/set4\r\ncommand: python $program/run.py $input/set5 $output/set5\r\n\r\nthat is 5 separate commands for the 5 datasets.\r\n\r\nI think this would solve the problem. Are there any possible potential side effects?\r\n"},"comment":{"url":"https://api.github.com/repos/codalab/codalab/issues/comments/71310957","html_url":"https://github.com/codalab/codalab/issues/816#issuecomment-71310957","issue_url":"https://api.github.com/repos/codalab/codalab/issues/816","id":71310957,"user":{"login":"madclam","id":7594063,"avatar_url":"https://avatars.githubusercontent.com/u/7594063?v=3","gravatar_id":"","url":"https://api.github.com/users/madclam","html_url":"https://github.com/madclam","followers_url":"https://api.github.com/users/madclam/followers","following_url":"https://api.github.com/users/madclam/following{/other_user}","gists_url":"https://api.github.com/users/madclam/gists{/gist_id}","starred_url":"https://api.github.com/users/madclam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/madclam/subscriptions","organizations_url":"https://api.github.com/users/madclam/orgs","repos_url":"https://api.github.com/users/madclam/repos","events_url":"https://api.github.com/users/madclam/events{/privacy}","received_events_url":"https://api.github.com/users/madclam/received_events","type":"User","site_admin":false},"created_at":"2015-01-24T10:58:30Z","updated_at":"2015-01-24T10:58:30Z","body":"I would like to add that we should leave open the possibility of executing in this way also multiple jobs that are executed in sequence, one feeding its output to the next one, instead of being independent jobs:\r\ncommand: python $program/run1.py $input $output/subdir1\r\ncommand: python $program/run2.py $output/subdir1 $output/subdir2\r\ncommand: python $program/run3.py $output/subdir2 $output/subdir3\r\n\r\nThe question is: how do we create the appropriate directory structure. For the log files, we can concatenate all the logs.\r\n"}}