{"action":"opened","issue":{"url":"https://api.github.com/repos/DemandCube/NeverwinterDP/issues/737","labels_url":"https://api.github.com/repos/DemandCube/NeverwinterDP/issues/737/labels{/name}","comments_url":"https://api.github.com/repos/DemandCube/NeverwinterDP/issues/737/comments","events_url":"https://api.github.com/repos/DemandCube/NeverwinterDP/issues/737/events","html_url":"https://github.com/DemandCube/NeverwinterDP/issues/737","id":84440141,"number":737,"title":"Improve dataflow api and configuration","user":{"login":"tuan08","id":1083869,"avatar_url":"https://avatars.githubusercontent.com/u/1083869?v=3","gravatar_id":"","url":"https://api.github.com/users/tuan08","html_url":"https://github.com/tuan08","followers_url":"https://api.github.com/users/tuan08/followers","following_url":"https://api.github.com/users/tuan08/following{/other_user}","gists_url":"https://api.github.com/users/tuan08/gists{/gist_id}","starred_url":"https://api.github.com/users/tuan08/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tuan08/subscriptions","organizations_url":"https://api.github.com/users/tuan08/orgs","repos_url":"https://api.github.com/users/tuan08/repos","events_url":"https://api.github.com/users/tuan08/events{/privacy}","received_events_url":"https://api.github.com/users/tuan08/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/DemandCube/NeverwinterDP/labels/0%20-%20Backlog","name":"0 - Backlog","color":"CCCCCC"}],"state":"open","locked":false,"assignee":{"login":"tuan08","id":1083869,"avatar_url":"https://avatars.githubusercontent.com/u/1083869?v=3","gravatar_id":"","url":"https://api.github.com/users/tuan08","html_url":"https://github.com/tuan08","followers_url":"https://api.github.com/users/tuan08/followers","following_url":"https://api.github.com/users/tuan08/following{/other_user}","gists_url":"https://api.github.com/users/tuan08/gists{/gist_id}","starred_url":"https://api.github.com/users/tuan08/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tuan08/subscriptions","organizations_url":"https://api.github.com/users/tuan08/orgs","repos_url":"https://api.github.com/users/tuan08/repos","events_url":"https://api.github.com/users/tuan08/events{/privacy}","received_events_url":"https://api.github.com/users/tuan08/received_events","type":"User","site_admin":false},"milestone":null,"comments":0,"created_at":"2015-06-03T08:46:26Z","updated_at":"2015-06-03T08:46:26Z","closed_at":null,"body":"1.  Currently when a task try to read the data from a stream, if it cannot read more the data from the stream, it consider the task is finished. This strategy is\ncorrect for stream like hdfs , S3. But it is not correct for stream like kafka , since the data can be avaiable after a period of time. How should we handle this situation.  \n\n@smorin  @KingOfPoptart \n\n2. In order to get the best performance , we should have an executor dedicate to a stream or task. This should avoid the task switch. But we need to have enough machine and thread to dedicate an executor to a thread or we switch the task when it complete only. Should we update the api, configuration to support different way of switching the task.\n\r\n\r\n<!---\r\n@huboard:{\"order\":2.6706284284949434e-65}\r\n-->\r\n"}}