{"action":"created","issue":{"url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/208","labels_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/208/labels{/name}","comments_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/208/comments","events_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/208/events","html_url":"https://github.com/amplab-extras/SparkR-pkg/pull/208","id":60085743,"number":208,"title":"[SPARKR-188] Add profiling of R execution on worker side","user":{"login":"lythesia","id":1119483,"avatar_url":"https://avatars.githubusercontent.com/u/1119483?v=3","gravatar_id":"","url":"https://api.github.com/users/lythesia","html_url":"https://github.com/lythesia","followers_url":"https://api.github.com/users/lythesia/followers","following_url":"https://api.github.com/users/lythesia/following{/other_user}","gists_url":"https://api.github.com/users/lythesia/gists{/gist_id}","starred_url":"https://api.github.com/users/lythesia/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lythesia/subscriptions","organizations_url":"https://api.github.com/users/lythesia/orgs","repos_url":"https://api.github.com/users/lythesia/repos","events_url":"https://api.github.com/users/lythesia/events{/privacy}","received_events_url":"https://api.github.com/users/lythesia/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":15,"created_at":"2015-03-06T10:40:48Z","updated_at":"2015-03-18T03:58:17Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/pulls/208","html_url":"https://github.com/amplab-extras/SparkR-pkg/pull/208","diff_url":"https://github.com/amplab-extras/SparkR-pkg/pull/208.diff","patch_url":"https://github.com/amplab-extras/SparkR-pkg/pull/208.patch"},"body":"@shivaram @sun-rui \r\n\r\nFew things might need further modification:\r\n\r\n1. not sure I've understood the definition of boot/init/compute/.. time exactly, we can change them later if need.\r\n2. to support profiling for `StringRRDD`, I modified code related to `dataStream` as well as the wirte ways on R worker side. Since we need a flag to tell if following data belongs to profling or string data, I made string writing line-by-line (on worker side) instead of `writeLines`."},"comment":{"url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/comments/82729564","html_url":"https://github.com/amplab-extras/SparkR-pkg/pull/208#issuecomment-82729564","issue_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/208","id":82729564,"user":{"login":"shivaram","id":143893,"avatar_url":"https://avatars.githubusercontent.com/u/143893?v=3","gravatar_id":"","url":"https://api.github.com/users/shivaram","html_url":"https://github.com/shivaram","followers_url":"https://api.github.com/users/shivaram/followers","following_url":"https://api.github.com/users/shivaram/following{/other_user}","gists_url":"https://api.github.com/users/shivaram/gists{/gist_id}","starred_url":"https://api.github.com/users/shivaram/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shivaram/subscriptions","organizations_url":"https://api.github.com/users/shivaram/orgs","repos_url":"https://api.github.com/users/shivaram/repos","events_url":"https://api.github.com/users/shivaram/events{/privacy}","received_events_url":"https://api.github.com/users/shivaram/received_events","type":"User","site_admin":false},"created_at":"2015-03-18T03:58:17Z","updated_at":"2015-03-18T03:58:17Z","body":"So one thing we could do is just use the stream in mixed mode ? For example in startStdInThread we write out some values using a DataOutputStream and then use a PrintWriter. Similarly we could first read the special lengths in binary format and after we reach the right number of bytes we could switch ?"}}