{"action":"created","issue":{"url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/131","labels_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/131/labels{/name}","comments_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/131/comments","events_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/131/events","html_url":"https://github.com/amplab-extras/SparkR-pkg/pull/131","id":55092390,"number":131,"title":"[SPARKR-177] New R to Java bridge","user":{"login":"shivaram","id":143893,"avatar_url":"https://avatars.githubusercontent.com/u/143893?v=3","gravatar_id":"","url":"https://api.github.com/users/shivaram","html_url":"https://github.com/shivaram","followers_url":"https://api.github.com/users/shivaram/followers","following_url":"https://api.github.com/users/shivaram/following{/other_user}","gists_url":"https://api.github.com/users/shivaram/gists{/gist_id}","starred_url":"https://api.github.com/users/shivaram/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shivaram/subscriptions","organizations_url":"https://api.github.com/users/shivaram/orgs","repos_url":"https://api.github.com/users/shivaram/repos","events_url":"https://api.github.com/users/shivaram/events{/privacy}","received_events_url":"https://api.github.com/users/shivaram/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":9,"created_at":"2015-01-21T23:19:41Z","updated_at":"2015-01-25T06:55:16Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/pulls/131","html_url":"https://github.com/amplab-extras/SparkR-pkg/pull/131","diff_url":"https://github.com/amplab-extras/SparkR-pkg/pull/131.diff","patch_url":"https://github.com/amplab-extras/SparkR-pkg/pull/131.patch"},"body":"This PR introduces a new R to Java bridge to be used on the driver side. The newly introduced SparkRBackend is a netty-based server that opens a socket for communicating with R. We use Java reflection to support creating objects and invoking methods on these objects. More details can be found in https://docs.google.com/document/d/1hscJX3VAp3GW2h24GvUIP1UXfcq8gmxTqml1dcDuL6I/edit\r\n\r\nThis is joint work with @sun-rui \r\n\r\n@concretevitamin @sun-rui -- It will be great if you can review this change for code structure, understandability and make sure I haven't missed anything that needs to be updated."},"comment":{"url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/comments/71356730","html_url":"https://github.com/amplab-extras/SparkR-pkg/pull/131#issuecomment-71356730","issue_url":"https://api.github.com/repos/amplab-extras/SparkR-pkg/issues/131","id":71356730,"user":{"login":"concretevitamin","id":592670,"avatar_url":"https://avatars.githubusercontent.com/u/592670?v=3","gravatar_id":"","url":"https://api.github.com/users/concretevitamin","html_url":"https://github.com/concretevitamin","followers_url":"https://api.github.com/users/concretevitamin/followers","following_url":"https://api.github.com/users/concretevitamin/following{/other_user}","gists_url":"https://api.github.com/users/concretevitamin/gists{/gist_id}","starred_url":"https://api.github.com/users/concretevitamin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/concretevitamin/subscriptions","organizations_url":"https://api.github.com/users/concretevitamin/orgs","repos_url":"https://api.github.com/users/concretevitamin/repos","events_url":"https://api.github.com/users/concretevitamin/events{/privacy}","received_events_url":"https://api.github.com/users/concretevitamin/received_events","type":"User","site_admin":false},"created_at":"2015-01-25T06:55:16Z","updated_at":"2015-01-25T06:55:16Z","body":"A weird behavior: \r\n```\r\n> rdd <- parallelize(sc, list(c(1, 2), c(3, 4)), 2L)\r\n> collect(rdd)\r\n[[1]]\r\n[1] 1 2\r\n\r\n[[2]]\r\n[1] 3 4\r\n\r\n> rdd <- parallelize(sc, list(c(NULL, NULL), c(NULL, NULL)), 2L)\r\n> collect(rdd)\r\n[[1]]\r\nNULL\r\n\r\n[[2]]\r\nNULL\r\n```\r\nI confirmed in Spark REPL that `sc.parallelize(Seq((None, None), (None, None))).collect` should work. \r\n"}}