{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/3233","labels_url":"https://api.github.com/repos/apache/spark/issues/3233/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/3233/comments","events_url":"https://api.github.com/repos/apache/spark/issues/3233/events","html_url":"https://github.com/apache/spark/pull/3233","id":48567814,"number":3233,"title":"[SPARK-2996] Implement userClassPathFirst for driver, yarn.","user":{"login":"vanzin","id":1694083,"avatar_url":"https://avatars.githubusercontent.com/u/1694083?v=3","gravatar_id":"","url":"https://api.github.com/users/vanzin","html_url":"https://github.com/vanzin","followers_url":"https://api.github.com/users/vanzin/followers","following_url":"https://api.github.com/users/vanzin/following{/other_user}","gists_url":"https://api.github.com/users/vanzin/gists{/gist_id}","starred_url":"https://api.github.com/users/vanzin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vanzin/subscriptions","organizations_url":"https://api.github.com/users/vanzin/orgs","repos_url":"https://api.github.com/users/vanzin/repos","events_url":"https://api.github.com/users/vanzin/events{/privacy}","received_events_url":"https://api.github.com/users/vanzin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":75,"created_at":"2014-11-12T21:53:03Z","updated_at":"2015-02-05T23:58:30Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/3233","html_url":"https://github.com/apache/spark/pull/3233","diff_url":"https://github.com/apache/spark/pull/3233.diff","patch_url":"https://github.com/apache/spark/pull/3233.patch"},"body":"Yarn's config option `spark.yarn.user.classpath.first` does not work the same way as\r\n`spark.files.userClassPathFirst`; Yarn's version is a lot more dangerous, in that it\r\nmodifies the system classpath, instead of restricting the changes to the user's class\r\nloader. So this change implements the behavior of the latter for Yarn, and deprecates\r\nthe more dangerous choice.\r\n\r\nTo be able to achieve feature-parity, I also implemented the option for drivers (the existing\r\noption only applies to executors). So now there are two options, each controlling whether\r\nto apply userClassPathFirst to the driver or executors. The old option was deprecated, and\r\naliased to the new one (`spark.executor.userClassPathFirst`).\r\n\r\nThe existing \"child-first\" class loader also had to be fixed. It didn't handle resources, and it\r\nwas also doing some things that ended up causing JVM errors depending on how things\r\nwere being called."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/73156118","html_url":"https://github.com/apache/spark/pull/3233#issuecomment-73156118","issue_url":"https://api.github.com/repos/apache/spark/issues/3233","id":73156118,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2015-02-05T23:58:30Z","updated_at":"2015-02-05T23:58:30Z","body":"Test PASSed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26863/\nTest PASSed."}}