{"action":"created","issue":{"url":"https://api.github.com/repos/broadinstitute/gatk/issues/1070","labels_url":"https://api.github.com/repos/broadinstitute/gatk/issues/1070/labels{/name}","comments_url":"https://api.github.com/repos/broadinstitute/gatk/issues/1070/comments","events_url":"https://api.github.com/repos/broadinstitute/gatk/issues/1070/events","html_url":"https://github.com/broadinstitute/gatk/issues/1070","id":114168478,"number":1070,"title":"`SparkContextFactory` should not take a `master` parameter","user":{"login":"laserson","id":118141,"avatar_url":"https://avatars.githubusercontent.com/u/118141?v=3","gravatar_id":"","url":"https://api.github.com/users/laserson","html_url":"https://github.com/laserson","followers_url":"https://api.github.com/users/laserson/followers","following_url":"https://api.github.com/users/laserson/following{/other_user}","gists_url":"https://api.github.com/users/laserson/gists{/gist_id}","starred_url":"https://api.github.com/users/laserson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/laserson/subscriptions","organizations_url":"https://api.github.com/users/laserson/orgs","repos_url":"https://api.github.com/users/laserson/repos","events_url":"https://api.github.com/users/laserson/events{/privacy}","received_events_url":"https://api.github.com/users/laserson/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2015-10-29T23:30:02Z","updated_at":"2015-10-30T14:19:21Z","closed_at":null,"body":"And `SparkCommandLineProgram` should not set a default master.  This should be left to `spark-submit`.  Otherwise, the settings actually mask my requested spark master that I set with `spark-submit --master ...`."},"comment":{"url":"https://api.github.com/repos/broadinstitute/gatk/issues/comments/152538633","html_url":"https://github.com/broadinstitute/gatk/issues/1070#issuecomment-152538633","issue_url":"https://api.github.com/repos/broadinstitute/gatk/issues/1070","id":152538633,"user":{"login":"droazen","id":798637,"avatar_url":"https://avatars.githubusercontent.com/u/798637?v=3","gravatar_id":"","url":"https://api.github.com/users/droazen","html_url":"https://github.com/droazen","followers_url":"https://api.github.com/users/droazen/followers","following_url":"https://api.github.com/users/droazen/following{/other_user}","gists_url":"https://api.github.com/users/droazen/gists{/gist_id}","starred_url":"https://api.github.com/users/droazen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/droazen/subscriptions","organizations_url":"https://api.github.com/users/droazen/orgs","repos_url":"https://api.github.com/users/droazen/repos","events_url":"https://api.github.com/users/droazen/events{/privacy}","received_events_url":"https://api.github.com/users/droazen/received_events","type":"User","site_admin":false},"created_at":"2015-10-30T14:19:21Z","updated_at":"2015-10-30T14:19:21Z","body":"One of our goals for alpha (https://github.com/broadinstitute/gatk/issues/961) is actually to wrap `spark-submit` and its many options to make it easier to run hellbender tools on spark. We want users to be able to type a simple command like `./hellbender ToolName [toolArgs] --sparkMaster X`, and have hellbender figure out whether to invoke `spark-submit` or `gcloud dataproc` on their behalf, and provide sensible defaults for all relevant spark options. \r\n\r\nPerhaps there is a way in `SparkCommandLineProgram` to detect whether an option has already been set externally, and allow the default to be overridden if it has been?"}}