Why does conf.set("spark.app.name", appName) not set the name in the UI?
<p>I am calling </p>

<pre class="lang-scala prettyprint-override"><code>val appName : String = arguments.getNameFromConfig

val conf = new SparkConf()
conf.set("spark.driver.maxResultSize", "30G")
conf.set("spark.app.name", appName)

println("Master: " + arguments.getMaster)

conf.setMaster(arguments.getMaster)    

val sc = new SparkContext(conf)
</code></pre>

<p>in order to identify my jobs in the UI more easily. However, it does not use this name in the scheduler. Instead it is using the path to the main class <code>Word2VecOnCluster</code>:</p>

<p><a href="https://i.stack.imgur.com/RmH6p.png" rel="noreferrer"><img src="https://i.stack.imgur.com/RmH6p.png" alt="enter image description here"></a></p>

<p>The name is only present in the title:</p>

<p><a href="https://i.stack.imgur.com/rXWQ6.png" rel="noreferrer"><img src="https://i.stack.imgur.com/rXWQ6.png" alt="enter image description here"></a></p>

<p>A colleague of mine is actually doing the same and there it works. What you cannot see here is that the name of my task is a little larger: </p>

<blockquote>
  <p>W2V_rtype-yelp_w2vpart-1_vsize-100_lr-0.025_dskeep-5.0perc_vocabsize-100000_epochs-1_iter-1</p>
</blockquote>

<p>So could it be that there is a limit regarding the length of the name? If so then it might should be added to the <a href="http://spark.apache.org/docs/latest/configuration.html#application-properties" rel="noreferrer">documentation</a> - or is there any other reason why it would do that?</p>