Is it possible to run spark yarn cluster from the code?
<p>I have a MapReduce task which I want to run on Spark YARN cluster from my java code. Also I want to retrieve reduce result (string and number pair, tuple) in my java code. Something like:</p>

<pre><code>// I know, it's wrong setMaster("YARN"), but just to describe what I want.
// I want to execute job ob the cluster.
SparkConf sparkConf = new SparkConf().setAppName("Test").setMaster("YARN");
JavaSparkContext sc = new JavaSparkContext(sparkConf);

JavaRDD&lt;Integer&gt; input = sc.parallelize(list);

// map
JavaPairRDD&lt;String, Integer&gt; results = input.mapToPair(new MapToPairExample());

// reduce
String max = results.max(new MyResultsComparator())._1();
</code></pre>

<p>It works if I set master to local, local[] or spark://master:7707.</p>

<p>So the question is: can I do the same with yarn cluster somehow?</p>