Pass parameters from driver to executors in spark
<p>I am using spark 2.0.0.
Is there a way to pass parameters from spark driver to executors? I tried the following.</p>

<pre><code>class SparkDriver {
     public static void main(String argv[]){
           SparkConf conf = new SparkConf().setAppName("test").setMaster("yarn");
           SparkSession sparkSession = SparkSession.builder().config(conf).getOrCreate(); 
           Dataset&lt;Row&gt; input = sparkSession.read().load("inputfilepath");
           Dataset&lt;Row&gt; modifiedinput = input.mapPartitions(new customMapPartition(5),Encoders.bean(Row.class));
     }

  class customMapPartition implements MapPartitionsFunction{
          private static final long serialVersionUID = -6513655566985939627L;
          private static Integer variableThatHastobePassed = null;

        public customMapPartition(Integer passedInteger){
             customMapPartition.variableThatHastobePassed= passedInteger;
         }
         @Override
          public Iterator&lt;Row&gt; call(Iterator&lt;Row&gt; input) throws Exception {
              System.out.println("number that is passed " + variableThatHastobePassed);
          }
   }
</code></pre>

<p>As mentioned above I wrote a custom mappartitionfunction to pass the parameters. and am accessing the static variable in call method of partitionfunction. This worked when i ran in my local with "setmaster("local"). But did not work when ran on a cluster with .setmaster("yarn"). (printed null in the system.out.println statements) </p>

<p>Is there a way to pass parameters from driver to executors.</p>