Using the same JavaSparkContext for multiple jobs to prevent using time on spark driver initialization
<p>I am trying to run Spark jobs on Google Dataproc. However, as its seen typically, the initialization of the Spark Driver takes up a good chunk of the execution time. I would like to know what is a good way to run multiple jobs of Spark on Google Dataproc using the same instance of JavaSparkContext so I don't lose performace every job due to spark driver initialization. Currently, my code looks something like this: </p>

<pre><code>public static void main(String[] args) {

    SparkConf configuration = new SparkConf().setAppName("App");
    final JavaSparkContext context = new JavaSparkContext(configuration);

    // Do stuff

    // Stop connection to Java Spark
    context.stop();
}
</code></pre>