How to manage a Apache Spark context in Django?
<p>I have a Django application that interacts with a Cassandra database and I want to try using Apache Spark to run operations on this database. I have some experience with Django and Cassandra but I'm new to Apache Spark.</p>

<p>I know that to interact with a Spark cluster first I need to create a SparkContext, something like this:</p>

<pre><code>from pyspark import SparkContext, SparkConf

conf = SparkConf().setAppName(appName).setMaster(master)
sc = SparkContext(conf=conf)
</code></pre>

<p>My question is the following: how should I treat this context? Should I instantiate it when my application starts and let it live during it's execution or should I start a SparkContext everytime before running an operation in the cluster and then kill it when the operation finishes? </p>

<p>Thank you in advance.</p>