Spark : DB connection per Spark RDD partition and do mapPartition
<p>I want to do a mapPartitions on my spark rdd, </p>

<pre><code>    val newRd = myRdd.mapPartitions(
      partition =&gt; {

        val connection = new DbConnection /*creates a db connection per partition*/

        val newPartition = partition.map(
           record =&gt; {
             readMatchingFromDB(record, connection)
         })
        connection.close()
        newPartition
      })
</code></pre>

<p>But, this gives me a connection already closed exception, as expected because before the control reaches the <code>.map()</code> my <code>connection</code> is closed. I want to create a connection per RDD partition, and close it properly. How can I achieve this?</p>

<p>Thanks!</p>