Creating a global connection object across an executor in spark when pushing data to mongodb
<p>In the <a href="http://spark.apache.org/docs/latest/streaming-programming-guide.html#design-patterns-for-using-foreachrdd" rel="nofollow">link</a> it was suggested to create a connection pool that is available across multiple rdds in the spark streaming job. </p>

<pre><code>rdd.foreachpartition( iter =&gt; {
val client = MongoClient(host,port)
val col = client.getDataBase("testDataBase").getCollection("testCollection")
</code></pre>

<p>// i am bascically inserting data in the iterator to the testcollection</p>

<p>})</p>

<p>However I was not able to figure out how to create a connection pool that returns a connection object to a mongodb collection. I was able to use foreachpartition to create a single connection for the whole partition. can someone please let me know how to create a connection object that available across the executor for reuse. </p>