Caching in Spark
<p>A function is defined to transform an RDD. Therefore, the function is called once for each element in the RDD.</p>

<p>The function needs to call an external web service to look up reference data, passing as a parameter data from the current element in the RDD.</p>

<p>Two questions:</p>

<ol>
<li><p>Is there an issue with issuing a web service call within Spark?</p></li>
<li><p>The data from the web service needs to be cached. What is the best way to hold (and subsequently reference) the cached data? The simple way would be to hold the cache in a collection with the Scala class which contains the function being passed to the RDD. Would this be efficient, or is there a better approach for caching in Spark?</p></li>
</ol>

<p>Thanks</p>