What is the most efficient way to iterate a collection in parallel in Scala (TOP N pattern)
<p>I am new to Scala and would like to build a real-time application to match some people. For a given Person, I would like to get the TOP 50 people with the highest matching score.</p>

<p>The idiom is as follows :</p>

<pre><code>val persons = new mutable.HashSet[Person]() // Collection of people
/* Feed omitted */
val personsPar = persons.par // Make it parall
val person = ... // The given person

res = personsPar
        .filter(...) // Some filters
        .map{p =&gt; (p,computeMatchingScoreAsFloat(person, p))}
        .toList
        .sortBy(-_._2)
        .take(50)
        .map(t =&gt; t._1 + "=" + t._2).mkString("\n")
</code></pre>

<p>In the sample code above, HashSet is used, but it can be any type of collection, as I am pretty sure it is not optimal</p>

<p>The problem is that persons contains over 5M elements, the computeMatchingScoreAsFloat méthods computes a kind a correlation value with 2 vectors of 200 floats. This computation takes ~2s on my computer with 6 cores.</p>

<p>My question is, what is the fastest way of doing this TOPN pattern in Scala ?</p>

<p>Subquestions :
- What implementation of collection (or something else?) should I use ?
- Should I use Futures ?</p>

<p><strong>NOTE:</strong> It has to be computed in parallel, the pure computation of computeMatchingScoreAsFloat alone (with no ranking/TOP N) takes more than a second, and &lt; 200 ms if multi-threaded on my computer</p>

<p><strong>EDIT:</strong> Thanks to Guillaume, compute time has been decreased from 2s to 700ms</p>

<pre><code>def top[B](n:Int,t: Traversable[B])(implicit ord: Ordering[B]):collection.mutable.PriorityQueue[B] = {

  val starter = collection.mutable.PriorityQueue[B]()(ord.reverse) // Need to reverse for us to capture the lowest (of the max) or the greatest (of the min)

  t.foldLeft(starter)(
    (myQueue,a) =&gt; {
      if( myQueue.length &lt;= n ){ myQueue.enqueue(a);myQueue}
      else if( ord.compare(a,myQueue.head) &lt; 0  ) myQueue
      else{
        myQueue.dequeue
        myQueue.enqueue(a)
        myQueue
      }
    }
  )
}
</code></pre>

<p>Thanks</p>