Similarity measurement between string element by element of vector in R
<p>I'm struggling with the performance of my approach for measuring similarity between strings.
What this approach does:</p>

<p>It's measuring similarity between every string elements with another string elements in <strong>sentenceMatch</strong> data frame. I'm using <strong>levenshteinSimFunction</strong> for the similarity measurement between strings  which is slightly corrected <strong>levenshteinDist</strong> function.</p>

<p>Below is my approach.</p>

<pre><code>sentenceMatch &lt;- data.frame(Sentence=c("job not ready time window pmg inc gvu austin timedout pmg inc plm",
                                   "data delay hpsb unable deliver icon scp action required http hpedia osp",
                                   "job completed abnormally wwapo bw kili inc promaster",
                                   "job completed abnormally apo ww promaster net apoww abend apo ww",
                                   "error occurred launching job apo ww inc promaster net errorlaunching apo"))

sentenceMatch$Sentence &lt;- as.character(sentenceMatch$Sentence)

overallMatrix &lt;- matrix(, nrow = dim(sentenceMatch)[1], ncol = dim(sentenceMatch)[1]) # creating output matrix

for (k in 1:dim(sentenceMatch)[1]) {
    for (l in 1:dim(sentenceMatch)[1]) {
      ifelse(k == l, overallMatrix[k, l] &lt;- 0, overallMatrix[k, l] &lt;- levenshteinSimFunction(sentenceMatch[k, ], sentenceMatch[l, ]))
      if (overallMatrix[k, l] &lt; .2) {overallMatrix[k, l] &lt;- 0}
    }
  }
</code></pre>

<p>This resulted into a matrix of these comparisons, where each element of the output matrix represents <strong>levenshteinSimFunction(sentenceMatch[k, ], sentenceMatch[l, ])</strong> that returns number (similarity measurement) between 0 and 1.</p>

<pre><code>levenshteinSimFunction = function (str1, str2) 
{
  if (str1 != "" &amp;&amp; str2 != "") {
  return(1 - (levenshteinDist(str1, str2)/max(nchar(str1), 
                                          nchar(str2))))
  }
  else {return (0)}
}

&gt; overallMatrix
          1         2         3         4         5
1 0.0000000 0.2394366 0.2615385 0.2307692 0.3055556
2 0.2394366 0.0000000 0.0000000 0.0000000 0.0000000
3 0.2615385 0.0000000 0.0000000 0.5156250 0.2916667
4 0.2307692 0.0000000 0.5156250 0.0000000 0.4444444
5 0.3055556 0.0000000 0.2916667 0.4444444 0.0000000
</code></pre>

<p>Everything works according to my expectation but I run into the performance issue because of two for loops.</p>

<p>Is there another (better), solution how to avoid these two for loops and speed up the performance. </p>

<p>The performance is driven by combination number n above 2 which exponentially increases processing time, so for e.g  1000 of sentences the computation time is 257.97 seconds. In my case I have 25k of sentences.</p>