Calculate Spearman correlation Rho using cor between paired samples between two matrices
<p>I'm having some problems with trying to run cor.test, based on two OTU tables (essentially an abundance matrix).</p>

<p>I have a table with 8 different sampling spots (columns 1 to 44) and for each sampling spot I measured a variable (VARIABLE1, first row) and the presence of a series of species (OTUs on the rows). Here you can see an example of what these OTU tables look like below:</p>

<p>Samples 1 - 8 are the columns</p>

<pre><code>#OTUID  1   2   3   4   5   6   7   8
OTU1    0   0   0   0   0   3   0   0
OTU2    0   0   0   0   0   0   13  0
OTU3    5   99  0   0   0   0   0   0
OTU4    0   0   0   0   0   0   0   0
OTU5    0   0   0   0   0   0   0   2
OTU6    0   0   19  0   9   236 59  2
OTU7    0   55  0   2   4   2   3   0
OTU8    0   44  10  5   0   0   7   0
OTU9    6   0   13  2   0   0   17  6
OTU10   0   100 0   0   0   3   0   0
OTU11   4   13  0   0   2   1   2   0
OTU12   0   0   0   0   0   101 1   0
</code></pre>

<p>I have two of these tables in .biom format. I want to calculate spearman's correlation coefficients (R) between paired samples (e.g  sample1(OTU table 1) vs sample1(OTU table 2); sample2(OTU table 1) vs sample2(OTU table 2), etc....etc) for EACH OTU. I have approximately 100 samples and abundance data for 4000 OTUs. Hence, I want to calculate the correlation coefficient for 44 pairs of data for each of the 4000 OTUs. So basically just correlating an ~4k long vector once per sample. </p>

<p>Finally I will take the average of the correlations for the 4k long vector for each sample and plot these averages on a scatterplot. </p>

<p>Keep in mind all my abundance data is in this .biom format! I've converted it all to .txt to make it easier to manipulate.</p>

<p>Thanks again!!</p>

<p>An example of how this would look as <a href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4584126/figure/Fig2/" rel="nofollow noreferrer">below</a>: </p>

<p><a href="https://i.stack.imgur.com/XVs0A.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XVs0A.jpg" alt="enter image description here"></a></p>

<p>Here is some sample data:</p>

<p>OTU table 1:
<a href="https://1drv.ms/u/s!Ah_Q0DzXKdX-gZUl8SCetQdYE8S3CA" rel="nofollow noreferrer">https://1drv.ms/u/s!Ah_Q0DzXKdX-gZUl8SCetQdYE8S3CA</a></p>

<p>OTU table 2:
<a href="https://1drv.ms/u/s!Ah_Q0DzXKdX-gZUnaIwEIJiYMnrQIA" rel="nofollow noreferrer">https://1drv.ms/u/s!Ah_Q0DzXKdX-gZUnaIwEIJiYMnrQIA</a></p>