Accelerating one-to-many correlation calculations in Python
<p>I'd like to calculate the Pearson's correlation coefficient between a vector and each row of an array in Python (numpy and or scipy are assumed).  Use of standard correlation matrix calculation functions will not be possible due to the size of the real data arrays and memory constraints.  Here's my naive implementation: </p>

<pre><code>import numpy as np
import scipy.stats as sps

np.random.seed(0)

def correlateOneWithMany(one, many):
    """Return Pearson's correlation coef of 'one' with each row of 'many'."""
    pr_arr = np.zeros((many.shape[0], 2), dtype=np.float64)
    pr_arr[:] = np.nan
    for row_num in np.arange(many.shape[0]):
        pr_arr[row_num, :] = sps.pearsonr(one, many[row_num, :])
    return pr_arr

obs, varz = 10 ** 3, 500
X = np.random.uniform(size=(obs, varz))

pr = correlateOneWithMany(X[0, :], X)

%timeit correlateOneWithMany(X[0, :], X)
# 10 loops, best of 3: 38.9 ms per loop
</code></pre>

<p>Any thoughts on accelerating this would be greatly appreciated!</p>