Numpy - automatic implicit conversion of np.arrays into lists
<p>I am trying to make a <code>numpy.array</code> of other <code>numpy.array's</code>, for a Monte Carlo simulation. I would like to take an element of the array (another 1d array) and perform some function on it which returns a subsection of that array, and re-assign it. <strong>Originally all arrays are the same length</strong> which seems to cause the problem.</p>

<p><strong>I want to do</strong>: e.g.</p>

<pre><code>c1 = np.array([np.array(xi) for xi in [[1,2],[1],[1,3]]])
c1[2] = np.array([5])
</code></pre>

<p>which works fine giving:</p>

<pre><code>array([array([1, 2]), array([1]), array([5])], dtype=object)
</code></pre>

<p>and I get the same sort of structure when I do a transform which leaves all the sub-arrays the same length.</p>

<pre><code>c2 = np.array([np.array(xi) for xi in [[1,2],[1],[1,3]]])
c2[1] = np.array([4,5])
</code></pre>

<p>giving</p>

<pre><code>array([array([1, 2]), array([4, 5]), array([1, 3])], dtype=object).
</code></pre>

<p><strong>Instead what happens is</strong>: e.g.</p>

<p>When I start all the arrays being the same length they are converted from <code>np.arrays</code> into <code>lists</code>.</p>

<pre><code>c3 = np.array([np.array(xi) for xi in [[1,2],[1,2],[1,4]]])
</code></pre>

<p>which gives</p>

<pre><code>array([[1, 2], [1, 2], [1, 4]]).
</code></pre>

<p>Now if I try to adjust the length of one of the lists</p>

<pre><code>c3[1] = np.array([5])
</code></pre>

<p>I get broadcasting</p>

<pre><code>array([[1, 2], [5, 5], [1, 4]])
</code></pre>

<p><em>when instead I wanted</em></p>

<pre><code>array([array([1, 2]), array([5]), array([1, 4])])
</code></pre>

<p><strong>Performance Issues</strong>:</p>

<p>This is all for a Monte Carlo simulation, and typically I am looking to handle an array of about 10'000 sub-arrays, where each sub-array is an array of around 1'000 elements (floats). I have been using numpy because its random number generation is very quick, and it has great performance for vectorised functions, where specifically I want to perform some sort of cumulative sum on these eventually (<code>np.cumsum</code>), and some other more esoteric functions. I was able to do this sort of thing with lists, but it took around 20 times longer, and am looking to optimise whenever possible.</p>

<p>Many Thank.</p>