speeding up urlib.urlretrieve
<p>I am downloading pictures from the internet, and as it turns out, I need to download lots of pictures. I am using a version of the following code fragment (actually looping through the links I intend to download and downloading the pictures :</p>

<pre><code>import urllib
urllib.urlretrieve(link, filename)
</code></pre>

<p>I am downloading roughly 1000 pictures every 15 minutes, which is awfully slow based on the number of pictures I need to download.</p>

<p>For efficiency, I set a timeout every 5 seconds (still many downloads last much longer):</p>

<pre><code>import socket
socket.setdefaulttimeout(5)
</code></pre>

<p>Besides running a job on a computer cluster to parallelize downloads, is there a way to make the picture download faster / more efficient?</p>