Python multiprocessing - Why is using functools.partial slower than default arguments?
<p>Consider the following function:</p>

<pre><code>def f(x, dummy=list(range(10000000))):
    return x
</code></pre>

<p>If I use <code>multiprocessing.Pool.imap</code>, I get the following timings:</p>

<pre><code>import time
import os
from multiprocessing import Pool

def f(x, dummy=list(range(10000000))):
    return x

start = time.time()
pool = Pool(2)
for x in pool.imap(f, range(10)):
    print("parent process, x=%s, elapsed=%s" % (x, int(time.time() - start)))

parent process, x=0, elapsed=0
parent process, x=1, elapsed=0
parent process, x=2, elapsed=0
parent process, x=3, elapsed=0
parent process, x=4, elapsed=0
parent process, x=5, elapsed=0
parent process, x=6, elapsed=0
parent process, x=7, elapsed=0
parent process, x=8, elapsed=0
parent process, x=9, elapsed=0
</code></pre>

<p>Now if I use <code>functools.partial</code> instead of using a default value:</p>

<pre><code>import time
import os
from multiprocessing import Pool
from functools import partial

def f(x, dummy):
    return x

start = time.time()
g = partial(f, dummy=list(range(10000000)))
pool = Pool(2)
for x in pool.imap(g, range(10)):
    print("parent process, x=%s, elapsed=%s" % (x, int(time.time() - start)))

parent process, x=0, elapsed=1
parent process, x=1, elapsed=2
parent process, x=2, elapsed=5
parent process, x=3, elapsed=7
parent process, x=4, elapsed=8
parent process, x=5, elapsed=9
parent process, x=6, elapsed=10
parent process, x=7, elapsed=10
parent process, x=8, elapsed=11
parent process, x=9, elapsed=11
</code></pre>

<p>Why is the version using <code>functools.partial</code> so much slower?</p>