threading.Lock() performance issues
<p>I have multiple threads:</p>

<pre><code>dispQ = Queue.Queue()
stop_thr_event = threading.Event()

def worker (stop_event):
    while not stop_event.wait(0):
        try:
            job = dispQ.get(timeout=1)
            job.waitcount -= 1
            dispQ.task_done()
        except Queue.Empty, msg:
            continue

# create job objects and put into dispQ here
for j in range(NUM_OF_JOBS):
    j = Job()
    dispQ.put(j)

# NUM_OF_THREADS could be 10-20 ish
running_threads = []
for t in range(NUM_OF_THREADS):
    t1 = threading.Thread( target=worker, args=(stop_thr_event,) )
    t1.daemon = True
    t1.start()
    running_threads.append(t1)


stop_thr_event.set()
for t in running_threads:
    t.join()
</code></pre>

<p>The code above was giving me some very strange behavior.
I've ended up finding out that it was due to decrementing waitcount with out a lock</p>

<p>I 've added an attribute to Job class <code>self.thr_lock = threading.Lock()</code>
Then I've changed it to</p>

<pre><code>with job.thr_lock:
    job.waitcount -= 1
</code></pre>

<p>This seems to fix the strange behavior but it looks like it has degraded in performance.</p>

<p>Is this expected? is there way to optimize locking?<br>
Would it be better to have one global lock rather than one lock per job object?</p>