How does Celery work?
<p>I have recently started working on distributed computing for increasing the computation speed. I opted for <a href="http://www.celeryproject.org/" rel="noreferrer">Celery</a>. However, I am not very familiar with some terms. So, I have several related questions.</p>
<p>From the <a href="http://docs.celeryproject.org/en/latest/getting-started/introduction.html" rel="noreferrer">Celery docs</a>:</p>
<blockquote>
<h3>What's a Task Queue?</h3>
<p>...</p>
<p>Celery communicates via messages, usually using a broker to mediate between clients and workers. To initiate a task the client adds a message to the queue, the broker then delivers that message to a worker.</p>
</blockquote>
<p><em>What are clients (here)? What is a broker? Why are messages delivered through a broker? Why would Celery use a backend and queues for interprocess communication?</em></p>
<p>When I execute the Celery console by issuing the command</p>
<pre><code>celery worker -A tasks --loglevel=info --concurrency 5
</code></pre>
<p><em>Does this mean that the Celery console is a worker process which is in charge of 5 different processes and keeps track of the task queue? When a new task is pushed into the task queue, does this worker assign the task/job to any of the 5 processes?</em></p>