How to design an async pipeline pattern in python
<p>I am trying to design an async pipeline that can easily make a data processing pipeline. The pipeline is composed of several functions. Input data goes in at one end of the pipeline and comes out at the other end. </p>

<p>I want to design the pipeline in a way that: </p>

<ol>
<li>Additional functions can be insert in the pipeline</li>
<li>Functions already in the pipeline can be popped out.  </li>
</ol>

<p>Here is what I came up with:</p>

<pre><code>import asyncio

@asyncio.coroutine
def add(x):
    return x + 1

@asyncio.coroutine
def prod(x):
    return x * 2

@asyncio.coroutine
def power(x):
    return x ** 3

def connect(funcs):
    def wrapper(*args, **kwargs):
        data_out = yield from funcs[0](*args, **kwargs)
        for func in funcs[1:]:
            data_out = yield from func(data_out)
        return data_out
    return wrapper

pipeline = connect([add, prod, power])
input = 1
output = asyncio.get_event_loop().run_until_complete(pipeline(input))
print(output)
</code></pre>

<p>This works, of course, but the problem is that if I want to add another function into (or pop out a function from) this pipeline, I have to disassemble and reconnect every function again.</p>

<p>I would like to know if there is a better scheme or design pattern to create such  a pipeline?</p>