<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Working with large data sets
Lazy evaluation, pure functions and higher order functions
Lazy and eager evaluation
A list comprehension is eager.
Step1:  A generator expression is lazy.
Step2:  You can use generators as iterators.
Step3:  A generator is single use.
Step4:  The list constructor forces evaluation of the generator.
Step5:  An eager function.
Step6:  A lazy generator.
Step7:  Pure and impure functions
A pure function is like a mathematical function. Given the same inputs, it always returns the same output, and has no side effects.
Step8:  An impure function has side effects.
Step9:  Quiz
Say if the following functions are pure or impure.
Step10:  Higher order functions
Step11:  Using the operator module
The operator module provides all the Python operators as functions.
Step12:  Using itertools
Step13:  Generate all Boolean combinations
Step14:  Using toolz
Step15:  Using pipes and the curried namespace
Step16:  Processing many sets of DNA strings without reading into memory
<ASSISTANT_TASK:>
Python Code:
[x*x for x in range(3)]
Explanation: Working with large data sets
Lazy evaluation, pure functions and higher order functions
Lazy and eager evaluation
A list comprehension is eager.
End of explanation
(x*x for x in range(3))
Explanation: A generator expression is lazy.
End of explanation
g = (x*x for x in range(3))
next(g)
next(g)
next(g)
next(g)
Explanation: You can use generators as iterators.
End of explanation
for i in g:
    print(i, end=", ")
g = (x*x for x in range(3))
for i in g:
    print(i, end=", ")
Explanation: A generator is single use.
End of explanation
list(x*x for x in range(3))
Explanation: The list constructor forces evaluation of the generator.
End of explanation
def eager_updown(n):
    xs = []
    for i in range(n):
        xs.append(i)
    for i in range(n, -1, -1):
        xs.append(i)
    return xs
eager_updown(3)
Explanation: An eager function.
End of explanation
def lazy_updown(n):
    for i in range(n):
        yield i
    for i in range(n, -1, -1):
        yield i
lazy_updown(3)
list(lazy_updown(3))
Explanation: A lazy generator.
End of explanation
def pure(alist):
    return [x*x for x in alist]
Explanation: Pure and impure functions
A pure function is like a mathematical function. Given the same inputs, it always returns the same output, and has no side effects.
End of explanation
def impure(alist):
    for i in range(len(alist)):
        alist[i] = alist[i]*alist[i]
    return alist
xs = [1,2,3]
ys = pure(xs)
print(xs, ys)
ys = impure(xs)
print(xs, ys)
Explanation: An impure function has side effects.
End of explanation
def f1(n):
    return n//2 if n % 2==0 else n*3+1
def f2(n):
    return np.random.random(n)
def f3(n):
    n = 23
    return n
def f4(a, n=[]):
    n.append(a)
    return n
Explanation: Quiz
Say if the following functions are pure or impure.
End of explanation
list(map(f1, range(10)))
list(filter(lambda x: x % 2 == 0, range(10)))
from functools import reduce
reduce(lambda x, y: x + y, range(10), 0)
reduce(lambda x, y: x + y, [[1,2], [3,4], [5,6]], [])
Explanation: Higher order functions
End of explanation
import operator as op
reduce(op.mul, range(1, 6), 1)
list(map(op.itemgetter(1), [[1,2,3],[4,5,6],[7,8,9]]))
Explanation: Using the operator module
The operator module provides all the Python operators as functions.
End of explanation
import itertools as it
list(it.combinations(range(1,6), 3))
Explanation: Using itertools
End of explanation
list(it.product([0,1], repeat=3))
list(it.starmap(op.add, zip(range(5), range(5))))
list(it.takewhile(lambda x: x < 3, range(10)))
data = sorted('the quick brown fox jumps over the lazy dog'.split(), key=len)
for k, g in it.groupby(data, key=len):
    print(k, list(g))
Explanation: Generate all Boolean combinations
End of explanation
import toolz as tz
list(tz.partition(3, range(10)))
list(tz.partition(3, range(10), pad=None))
n = 30
dna = ''.join(np.random.choice(list('ACTG'), n))
dna
tz.frequencies(tz.sliding_window(2, dna))
Explanation: Using toolz
End of explanation
from toolz import curried as c
tz.pipe(
    dna,    
    c.sliding_window(2), # using curry
    c.frequencies,
)
composed = tz.compose(
    c.frequencies,
    c.sliding_window(2),  
)
composed(dna)
Explanation: Using pipes and the curried namespace
End of explanation
m = 10000
n = 300
dnas = (''.join(np.random.choice(list('ACTG'), n, p=[.1, .2, .3, .4])) 
        for i in range(m))
dnas
tz.merge_with(sum, 
              tz.map(
                  composed,
                  dnas
              )
             )
Explanation: Processing many sets of DNA strings without reading into memory
End of explanation
<END_TASK>