<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 SQL-type operations
If you know something about relational databases and SQL, you may have heard of JOIN and GROUP BY.
Step1:  Joining
Step2:  We can concatenate the two DataFrames.
Step3:  Alternatively, the above can be obtained with the self-describing append() method.
Step4:  By default, concat() concatenates along the rows (axis 0). What we did previously was 'concatenating' along the columns (axis 1). It is actually a join operation, on (index or key) Date.
Step5:  Indeed, we could alternatively use the join() method.
Step6:  Wait! Which frame's index is used? JOIN can be left, right, outer, or inner (picture unions and intersections).
Step7:  Hands-on exercise
How else, still using the join() method, could we obtain Average and Average_gl for dates between 1980-01 and 1980-05?
Grouping
We introduce the split-apply-combine approach
Step8:  The function in question is an aggregation (for each group, return the minimum value). The length of the result is the number of different groups (here, number of unique x values). Aggregation reduces the size of the data structure.
Step9:  The aggregation function can be applied directly, if it is available.
Step10:  With a hierarchical index, the level parameter can even be passed directly to certain aggregation functions.
Step11:  Counting the number of records in each group is also an aggregation.
Step12:  For each unique values of x (which are 0 and 1), we have two entries.
The grouping object (here, x) is referred to as the key.
Step13:  Filtering is another kind of operation which can be applied to each group. Filtering may reduce the size of the data structure, since some groups might get filtered out.
Step14:  The third kind of operation is transformation, where the size of the data structure is preserved.
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
Explanation: SQL-type operations
If you know something about relational databases and SQL, you may have heard of JOIN and GROUP BY.
End of explanation
mlo, gl = pd.read_csv('../data/co2-mm-mlo.csv', na_values=-99.99, index_col='Date', parse_dates=True), \
    pd.read_csv('../data/co2-mm-gl.csv', na_values=-99.99, index_col='Date', parse_dates=True)
    # pd.read_csv('https://python.g-node.org/wiki/_media/co2-mm-mlo.csv')
ml, gl = mlo[['Average']], gl[['Average']]
ml.columns, gl.columns = ['Average_mlo'], ['Average_gl']
ml = ml[ml.index >= '1980-01']
ml, gl = ml.head(), gl.head()
Explanation: Joining
End of explanation
pd.concat([ml, gl])
Explanation: We can concatenate the two DataFrames.
End of explanation
ml.append(gl)
Explanation: Alternatively, the above can be obtained with the self-describing append() method.
End of explanation
pd.concat([ml, gl], axis=1)
Explanation: By default, concat() concatenates along the rows (axis 0). What we did previously was 'concatenating' along the columns (axis 1). It is actually a join operation, on (index or key) Date.
End of explanation
ml.join(gl)
Explanation: Indeed, we could alternatively use the join() method.
End of explanation
mlo = pd.read_csv('../data/co2-mm-mlo.csv', na_values=-99.99, index_col='Date', parse_dates=True)
mlo = mlo[['Average']]
mlo.head()
mlo.join(gl)
mlo.join(gl, how='inner')
Explanation: Wait! Which frame's index is used? JOIN can be left, right, outer, or inner (picture unions and intersections).
End of explanation
z = pd.Series([0.5, 0.8, 0.6, 0.3], index=pd.MultiIndex.from_product([[0, 1], [0, 1]], names=['x', 'y']))
z
z.groupby('x')
z.groupby('x').apply(lambda u: u.min())
Explanation: Hands-on exercise
How else, still using the join() method, could we obtain Average and Average_gl for dates between 1980-01 and 1980-05?
Grouping
We introduce the split-apply-combine approach:
split the data into groups;
apply a function to each group independently;
combine the results into an appropriate data structure.
End of explanation
z.groupby(level=0).apply(lambda u: u.min())
Explanation: The function in question is an aggregation (for each group, return the minimum value). The length of the result is the number of different groups (here, number of unique x values). Aggregation reduces the size of the data structure.
End of explanation
z.groupby('x').min()
Explanation: The aggregation function can be applied directly, if it is available.
End of explanation
z.min(level='x')
Explanation: With a hierarchical index, the level parameter can even be passed directly to certain aggregation functions.
End of explanation
z.groupby('x').size()
Explanation: Counting the number of records in each group is also an aggregation.
End of explanation
z.groupby(['x', 'y']).size()
z.groupby('y').describe()
import numpy as np
z.groupby('y').apply(lambda u: np.std(u, ddof=1))
z.groupby('y').apply(lambda u: np.std(u))
z.groupby('y').apply(np.std)
Explanation: For each unique values of x (which are 0 and 1), we have two entries.
The grouping object (here, x) is referred to as the key.
End of explanation
z
z.groupby('y').apply(lambda u: u.min() > 0.4)
z.groupby('y').filter(lambda u: u.min() > 0.4)
Explanation: Filtering is another kind of operation which can be applied to each group. Filtering may reduce the size of the data structure, since some groups might get filtered out.
End of explanation
z.groupby('y').transform(lambda u: u.min())
Explanation: The third kind of operation is transformation, where the size of the data structure is preserved.
End of explanation
<END_TASK>