<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
Step1:  1 Least squares and linear basis functions models
1.1 Least squares
Step2:  Load the data
Here we will reuse the dataset height_weight_genders.csv from previous exercise section to check the correctness of your implementation. Please compare it with your previous result.
Step3:  Test it here
Step5:  1.2 Least squares with a linear basis function model
Start from this section, we will use the dataset dataEx3.csv.
Implement polynomial basis functions
Step7:  Let us play with polynomial regression. Note that we will use your implemented function compute_mse. Please copy and paste your implementation from exercise02.
Step8:  Run polynomial regression
Step10:  2 Evaluating model predication performance
Let us show the train and test splits for various polynomial degrees. First of all, please fill in the function split_data()
Step12:  Then, test your split_data function below.
Step13:  Demo time
Step16:  Ridge Regression
Please fill in the function below.
Step17:  Demo time
<ASSISTANT_TASK:>
Python Code:
def least_squares(y, tx):
    calculate the least squares solution.
    a = tx.T.dot(tx)
    b = tx.T.dot(y)
    return np.linalg.solve(a, b)
Explanation: 1 Least squares and linear basis functions models
1.1 Least squares
End of explanation
from helpers import *
def test_your_least_squares():
    height, weight, gender = load_data_from_ex02(sub_sample=False, add_outlier=False)
    x, mean_x, std_x = standardize(height)
    y, tx = build_model_data(x, weight)
    weight = least_squares(y, tx)
    print(weight)
Explanation: Load the data
Here we will reuse the dataset height_weight_genders.csv from previous exercise section to check the correctness of your implementation. Please compare it with your previous result.
End of explanation
test_your_least_squares()
Explanation: Test it here
End of explanation
# load dataset
x, y = load_data()
print("shape of x {}".format(x.shape))
print("shape of y {}".format(y.shape))
def build_poly(x, degree):
    polynomial basis functions for input data x, for j=0 up to j=degree.
    poly = np.ones((len(x), 1))
    for deg in range(1, degree+1):
        poly = np.c_[poly, np.power(x, deg)]
    return poly
Explanation: 1.2 Least squares with a linear basis function model
Start from this section, we will use the dataset dataEx3.csv.
Implement polynomial basis functions
End of explanation
from costs import compute_mse
from plots import *
def polynomial_regression():
    Constructing the polynomial basis function expansion of the data,
       and then running least squares regression.
    # define parameters
    degrees = [1, 3, 7, 12]
    
    # define the structure of the figure
    num_row = 2
    num_col = 2
    f, axs = plt.subplots(num_row, num_col)
    for ind, degree in enumerate(degrees):
        # form dataset to do polynomial regression.
        tx = build_poly(x, degree)
        # least squares
        weights = least_squares(y, tx)
        # compute RMSE
        rmse = np.sqrt(2 * compute_mse(y, tx, weights))
        print("Processing {i}th experiment, degree={d}, rmse={loss}".format(
              i=ind + 1, d=degree, loss=rmse))
        # plot fit
        plot_fitted_curve(
            y, x, weights, degree, axs[ind // num_col][ind % num_col])
    plt.tight_layout()
    plt.savefig("visualize_polynomial_regression")
    plt.show()
Explanation: Let us play with polynomial regression. Note that we will use your implemented function compute_mse. Please copy and paste your implementation from exercise02.
End of explanation
polynomial_regression()
Explanation: Run polynomial regression
End of explanation
def split_data(x, y, ratio, seed=1):
    split the dataset based on the split ratio.
    # set seed
    np.random.seed(seed)
    # generate random indices
    num_row = len(y)
    indices = np.random.permutation(num_row)
    index_split = int(np.floor(ratio * num_row))
    index_tr = indices[: index_split]
    index_te = indices[index_split:]
    # create split
    x_tr = x[index_tr]
    x_te = x[index_te]
    y_tr = y[index_tr]
    y_te = y[index_te]
    return x_tr, x_te, y_tr, y_te
Explanation: 2 Evaluating model predication performance
Let us show the train and test splits for various polynomial degrees. First of all, please fill in the function split_data()
End of explanation
def train_test_split_demo(x, y, degree, ratio, seed):
    polynomial regression with different split ratios and different degrees.
    x_tr, x_te, y_tr, y_te = split_data(x, y, ratio, seed)
    # form tx
    tx_tr = build_poly(x_tr, degree)
    tx_te = build_poly(x_te, degree)
    weight = least_squares(y_tr, tx_tr)
    # calculate RMSE for train and test data.
    rmse_tr = np.sqrt(2 * compute_mse(y_tr, tx_tr, weight))
    rmse_te = np.sqrt(2 * compute_mse(y_te, tx_te, weight))
    print("proportion={p}, degree={d}, Training RMSE={tr:.3f}, Testing RMSE={te:.3f}".format(
          p=ratio, d=degree, tr=rmse_tr, te=rmse_te))
Explanation: Then, test your split_data function below.
End of explanation
seed = 6
degrees = [1, 3, 7, 12]
split_ratios = [0.9, 0.5, 0.1]
for split_ratio in split_ratios:
    for degree in degrees:
        train_test_split_demo(x, y, degree, split_ratio, seed)
Explanation: Demo time
End of explanation
def ridge_regression(y, tx, lambda_):
    implement ridge regression.
    aI = 2 * tx.shape[0] * lambda_ * np.identity(tx.shape[1])
    a = tx.T.dot(tx) + aI
    b = tx.T.dot(y)
    return np.linalg.solve(a, b)
def ridge_regression_demo(x, y, degree, ratio, seed):
    ridge regression demo.
    # define parameter
    lambdas = np.logspace(-5, 0, 15)
    # split data
    x_tr, x_te, y_tr, y_te = split_data(x, y, ratio, seed)
    # form tx
    tx_tr = build_poly(x_tr, degree)
    tx_te = build_poly(x_te, degree)
    # ridge regression with different lambda
    rmse_tr = []
    rmse_te = []
    for ind, lambda_ in enumerate(lambdas):
        # ridge regression
        weight = ridge_regression(y_tr, tx_tr, lambda_)
        rmse_tr.append(np.sqrt(2 * compute_mse(y_tr, tx_tr, weight)))
        rmse_te.append(np.sqrt(2 * compute_mse(y_te, tx_te, weight)))
        print("proportion={p}, degree={d}, lambda={l:.3f}, Training RMSE={tr:.3f}, Testing RMSE={te:.3f}".format(
               p=ratio, d=degree, l=lambda_, tr=rmse_tr[ind], te=rmse_te[ind]))
    plot_train_test(rmse_tr, rmse_te, lambdas, degree)
Explanation: Ridge Regression
Please fill in the function below.
End of explanation
seed = 56
degree = 7
split_ratio = 0.5
ridge_regression_demo(x, y, degree, split_ratio, seed)
Explanation: Demo time
End of explanation
<END_TASK>