<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Machine Intelligence II (week 3) - Team MensaNord
Nikolai Zaki
Alexander Moore
Johannes Rieke
Georg Hoelger
Oliver Atanaszov
Step1:  Exercise 1
A
Step2:  B
Step3:  By deleting 2 of the 500 datapoints, the projection on the PCs change about 10 degrees. Hence the PCs itself change significantly.
Exercise 2
A
Step4:  The data points at indices 99, 111, 199 and 211 (marked in red in the plot above) are apparently outliers and will be discarded.
Step5:  B
Step6:  The first two PCs seem sufficient to represent the data. The third and fourth PCs are negligible.
Step7:  C
Step8:  D
<ASSISTANT_TASK:>
Python Code:
from __future__ import division, print_function
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
Explanation: Machine Intelligence II (week 3) - Team MensaNord
Nikolai Zaki
Alexander Moore
Johannes Rieke
Georg Hoelger
Oliver Atanaszov
End of explanation
data1 = pd.read_csv('pca2.csv', sep=',')
data1.ix[0:2, 'X1':'X2']
data1.ix[0:2, :]
data1.shape
m = np.mean(data1, 0)
data1_centered = data1 - m
data1_centered.ix[0:2, :]
covariance1 = np.cov(data1_centered.T)
evals1, evecs1 = np.linalg.eig(covariance1)
transmat1 = evecs1.T
evec1 = transmat1[0]
evec2 = transmat1[1]
data1_trans = np.array([[0.0, 0.0] for i in range(len(data1))])
for i in range(len(data1)):
    data1_trans[i] = np.dot(transmat1, data1_centered.ix[i, :])
data1_trans[0:3, :]
# plt.figure(figsize=(10, 10))
fig, ax = plt.subplots(figsize=(10, 10))
plt.scatter(data1_centered.ix[:, 'X1'], data1_centered.ix[:, 'X2'])
ax.set_color_cycle(['black', 'red'])
plt.plot([0, evec1[0]], [0, evec1[1]])
plt.plot([0, evec2[0]], [0, evec2[1]])
plt.grid()
plt.show()
plt.figure(figsize=(10, 10))
plt.scatter(data1_trans.T[0], data1_trans.T[1])
plt.grid()
plt.show()
Explanation: Exercise 1
A
End of explanation
data1b_c = np.copy(data1_centered)
data1b_c[16, :] = 0
data1b_c[156, :] = 0
cov1b = np.cov(data1b_c.T)
evals1b, evecs1b = np.linalg.eig(cov1b)
transmat1b = evecs1b.T
evec1b = transmat1b[0]
evec2b = transmat1b[1]
data1b_t = np.array([[0.0, 0.0] for i in range(len(data1))])
for i in range(len(data1)):
    data1b_t[i] = np.dot(transmat1b, data1b_c[i, :])
plt.figure(figsize=(10, 10))
plt.scatter(data1_trans.T[0], data1_trans.T[1])
plt.scatter(data1b_t.T[0], data1b_t.T[1])
plt.grid()
plt.show()
Explanation: B
End of explanation
data = np.loadtxt('pca4.csv', skiprows=1, delimiter=',')
data.shape, data[:10]
fig, axes = plt.subplots(2, 2, figsize=(10, 10))
outliers = [99, 199, 111, 211]
for i, ax in enumerate(axes.flatten()):
    plt.sca(ax)
    plt.scatter(range(len(data)), data[:, i], c=['r' if a in outliers else 'b' for a in range(len(data))])
    plt.ylabel('x{}'.format(i+1))
    plt.xlabel('Index')
    plt.ylim(-14, 14)
Explanation: By deleting 2 of the 500 datapoints, the projection on the PCs change about 10 degrees. Hence the PCs itself change significantly.
Exercise 2
A
End of explanation
filtered_data = np.delete(data, outliers, axis=0)
filtered_data.shape
Explanation: The data points at indices 99, 111, 199 and 211 (marked in red in the plot above) are apparently outliers and will be discarded.
End of explanation
centered_data = filtered_data - filtered_data.mean(axis=0)
centered_data[:10]
cov = np.cov(centered_data.T)
cov.shape
evals, evecs = np.linalg.eig(cov)
plt.plot(evals, 'o')
plt.ylabel('Eigenvalue')
plt.xlabel('Component')
plt.ylim(0)
evals
Explanation: B
End of explanation
projected_data = np.dot(evecs.T, centered_data.T).T
fig, axes = plt.subplots(2, 2, figsize=(10, 10))
for i, ax in enumerate(axes.flatten()):
    plt.sca(ax)
    plt.plot(projected_data[:, i], '.')
    plt.ylabel('u{}'.format(i+1))
    plt.xlabel('Index')
    plt.ylim(-6, 6)
Explanation: The first two PCs seem sufficient to represent the data. The third and fourth PCs are negligible.
End of explanation
whitened_data = evecs.T.dot(centered_data.T).T.dot(np.diag(1 / np.sqrt(evals)))
fig, axes = plt.subplots(2, 2, figsize=(10, 10))
for i, ax in enumerate(axes.flatten()):
    plt.sca(ax)
    plt.plot(whitened_data[:, i], '.')
    plt.ylabel('w{}'.format(i+1))
    plt.xlabel('Index')
    plt.ylim(-6, 6)
Explanation: C
End of explanation
plt.imshow(cov, cmap='Blues')
plt.colorbar(label='Covariance of original data')
plt.xlabel('Dimension')
plt.ylabel('Dimension')
projected_cov = np.cov(projected_data.T)
plt.imshow(projected_cov, cmap='Blues')
plt.colorbar(label='Covariance of projected data')
plt.xlabel('Dimension')
plt.ylabel('Dimension')
whitened_cov = np.cov(whitened_data.T)
plt.imshow(whitened_cov, cmap='Blues')
plt.colorbar(label='Covariance of whitened data')
plt.xlabel('Dimension')
plt.ylabel('Dimension')
Explanation: D
End of explanation
<END_TASK>