<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Dimensionality Reduction
Problems
Step1:  Question
<ASSISTANT_TASK:>
Python Code:
from sklearn.feature_selection import VarianceThreshold
X = [[0, 2, 0, 3],
     [0, 1, 4, 3],
     [0, 1, 1, 3]]
selector = VarianceThreshold()
selector.fit_transform(X)
import pandas as pd
import seaborn as sns
%matplotlib inline
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
sns.heatmap(pd.DataFrame(X).corr(), cmap=sns.diverging_palette(220, 10, as_cmap=True));
from sklearn.datasets import load_iris
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import f_classif
X, y = load_iris(return_X_y=True)
print('Before feature selection:', X.shape)
selector = SelectKBest(f_classif, k=2).fit(X, y)
X_new = selector.transform(X)
print('After feature selection:', X_new.shape)
print('Scores:', selector.scores_)
Explanation: Dimensionality Reduction
Problems:
* Memory
* Speed
* Accuracy
* Curse of dimensionality (Clustering)
Feature Selection
Variance Threshold
ANOVA
Recursive Feature Elimination
Using Models
$$ y = x_1 + sin(6 \cdot pi \cdot x_2) + 0.1 \cdot \mathcal{N}(0, 1) $$
End of explanation
import sklearn 
import numpy as np 
import pandas as pd 
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
import gzip, pickle, sys 
f = gzip.open('Datasets/mnist.pkl.gz', 'rb')
(input_train, output_train), (input_test, output_test), _ = pickle.load(f, encoding='bytes')
for i in range(4):
    plt.subplot(2,2,i+1)
    plt.imshow(input_train[i].reshape((28,28)), cmap=plt.cm.gray_r, interpolation='nearest')
plt.show()
from sklearn.ensemble import RandomForestClassifier
randomforest = RandomForestClassifier(n_estimators=30)
randomforest.fit(input_train,output_train)
from sklearn.metrics import classification_report
print(classification_report(output_test, randomforest.predict(input_test)))
from sklearn.decomposition import PCA
pca = PCA(n_components=500)
pca.fit(input_train)
plt.figure(figsize=(12,6))
plt.plot(np.cumsum(pca.explained_variance_ratio_[0:500]),marker = 'o')
plt.show()
np.cumsum(pca.explained_variance_ratio_[0:500])[200]
pca = PCA(n_components= 200)
pca.fit(input_train)
x_train = pca.transform(input_train)
x_test = pca.transform(input_test)
for i in range(4):
    plt.subplot(2,2,i+1)
    plt.imshow(pca.inverse_transform(x_train)[i].reshape((28, 28)), cmap=plt.cm.gray_r, interpolation='nearest' )
plt.show()
randomforest = RandomForestClassifier(n_estimators=30)
randomforest.fit(x_train,output_train)
from sklearn.metrics import classification_report
print(classification_report(output_test, randomforest.predict(x_test)))
Explanation: Question: Consider there is a feature with a score far better than other features. Is this feature a good choice?
Unsupervised Learning: Principal Component Analysis (PCA)
Principal component analysis is a dimensionality reduction algorithm that we can use to find structure in our data.The main aim is to find surface onto which projection errros are minimized.This surface is a lower the dimensional subspace spanned by principal components of the data. These principal components are the direction along which the projection of the data onto that axis hae the maximum variance.The main component along which the data varies is called the principal axis of variation.
Intuitive interpretation of PCA
At the first step we plot a data an with 2 dimenstion and try to reduce it to one 
 Don't get confused 
PCA is different from linear regression
How about If we have three-dimensional
Using PCA in Sklearn
End of explanation
<END_TASK>