<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 k-Nearest Neighbours (kNN)
kNN is a supervised method that can be used to solve classification problems and regression problems.
Given
a training set $\mathcal{T} = \left{(x_1, y_1), (x_2, y_2), \cdots, (x_n, y_n) \right}$
a positive integer $k$ 
a test observation $x'$
Let $\mathcal{N}$ denote the $k$ data points in the training set $\mathcal{T}$ that are closest to $x'$.
In kNN classification, the method selects the most frequent $y_i \in \mathcal{N}$ where $i \in {1,2, \cdots, k}$.
In kNN regression, the method estimates $\hat{y}$ for $x'$ by computing the mean of the $k$ points in the training set that are closest to $x'$
Step1:  Check if there are missing data.
Step2:  Normalise the Data
The features V1, V2, ... and V28 are normalised because of the PCA transformation. However, the Amount feature is not. Since the kNN method relies distances between the observation, we need to rescale this feature.
Step3:  We can't use the Time feature, so let us get rid of it. We don't need the Amount feature either.
Step4:  Re-sample the Data
Now let us check the size of the dataset.
Step5:  The data set contains 284,807 examples. As kNN does not perform well with large data sets, we must find a way to reduce of size. We also need to ensure that the data is balanced before we run our kNN classifier.
Step6:  Just by looking at the numbers above, we can see that the data is not balanced. Let us print out the proportion of data that is in class 0.
Step7:  The data set is highly unbalanced. Only a very small proportion (0.17%) of the transactions are fraudulent while the majority of the examples are normal. In the real world, this makes sense. However, training a classifier with unbalanced data results in predictions that are biased towards predicting the majority class. Therefore, we must re-sample the data set so both classes have approximately 50-50 ratio. We can either
Step8:  Now the ratio should be around 50 percent.
Step9:  Partition the Data
Now we are ready to partition the data into training set and test set.
Step10:  Pick an Appropriate $k$ Value
An appropriate $k$ value for the kNN algorithm is the one that yields the lowest error or the highest score. We can use 10-Fold Cross Validation to estimate the score associated with each $k$.
Step11:  It is clear that the score peaks at $k=5$.
Evaluate Predictions
It is time to evaluate kNN where $k=5$ using a classification report.
Step12:  These results for the under-sampled data are quite good. Here is the confusion matrix
Step13:  Now let us predict on a subset of the whole data.
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
df = pd.read_csv('data/creditcard.csv')
df.head()
Explanation: k-Nearest Neighbours (kNN)
kNN is a supervised method that can be used to solve classification problems and regression problems.
Given
a training set $\mathcal{T} = \left{(x_1, y_1), (x_2, y_2), \cdots, (x_n, y_n) \right}$
a positive integer $k$ 
a test observation $x'$
Let $\mathcal{N}$ denote the $k$ data points in the training set $\mathcal{T}$ that are closest to $x'$.
In kNN classification, the method selects the most frequent $y_i \in \mathcal{N}$ where $i \in {1,2, \cdots, k}$.
In kNN regression, the method estimates $\hat{y}$ for $x'$ by computing the mean of the $k$ points in the training set that are closest to $x'$:
$$
\hat{y} = \frac{1}{k} \sum_{x_i \in \mathcal{N} } y_i
$$
Performance of the kNN algorithm depends on several factors including:
a suitable distance measure: we can use the Euclidean distance for numeric features but it has issues in higher dimensions. There exist many <a href="https://www.rdocumentation.org/packages/fingerprint/versions/3.5.4/topics/distance-methods">distance metrics</a>.
an appropriate $k$ value: Choosing an appropriate $k$ can be difficult and depends on the <a href="http://scott.fortmann-roe.com/docs/BiasVariance.html">bias-variance tradeoff</a>. A small $k$ value has low bias but high variance whereas a large $k$ value provide a smoother and less variable fit. There are some rule-of-thumb for picking the $k$ value. The value of $k$ must not be a multiple of the number of classes to avoid ties. For instance, the value of $k$ must be odd if problems with only two classes.
Unlike other methods that build a model of the training (labelled) data, kNN has no explicit knowledge representation of a model. This means that predicting using kNN is associated with a high computational cost especially for large data sets. In addition, kNN performs poorly for high dimensional data because "the ratio of the distances of the nearest and farthest neighbors to a given target in high dimensional space is almost 1 for a wide variety of data distributions and distance functions" [<a href="https://bib.dbvis.de/uploadedFiles/155.pdf">#<a>].
Explore Credit Card Fraud Detection Data Set
Next, we explore the <a href="https://www.kaggle.com/dalpozz/creditcardfraud">Credit Card Fraud Detection Dataset</a> on Kaggle.
End of explanation
df.isnull().any()
Explanation: Check if there are missing data.
End of explanation
from sklearn.preprocessing import StandardScaler
ft_amount = df['Amount'].values.reshape(-1, 1)
scaler = StandardScaler()
scaler.fit(ft_amount)
df['AmountNormalised'] = scaler.transform(ft_amount)
df.head()
Explanation: Normalise the Data
The features V1, V2, ... and V28 are normalised because of the PCA transformation. However, the Amount feature is not. Since the kNN method relies distances between the observation, we need to rescale this feature.
End of explanation
df.drop(['Time', 'Amount'], axis=1, inplace=True)
df.head()
Explanation: We can't use the Time feature, so let us get rid of it. We don't need the Amount feature either.
End of explanation
df.shape
Explanation: Re-sample the Data
Now let us check the size of the dataset.
End of explanation
df['Class'].value_counts()
Explanation: The data set contains 284,807 examples. As kNN does not perform well with large data sets, we must find a way to reduce of size. We also need to ensure that the data is balanced before we run our kNN classifier.
End of explanation
def print_class0_proportion(target_column):
    class_0, class_1 = pd.Series(target_column).value_counts()
    class_0_pct = round(class_0 / target_column.shape[0] * 100, 2)
    print('Class 0 represents {0}% of all the observations.'.format(class_0_pct))
print_class0_proportion(df['Class'])
Explanation: Just by looking at the numbers above, we can see that the data is not balanced. Let us print out the proportion of data that is in class 0.
End of explanation
X = df.drop('Class', axis=1)
y = df['Class']
from imblearn.under_sampling import RandomUnderSampler
sampler = RandomUnderSampler(random_state=42)
X_resampled, y_resampled = sampler.fit_sample(X, y)
Explanation: The data set is highly unbalanced. Only a very small proportion (0.17%) of the transactions are fraudulent while the majority of the examples are normal. In the real world, this makes sense. However, training a classifier with unbalanced data results in predictions that are biased towards predicting the majority class. Therefore, we must re-sample the data set so both classes have approximately 50-50 ratio. We can either:
Undersample the majority class or
Oversample the minority class
Undersampling the majority class is better for this data set because we want to reduce the number of examples to get kNN to perform well.
We use the imbalance-learn library from scikit-learn-contrib project. Remember to install it: conda install -c glemaitre imbalanced-learn.
End of explanation
print_class0_proportion(y_resampled)
Explanation: Now the ratio should be around 50 percent.
End of explanation
from sklearn.model_selection import train_test_split
X_resampled_train, X_resampled_test, y_resampled_train, y_resampled_test = train_test_split(X_resampled, y_resampled, test_size=.3)
Explanation: Partition the Data
Now we are ready to partition the data into training set and test set.
End of explanation
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import cross_val_score
k_values = np.arange(1, 42, 2) # odd values from 1 to 42
k_scores = []
for k in k_values:
    knn = KNeighborsClassifier(n_neighbors=k)
    scores = cross_val_score(knn, X_resampled_train, y_resampled_train, cv=10, scoring='accuracy')
    avg_score = np.mean(scores)
    k_scores.append(avg_score)
    
# Plot the scores
fig, ax = plt.subplots(figsize=(10,4))
ax.plot(k_values, k_scores, 'o-')
ax.set_xlabel('$k$')
ax.set_ylabel('Score')
Explanation: Pick an Appropriate $k$ Value
An appropriate $k$ value for the kNN algorithm is the one that yields the lowest error or the highest score. We can use 10-Fold Cross Validation to estimate the score associated with each $k$.
End of explanation
from sklearn.metrics import classification_report
knn = KNeighborsClassifier(n_neighbors=5)
knn.fit(X_resampled_train, y_resampled_train)
y_resampled_predictions = knn.predict(X_resampled_test)
print(classification_report(y_resampled_test, y_resampled_predictions))
Explanation: It is clear that the score peaks at $k=5$.
Evaluate Predictions
It is time to evaluate kNN where $k=5$ using a classification report.
End of explanation
from sklearn.metrics import confusion_matrix
confusion_matrix(y_resampled_test, y_resampled_predictions)
Explanation: These results for the under-sampled data are quite good. Here is the confusion matrix:
End of explanation
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.3)
predicted = knn.predict(X_test)
print(classification_report(y_test, predicted))
print(confusion_matrix(y_test, predicted))
Explanation: Now let us predict on a subset of the whole data.
End of explanation
<END_TASK>