<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <a href='http
Step1:  Task #2
Step2:  Task #3
Step3:  Task #4
Step4:  Task #5
Step5:  Task #6
Step6:  Task #7
<ASSISTANT_TASK:>
Python Code:
import numpy as np
import pandas as pd
df = pd.read_csv('../TextFiles/moviereviews2.tsv', sep='\t')
df.head()
Explanation: <a href='http://www.pieriandata.com'> <img src='../Pierian_Data_Logo.png' /></a>
Text Classification Assessment - Solution
This assessment is very much like the Text Classification Project we just completed, and the dataset is very similar.
The moviereviews2.tsv dataset contains the text of 6000 movie reviews. 3000 are positive, 3000 are negative, and the text has been preprocessed as a tab-delimited file. As before, labels are given as pos and neg. 
We've included 20 reviews that contain either NaN data, or have strings made up of whitespace.
For more information on this dataset visit http://ai.stanford.edu/~amaas/data/sentiment/
Task #1: Perform imports and load the dataset into a pandas DataFrame
For this exercise you can load the dataset from '../TextFiles/moviereviews2.tsv'.
End of explanation
# Check for NaN values:
df.isnull().sum()
# Check for whitespace strings (it's OK if there aren't any!):
blanks = []  # start with an empty list
for i,lb,rv in df.itertuples():  # iterate over the DataFrame
    if type(rv)==str:            # avoid NaN values
        if rv.isspace():         # test 'review' for whitespace
            blanks.append(i)     # add matching index numbers to the list
        
len(blanks)
Explanation: Task #2: Check for missing values:
End of explanation
df.dropna(inplace=True)
Explanation: Task #3:  Remove NaN values:
End of explanation
df['label'].value_counts()
Explanation: Task #4: Take a quick look at the label column:
End of explanation
from sklearn.model_selection import train_test_split
X = df['review']
y = df['label']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)
Explanation: Task #5: Split the data into train & test sets:
You may use whatever settings you like. To compare your results to the solution notebook, use test_size=0.33, random_state=42
End of explanation
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.svm import LinearSVC
text_clf = Pipeline([('tfidf', TfidfVectorizer()),
                     ('clf', LinearSVC()),
])
# Feed the training data through the pipeline
text_clf.fit(X_train, y_train)
Explanation: Task #6: Build a pipeline to vectorize the date, then train and fit a model
You may use whatever model you like. To compare your results to the solution notebook, use LinearSVC.
End of explanation
# Form a prediction set
predictions = text_clf.predict(X_test)
# Report the confusion matrix
from sklearn import metrics
print(metrics.confusion_matrix(y_test,predictions))
# Print a classification report
print(metrics.classification_report(y_test,predictions))
# Print the overall accuracy
print(metrics.accuracy_score(y_test,predictions))
Explanation: Task #7: Run predictions and analyze the results
End of explanation
<END_TASK>