<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Identifying Spam from SMS Text Messages
This analysis attempts to identify spam messages from a corpus of 5,574 SMS text messages.  The corpus is labeled as either spam or ham (legitimate messages) with 4,827 as ham and 747 as spam.  Using Sci-kit Learn and the Multinomial Naive Bayes model to classify messages as spam and ham.
We will look at various options to tune the model to see if we can get to 0 false positives in which legitimate messages are labled as spam.  It is expected that a small percentage of spam messages making it through the spam filter is preferable to legitimate messages being excluded.
Sources
Step1:  Loading the data from the UCI Machine Learning Repository
Step2:  Data Exploration
Step3:  Since the data is labeled for us, we can do further data exploration by taking a look at how spam and ham differ.
Step4:  In addition to the difference in the number of ham vs. spam, it appears that spam messages are generally longer than spam messages and more normally distributed than ham messages.
Step5:  Define the feature set through vectorization.
Step6:  Using Yellowbrick
Step7:  Using the default settings for our model does a pretty good job predicting spam and ham although not perfect.  The confusion matrix shows us that there are 12 false positives ( 5 actual spam messages that are predicted to be ham with 7 actual ham message predicted as spam).
I think it is more important to a user to receive 100% of their real messages while tolerating a few spam messages.  So let's see if we can tune the model to eliminate the false positives that are tagged as spam but are really ham.
We can use grid search with cross-validation to find the optimal alpha value.
Step8:  Since we are more concerned with minimizing the false positives especially with ham classified as spam, we will use an alpha value of 3.0 with fit_prior = True.
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import os
import json
import time
import pickle
import requests
from io import BytesIO
from zipfile import ZipFile
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction import text
import seaborn as sns
sns.set(font_scale=1.5)
Explanation: Identifying Spam from SMS Text Messages
This analysis attempts to identify spam messages from a corpus of 5,574 SMS text messages.  The corpus is labeled as either spam or ham (legitimate messages) with 4,827 as ham and 747 as spam.  Using Sci-kit Learn and the Multinomial Naive Bayes model to classify messages as spam and ham.
We will look at various options to tune the model to see if we can get to 0 false positives in which legitimate messages are labled as spam.  It is expected that a small percentage of spam messages making it through the spam filter is preferable to legitimate messages being excluded.
Sources:
https://archive.ics.uci.edu/ml/datasets/sms+spam+collection
https://radimrehurek.com/data_science_python/
http://adataanalyst.com/scikit-learn/countvectorizer-sklearn-example/
End of explanation
URL = 'https://archive.ics.uci.edu/ml/machine-learning-databases/00228/smsspamcollection.zip'
SMS_PATH = os.path.join('datasets', 'sms')
file_name = requests.get(URL)
zipfile = ZipFile(BytesIO(file_name.content))
zip_names = zipfile.namelist()
def fetch_data(file='SMSSPamCollection'):
    for file in zip_names:
        if not os.path.isdir(SMS_PATH):
            os.makedirs(SMS_PATH)
        outpath = os.path.join(SMS_PATH, file)
        extracted_file = zipfile.read(file)
        with open(outpath, 'wb') as f:
            f.write(extracted_file)           
        
        return outpath
               
DATA = fetch_data()
df = pd.read_csv(DATA, sep='\t', header=None)
df.columns = ['Label', 'Text']
Explanation: Loading the data from the UCI Machine Learning Repository
End of explanation
pd.set_option('max_colwidth', 220)
df.head(20)
df.describe()
df.info()
Explanation: Data Exploration
End of explanation
# Add a field to our dataframe with the length of each message.
df['Length'] = df['Text'].apply(len)
df.head()
df.groupby('Label').describe()
Explanation: Since the data is labeled for us, we can do further data exploration by taking a look at how spam and ham differ.
End of explanation
df.Length.plot(bins=100, kind='hist')
df.hist(column='Length', by='Label', bins=50, figsize=(10,4))
Explanation: In addition to the difference in the number of ham vs. spam, it appears that spam messages are generally longer than spam messages and more normally distributed than ham messages.
End of explanation
text_data = df['Text']
text_data.shape
# Give our target labels numbers.
df['Label_'] = df['Label'].map({'ham': 0, 'spam': 1})
#stop_words = text.ENGLISH_STOP_WORDS
#Adding stop words did not significantly improve the model.
#textWithoutNums = text_data.replace('\d+', 'NUM_', regex=True)
#Removing all of the numbers in the messages and replacing with a text string did not improve the model either.
vectorizer = CountVectorizer(analyzer='word') #, stop_words=stop_words)
#vectorizer.fit(textWithoutNums)
vectorizer.fit(text_data)
vectorizer.get_feature_names()
pd.DataFrame.from_dict(vectorizer.vocabulary_, orient='index').sort_values(by=0, ascending=False).head()
dtm = vectorizer.transform(text_data)
features = pd.DataFrame(dtm.toarray(), columns=vectorizer.get_feature_names())
features.shape
features.head()
X = features
y = np.array(df['Label_'].tolist())
from sklearn.cross_validation import train_test_split
from sklearn.naive_bayes import MultinomialNB
from sklearn import metrics
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)
print(X_train.shape, y_train.shape)
model = MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)
model.fit(X_train, y_train)
y_pred_class = model.predict(X_test)
print(metrics.classification_report(y_test, y_pred_class))
print('Accuracy Score: ', metrics.accuracy_score(y_test, y_pred_class))
Explanation: Define the feature set through vectorization.
End of explanation
from yellowbrick.classifier import ClassificationReport
bayes = MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)
visualizer = ClassificationReport(bayes, classes=['ham', 'spam'])
visualizer.fit(X_train, y_train)
visualizer.score(X_test, y_test)
g = visualizer.poof()
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred_class)
sns.set(font_scale=1.5)
ax = plt.subplot()
sns.heatmap(cm, annot=True, ax=ax, fmt='g', cbar=False)
ax.set_xlabel('Predicted')
ax.set_ylabel('Actual')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(['Ham', 'Spam'])
ax.yaxis.set_ticklabels(['Ham', 'Spam'])
plt.show()
Explanation: Using Yellowbrick
End of explanation
from sklearn.model_selection import train_test_split
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import classification_report
# Split the dataset in two equal parts
X_train_, X_test_, y_train_, y_test_ = train_test_split(
    X, y, test_size=0.5, random_state=1)
# Set the parameters by cross-validation
tuned_parameters = [{'alpha': [0.5, 1.0, 1.5, 2.0, 2.5, 3.0], 'class_prior':[None], 'fit_prior': [True, False]}]
scores = ['precision', 'recall']
for score in scores:
    print("### Tuning hyper-parameters for %s ###" % score)
    print()
    clf = GridSearchCV(MultinomialNB(), tuned_parameters, cv=5,
                       scoring='%s_macro' % score)
    clf.fit(X_train_, y_train_)
    print("Best parameters set found on development set:")
    print()
    print(clf.best_params_)
    print()
    print("Grid scores on development set:")
    print()
    means = clf.cv_results_['mean_test_score']
    stds = clf.cv_results_['std_test_score']
    for mean, std, params in zip(means, stds, clf.cv_results_['params']):
        print("%0.3f (+/-%0.03f) for %r"
              % (mean, std * 2, params))
    print()
    print("Detailed classification report:")
    print()
    print("The model is trained on the full development set.")
    print("The scores are computed on the full evaluation set.")
    print()
    y_true, y_pred = y_test, clf.predict(X_test)
    print(classification_report(y_true, y_pred))
    print()
    print('Accuracy Score: ', metrics.accuracy_score(y_test, y_pred))
    print()
Explanation: Using the default settings for our model does a pretty good job predicting spam and ham although not perfect.  The confusion matrix shows us that there are 12 false positives ( 5 actual spam messages that are predicted to be ham with 7 actual ham message predicted as spam).
I think it is more important to a user to receive 100% of their real messages while tolerating a few spam messages.  So let's see if we can tune the model to eliminate the false positives that are tagged as spam but are really ham.
We can use grid search with cross-validation to find the optimal alpha value.
End of explanation
from yellowbrick.classifier import ClassificationReport
bayes = MultinomialNB(alpha=3.0, class_prior=None, fit_prior=True)
visualizer = ClassificationReport(bayes, classes=['ham', 'spam'])
visualizer.fit(X_train, y_train)
visualizer.score(X_test, y_test)
g = visualizer.poof()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)
print(X_train.shape, y_train.shape)
model = MultinomialNB(alpha=3.0, class_prior=None, fit_prior=True)
model.fit(X_train, y_train)
y_pred_class = model.predict(X_test)
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred_class)
sns.set(font_scale=1.5)
ax = plt.subplot()
sns.heatmap(cm, annot=True, ax=ax, fmt='g', cbar=False)
ax.set_xlabel('Predicted')
ax.set_ylabel('Actual')
ax.set_title('Confusion Matrix')
ax.xaxis.set_ticklabels(['Ham', 'Spam'])
ax.yaxis.set_ticklabels(['Ham', 'Spam'])
plt.show()
Explanation: Since we are more concerned with minimizing the false positives especially with ham classified as spam, we will use an alpha value of 3.0 with fit_prior = True.
End of explanation
<END_TASK>