<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Improving the Text Classifier
Goals
Try Random Forest on our Sentiment Data
Try Support Vector Machines on our Sentiment Data
Introduction to Hyperparameters
Introduction
There are three ways to improve a machine learning model
Improve the training data
Improve the feature extraction
Improve the learning algorithm
Typically the easiest way is to get more data or cleaner data if you can do it.  If that's not possible adding more features is the next easiest.  The toughest way is to try to improve the learning algorithm.  Not that we have a baseline in place we will try improving the learning algorithm and adding a few features.
Support Vector Machines
Support vector machines or SVMs are a popular way to model data when you have many more features that records.
In our case we have about the same number of features as records so they might might sense to try.  We actually used it in Lesson 3 but now we can try it out with Cross Validation.  These take a little longer to train.
Step1:  This SVM is 67% accurate.
Recall that the Naive Bayes Classifier was 65% accurate.
This does seem slightly better although there is a fair amount of variation in the folds and it probably wouldn't
hold up to a statistical test.
There are other versions of SVMs we an try, but most of the others have a fit time that is quadratic in the number of records so they will be really slow.
Random Forests
Random forest is a really useful algorithm.  It starts with a very simple and effective algorithm called a decision tree which looks at a single feature at a time and branches based on whether or not that feature is above some threshold.  It then builds hundreds or thousands of these trees in parallel, each on different subsets of the data and gives each tree a single vote on which class the data is in.  Random Forests take a while to train, but will probably run faster in production than an SVM.
I introduce Random Forests herebecause they are incredibly robust - if I only had one algorithm to use in any scenario I would probably use Random Forests or their close cousin Boosted Decision Trees.
Let's try Random Forests on our dataset - this may take a while.
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import numpy as np
from sklearn.model_selection import cross_val_score
df = pd.read_csv('../scikit/tweets.csv')
target = df['is_there_an_emotion_directed_at_a_brand_or_product']
text = df['tweet_text']
# We need to remove the empty rows from the text before we pass into CountVectorizer
fixed_text = text[pd.notnull(text)]
fixed_target = target[pd.notnull(text)]
# Do the feature extraction
from sklearn.feature_extraction.text import CountVectorizer
count_vect = CountVectorizer()              # initialize the count vectorizer
count_vect.fit(fixed_text)                  # set up the columns for the feature matrix
counts = count_vect.transform(fixed_text)   # counts is the feature matrix
from sklearn.svm import LinearSVC
# Build a classifier using the LinearSVC algorithm
clf = LinearSVC()                           # initialize our classifier
clf.fit(counts, fixed_target)               # fit our classifier to the training data
scores = cross_val_score(clf, counts, fixed_target, cv=10)
print(scores)
print(scores.mean())
Explanation: Improving the Text Classifier
Goals
Try Random Forest on our Sentiment Data
Try Support Vector Machines on our Sentiment Data
Introduction to Hyperparameters
Introduction
There are three ways to improve a machine learning model
Improve the training data
Improve the feature extraction
Improve the learning algorithm
Typically the easiest way is to get more data or cleaner data if you can do it.  If that's not possible adding more features is the next easiest.  The toughest way is to try to improve the learning algorithm.  Not that we have a baseline in place we will try improving the learning algorithm and adding a few features.
Support Vector Machines
Support vector machines or SVMs are a popular way to model data when you have many more features that records.
In our case we have about the same number of features as records so they might might sense to try.  We actually used it in Lesson 3 but now we can try it out with Cross Validation.  These take a little longer to train.
End of explanation
from sklearn.ensemble import RandomForestClassifier
clf = RandomForestClassifier(n_estimators=50) # n_estimators is the number of trees
from sklearn.model_selection import cross_val_score
scores = cross_val_score(clf, counts, fixed_target, cv=10)
print(scores)
print(scores.mean())
Explanation: This SVM is 67% accurate.
Recall that the Naive Bayes Classifier was 65% accurate.
This does seem slightly better although there is a fair amount of variation in the folds and it probably wouldn't
hold up to a statistical test.
There are other versions of SVMs we an try, but most of the others have a fit time that is quadratic in the number of records so they will be really slow.
Random Forests
Random forest is a really useful algorithm.  It starts with a very simple and effective algorithm called a decision tree which looks at a single feature at a time and branches based on whether or not that feature is above some threshold.  It then builds hundreds or thousands of these trees in parallel, each on different subsets of the data and gives each tree a single vote on which class the data is in.  Random Forests take a while to train, but will probably run faster in production than an SVM.
I introduce Random Forests herebecause they are incredibly robust - if I only had one algorithm to use in any scenario I would probably use Random Forests or their close cousin Boosted Decision Trees.
Let's try Random Forests on our dataset - this may take a while.
End of explanation
<END_TASK>