<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Load review dataset
Step1:  1. listing the name of the first 10 products in the dataset.
Step2:  2. counting the number of positive and negative reviews.
Step3:  Apply text cleaning on the review data
3. load the features
Step4:  4. data transformations
Step5:  5. compute a count for the number of times the word occurs in the review
Step6:  ### 7. compute the number of product reviews that contain the word perfect.
Step7:  1. Quiz Question.
How many reviews contain the word perfect?
Answer
2955
Convert data frame to multi-dimensional array
8.  convert our data frame to a multi-dimensional array.
The function should accept three parameters
Step8:  9. extract two arrays feature_matrix and sentiment
Step9:  2. Quiz Question
Step10:  2. Answer
Step12:  Compute derivative of log likelihood with respect to a single coefficient
11. computes the derivative of log likelihood with respect to a single coefficient w_j
The function should do the following
Step13:  12. Write a function compute_log_likelihood
Step15:  Taking gradient steps
13. Write a function logistic_regression to fit a logistic regression model using gradient ascent.
The function accepts the following parameters
Step16:  14. run the logistic regression solver
feature_matrix = feature_matrix extracted in #9
sentiment = sentiment extracted in #9
initial_coefficients = a 194-dimensional vector filled with zeros
step_size = 1e-7
max_iter = 301
Step18:  5. Quiz question 
Step19:  6. Quiz question
Step20:  7. Quiz question
Step21:  Ten "most positive" words
18. Compute the 10 most positive words
Step22:  8. Quiz question
Step23:  9. Quiz question
<ASSISTANT_TASK:>
Python Code:
products = pd.read_csv('amazon_baby_subset.csv')
Explanation: Load review dataset
End of explanation
products['name'][:10]
Explanation: 1. listing the name of the first 10 products in the dataset.
End of explanation
print (products['sentiment'] == 1).sum()
print (products['sentiment'] == -1).sum()
print (products['sentiment']).count()
Explanation: 2. counting the number of positive and negative reviews.
End of explanation
import json
with open('important_words.json') as important_words_file:    
    important_words = json.load(important_words_file)
print important_words[:3]
Explanation: Apply text cleaning on the review data
3. load the features
End of explanation
products = products.fillna({'review':''})  # fill in N/A's in the review column
def remove_punctuation(text):
    import string
    return text.translate(None, string.punctuation) 
products['review_clean'] = products['review'].apply(remove_punctuation)
products.head(3)
Explanation: 4. data transformations:
fill n/a values in the review column with empty strings
Remove punctuation
Compute word counts (only for important_words)
End of explanation
for word in important_words:
    products[word] = products['review_clean'].apply(lambda s : s.split().count(word))
products.head(1)
Explanation: 5. compute a count for the number of times the word occurs in the review
End of explanation
products['contains_perfect'] = products['perfect'] >=1
print products['contains_perfect'].sum()
Explanation: ### 7. compute the number of product reviews that contain the word perfect.
End of explanation
def get_numpy_data(dataframe, features, label):
    dataframe['constant'] = 1
    features = ['constant'] + features
    features_frame = dataframe[features]
    feature_matrix = features_frame.as_matrix()
    label_sarray = dataframe[label]
    label_array = label_sarray.as_matrix()
    return(feature_matrix, label_array)
Explanation: 1. Quiz Question.
How many reviews contain the word perfect?
Answer
2955
Convert data frame to multi-dimensional array
8.  convert our data frame to a multi-dimensional array.
The function should accept three parameters:
- dataframe: a data frame to be converted
- features: a list of string, containing the names of the columns that are used as features.
- label: a string, containing the name of the single column that is used as class labels.
The function should return two values:
one 2D array for features
one 1D array for class labels
The function should do the following:
- Prepend a new column constant to dataframe and fill it with 1's. This column takes account of the intercept term. Make sure that the constant column appears first in the data frame.
- Prepend a string 'constant' to the list features. Make sure the string 'constant' appears first in the list.
- Extract columns in dataframe whose names appear in the list features.
- Convert the extracted columns into a 2D array using a function in the data frame library. If you are using Pandas, you would use as_matrix() function.
- Extract the single column in dataframe whose name corresponds to the string label.
- Convert the column into a 1D array.
- Return the 2D array and the 1D array.
End of explanation
feature_matrix, sentiment = get_numpy_data(products, important_words, 'sentiment')
Explanation: 9. extract two arrays feature_matrix and sentiment
End of explanation
print feature_matrix.shape
Explanation: 2. Quiz Question:
How many features are there in the feature_matrix?
End of explanation
'''
feature_matrix: N * D
coefficients: D * 1
predictions: N * 1
produces probablistic estimate for P(y_i = +1 | x_i, w).
estimate ranges between 0 and 1.
'''
def predict_probability(feature_matrix, coefficients):
    # Take dot product of feature_matrix and coefficients  
    # YOUR CODE HERE
    score = np.dot(feature_matrix, coefficients) # N * 1
    
    # Compute P(y_i = +1 | x_i, w) using the link function
    # YOUR CODE HERE
    predictions = 1.0/(1+np.exp(-score))
    
    # return predictions
    return predictions
Explanation: 2. Answer:
194
3. Quiz Question:
Assuming that the intercept is present, how does the number of features in feature_matrix relate to the number of features in the logistic regression model?
Estimating conditional probability with link function
10. Compute predictions given by the link function.
Take two parameters: feature_matrix and coefficients.
First compute the dot product of feature_matrix and coefficients.
Then compute the link function P(y=+1|x,w).
Return the predictions given by the link function.
End of explanation
errors: N * 1
feature: N * 1
derivative: 1 
def feature_derivative(errors, feature):     
    # Compute the dot product of errors and feature
    derivative = np.dot(np.transpose(errors), feature)
    # Return the derivative
    return derivative
Explanation: Compute derivative of log likelihood with respect to a single coefficient
11. computes the derivative of log likelihood with respect to a single coefficient w_j
The function should do the following:
Take two parameters errors and feature.
Compute the dot product of errors and feature.
Return the dot product. This is the derivative with respect to a single coefficient w_j.
End of explanation
def compute_log_likelihood(feature_matrix, sentiment, coefficients):
    indicator = (sentiment==+1)
    scores = np.dot(feature_matrix, coefficients)
    # scores.shape (53072L, 1L)
    # indicator.shape (53072L,)
    lp = np.sum((np.transpose(np.array([indicator]))-1)*scores - np.log(1. + np.exp(-scores)))
    return lp
Explanation: 12. Write a function compute_log_likelihood
End of explanation
# coefficients: D * 1
from math import sqrt
def logistic_regression(feature_matrix, sentiment, initial_coefficients, step_size, max_iter):
    coefficients = np.array(initial_coefficients) # make sure it's a numpy array
    # lplist = []
    for itr in xrange(max_iter):
        # Predict P(y_i = +1|x_1,w) using your predict_probability() function
        # YOUR CODE HERE
        predictions = predict_probability(feature_matrix, coefficients)
        # Compute indicator value for (y_i = +1)
        indicator = (sentiment==+1)
        # Compute the errors as indicator - predictions
        errors = np.transpose(np.array([indicator])) - predictions
        for j in xrange(len(coefficients)): # loop over each coefficient
            # Recall that feature_matrix[:,j] is the feature column associated with coefficients[j]
            # compute the derivative for coefficients[j]. Save it in a variable called derivative
            # YOUR CODE HERE
            derivative = feature_derivative(errors, feature_matrix[:,j])
            # add the step size times the derivative to the current coefficient
            # YOUR CODE HERE
            coefficients[j] += step_size*derivative
        # Checking whether log likelihood is increasing
        if itr <= 15 or (itr <= 100 and itr % 10 == 0) or (itr <= 1000 and itr % 100 == 0) \
        or (itr <= 10000 and itr % 1000 == 0) or itr % 10000 == 0:
            # lplist.append(compute_log_likelihood(feature_matrix, sentiment, coefficients))
            lp = compute_log_likelihood(feature_matrix, sentiment, coefficients)
            print 'iteration %*d: log likelihood of observed labels = %.8f' % \
                (int(np.ceil(np.log10(max_iter))), itr, lp)
    
    import matplotlib.pyplot as plt
    x= [i for i in range(len(lplist))]
    plt.plot(x,lplist,'ro')
    plt.show()
    
    return coefficients
Explanation: Taking gradient steps
13. Write a function logistic_regression to fit a logistic regression model using gradient ascent.
The function accepts the following parameters:
feature_matrix: 2D array of features
sentiment: 1D array of class labels
initial_coefficients: 1D array containing initial values of coefficients
step_size: a parameter controlling the size of the gradient steps
max_iter: number of iterations to run gradient ascent
The function returns the last set of coefficients after performing gradient ascent.
The function carries out the following steps:
Initialize vector coefficients to initial_coefficients.
Predict the class probability P(yi=+1|xi,w) using your predict_probability function and save it to variable predictions.
Compute indicator value for (yi=+1) by comparing sentiment against +1. Save it to variable indicator.
Compute the errors as difference between indicator and predictions. Save the errors to variable errors.
For each j-th coefficient, compute the per-coefficient derivative by calling feature_derivative with the j-th column of feature_matrix. Then increment the j-th coefficient by (step_size*derivative).
Once in a while, insert code to print out the log likelihood.
Repeat steps 2-6 for max_iter times.
End of explanation
initial_coefficients = np.zeros((194,1))
step_size = 1e-7
max_iter = 301
coefficients = logistic_regression(feature_matrix, sentiment, initial_coefficients, step_size, max_iter)
coefficients = logistic_regression(feature_matrix, sentiment, initial_coefficients, step_size, max_iter)
Explanation: 14. run the logistic regression solver
feature_matrix = feature_matrix extracted in #9
sentiment = sentiment extracted in #9
initial_coefficients = a 194-dimensional vector filled with zeros
step_size = 1e-7
max_iter = 301
End of explanation
feature_matrix: N * D
coefficients: D * 1
predictions: N * 1
predictions = predict_probability(feature_matrix, coefficients)
NumPositive = (predictions > 0.5).sum()
print NumPositive
score = np.dot(feature_matrix, coefficients) # N * 1
print (score > 0).sum()
Explanation: 5. Quiz question :
As each iteration of gradient ascent passes, does the log likelihood increase or decrease?
Answer
increase
15. compute class predictions
First compute the scores using feature_matrix and coefficients using a dot product.
Then apply threshold 0 on the scores to compute the class predictions. Refer to the formula above.
End of explanation
print 0 in products['sentiment']
print -1 in products['sentiment']
print np.transpose(predictions.flatten()).shape
print (products['sentiment']).shape
print (np.transpose(predictions.flatten()))[:5]
correct_num = np.sum((np.transpose(predictions.flatten())> 0.5) == np.array(products['sentiment']>0))
total_num = len(products['sentiment'])
print "correct_num: {}, total_num: {}".format(correct_num, total_num)
accuracy = correct_num * 1./ total_num
print accuracy
np.transpose(predictions.flatten())> 0.5
np.array(products['sentiment']>0)
correct_num = np.sum((np.transpose(score.flatten())> 0) == np.array(products['sentiment']>0))
total_num = len(products['sentiment'])
print "correct_num: {}, total_num: {}".format(correct_num, total_num)
accuracy = correct_num * 1./ total_num
print accuracy
Explanation: 6. Quiz question:
How many reviews were predicted to have positive sentiment?
Answer:
25126
Measuring accuracy
End of explanation
coefficients = list(coefficients[1:]) # exclude intercept
word_coefficient_tuples = [(word, coefficient) for word, coefficient in zip(important_words, coefficients)]
word_coefficient_tuples = sorted(word_coefficient_tuples, key=lambda x:x[1], reverse=True)
Explanation: 7. Quiz question:
What is the accuracy of the model on predictions made above? (round to 2 digits of accuracy)
Answer:
0.75
Which words contribute most to positive & negative sentiments
17.compute the "most positive words"
Treat each coefficient as a tuple, i.e. (word, coefficient_value). The intercept has no corresponding word, so throw it out.
Sort all the (word, coefficient_value) tuples by coefficient_value in descending order. Save the sorted list of tuples to word_coefficient_tuples.
End of explanation
word_coefficient_tuples[:10]
Explanation: Ten "most positive" words
18. Compute the 10 most positive words
End of explanation
word_coefficient_tuples[-10:]
Explanation: 8. Quiz question:
Which word is not present in the top 10 "most positive" words?
Ten "most negative" words
End of explanation
print np.array([1,2,3])==np.array([1,3,2])
Explanation: 9. Quiz question:
Which word is not present in the top 10 "most negative" words?
End of explanation
<END_TASK>