<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 SVD Applied to a Word-Document Matrix
This notebook applies the SVD to a simple word-document matrix. The aim is to see what the reconstructed reduced dimension matrix looks like.
Step1:  A Simple Word-Document Matrix
The example word-document matrix is taken from http
Step2:  Word-Document Matrix is A
Step3:  Now Take the SVD
Step4:  We can see above that the values in the diagonal S matrix are ordered by magnitide. There is a significant different between the biggest value 1.1, and the smallest 0.05. The halfway value of 0.28 is still much smaller than the largest.
Check U, S and V Do Actually Reconstruct A
Step5:  Yes, that worked .. the reconstructed A2 is the same as the original A (within the bounds of small floating point accuracy)
Now Reduce Dimensions, Extract Topics
Here we use only the top 2 values of the S singular value matrix, pretty brutal reduction in dimensions!
Step6:  New View Of Documents
Step7:  The above shows that there are 2 clusters of documents. That matches our expectations as we constructed the example data set that way.
Topics from New View of Words
<ASSISTANT_TASK:>
Python Code:
#import pandas for conviently labelled arrays
import pandas
# import numpy for SVD function
import numpy
# import matplotlib.pyplot for visualising arrays
import matplotlib.pyplot as plt
Explanation: SVD Applied to a Word-Document Matrix
This notebook applies the SVD to a simple word-document matrix. The aim is to see what the reconstructed reduced dimension matrix looks like.
End of explanation
# create a simple word-document matrix as a pandas dataframe, the content values have been normalised
words = ['wheel', ' seat', ' engine', ' slice', ' oven', ' boil']
print(words)
documents = ['doc1', 'doc2', 'doc3', 'doc4', 'doc5', 'doc6']
word_doc = pandas.DataFrame([[0.5,0.3333, 0.25, 0, 0, 0],
                      [0.25, 0.3333, 0, 0, 0, 0],
                      [0.25, 0.3333, 0.75, 0,0,0],
                      [0, 0, 0, 0.5, 0.5, 0.6],
                      [0, 0, 0, 0.3333, 0.1667, 0],
                      [0, 0, 0, 0.1667, 0.3333, 0.4]], index=words, columns=documents)
# and show it
word_doc
Explanation: A Simple Word-Document Matrix
The example word-document matrix is taken from http://makeyourowntextminingtoolkit.blogspot.co.uk/2016/11/so-many-dimensions-and-how-to-reduce.html
End of explanation
# create a numpy array from the pandas dataframe
A = word_doc.values
Explanation: Word-Document Matrix is A
End of explanation
# break it down into an SVD
U, s, VT = numpy.linalg.svd(A, full_matrices=False)
S = numpy.diag(s)
# what are U, S and V
print("U =\n", numpy.round(U, decimals=2), "\n")
print("S =\n", numpy.round(S, decimals=2), "\n")
print("V^T =\n", numpy.round(VT, decimals=2), "\n")
Explanation: Now Take the SVD
End of explanation
# rebuild A2 from U.S.V
A2 = numpy.dot(U,numpy.dot(S,VT))
print("A2 =\n", numpy.round(A2, decimals=2))
Explanation: We can see above that the values in the diagonal S matrix are ordered by magnitide. There is a significant different between the biggest value 1.1, and the smallest 0.05. The halfway value of 0.28 is still much smaller than the largest.
Check U, S and V Do Actually Reconstruct A
End of explanation
# S_reduced is the same as S but with only the top 2 elements kept
S_reduced = numpy.zeros_like(S)
# only keep top two eigenvalues
l = 2
S_reduced[:l, :l] = S[:l,:l]
# show S_reduced which has less info than original S
print("S_reduced =\n", numpy.round(S_reduced, decimals=2))
Explanation: Yes, that worked .. the reconstructed A2 is the same as the original A (within the bounds of small floating point accuracy)
Now Reduce Dimensions, Extract Topics
Here we use only the top 2 values of the S singular value matrix, pretty brutal reduction in dimensions!
End of explanation
# what is the document matrix now?
S_reduced_VT = numpy.dot(S_reduced, VT)
print("S_reduced_VT = \n", numpy.round(S_reduced_VT, decimals=2))
# plot the array
p = plt.subplot(111)
p.axis('scaled'); p.axis([-2, 2, -2, 2]); p.axhline(y=0, color='lightgrey'); p.axvline(x=0, color='lightgrey')
p.set_yticklabels([]); p.set_xticklabels([])
p.set_title("S_reduced_VT")
p.plot(S_reduced_VT[0,],S_reduced_VT[1,],'ro')
plt.show()
Explanation: New View Of Documents
End of explanation
# topics are a linear combination of original words
U_S_reduced = numpy.dot(U, S_reduced)
df = pandas.DataFrame(numpy.round(U_S_reduced, decimals=2), index=words)
# show colour coded so it is easier to see significant word contributions to a topic
df.style.background_gradient(cmap=plt.get_cmap('Blues'), low=0, high=2)
Explanation: The above shows that there are 2 clusters of documents. That matches our expectations as we constructed the example data set that way.
Topics from New View of Words
End of explanation
<END_TASK>