<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Document retrieval from wikipedia data
Fire up GraphLab Create
Step1:  Load some text data - from wikipedia, pages on people
Step2:  Data contains
Step3:  Explore the dataset and checkout the text it contains
Exploring the entry for president Obama
Step4:  Exploring the entry for actor George Clooney
Step5:  Get the word counts for Obama article
Step6:  Sort the word counts for the Obama article
Turning dictonary of word counts into a table
Step7:  Sorting the word counts to show most common words at the top
Step8:  Most common words include uninformative words like "the", "in", "and",...
Compute TF-IDF for the corpus
To give more weight to informative words, we weigh them by their TF-IDF scores.
Step9:  Examine the TF-IDF for the Obama article
Step10:  Words with highest TF-IDF are much more informative.
Manually compute distances between a few people
Let's manually compare the distances between the articles for a few famous people.
Step11:  Is Obama closer to Clinton than to Beckham?
We will use cosine distance, which is given by
(1-cosine_similarity) 
and find that the article about president Obama is closer to the one about former president Clinton than that of footballer David Beckham.
Step12:  Build a nearest neighbor model for document retrieval
We now create a nearest-neighbors model and apply it to document retrieval.
Step13:  Applying the nearest-neighbors model for retrieval
Who is closest to Obama?
Step14:  As we can see, president Obama's article is closest to the one about his vice-president Biden, and those of other politicians.  
Other examples of document retrieval
Step15:  Exercise
1) Compare top words according to word counts to TF-IDF
In the notebook we covered in the module, explored two document representations
Step16:  2) Measuring distance
Elton John is a famous singer; let’s compute the distance between his article and those of two other famous singers. In this assignment, you will use the cosine distance, which one measure of similarity between vectors, similar to the one discussed in the lectures. You can compute this distance using the graphlab.distances.cosine function. What’s the cosine distance between the articles on ‘Elton John’ and ‘Victoria Beckham’? What’s the cosine distance between the articles on ‘Elton John’ and Paul McCartney’? Which one of the two is closest to Elton John? Does this result make sense to you?
Step17:  3) Building nearest neighbors models with different input features and setting the distance metric
In the sample notebook, we built a nearest neighbors model for retrieving articles using TF-IDF as features and using the default setting in the construction of the nearest neighbors model. Now, you will build two nearest neighbors models
Step18:  Now we are ready to use our model to retrieve documents. Use these two models to collect the following results
Step19:  What’s the most similar article, other than itself, to the one on ‘Elton John’ using TF-IDF features?
Step20:  What’s the most similar article, other than itself, to the one on ‘Victoria Beckham’ using word count features?
Step21:  What’s the most similar article, other than itself, to the one on ‘Victoria Beckham’ using TF-IDF features?
<ASSISTANT_TASK:>
Python Code:
import graphlab
Explanation: Document retrieval from wikipedia data
Fire up GraphLab Create
End of explanation
people = graphlab.SFrame('people_wiki.gl/')
Explanation: Load some text data - from wikipedia, pages on people
End of explanation
people.head()
len(people)
Explanation: Data contains:  link to wikipedia article, name of person, text of article.
End of explanation
obama = people[people['name'] == 'Barack Obama']
obama
obama['text']
Explanation: Explore the dataset and checkout the text it contains
Exploring the entry for president Obama
End of explanation
clooney = people[people['name'] == 'George Clooney']
clooney['text']
Explanation: Exploring the entry for actor George Clooney
End of explanation
obama['word_count'] = graphlab.text_analytics.count_words(obama['text'])
print obama['word_count']
Explanation: Get the word counts for Obama article
End of explanation
obama_word_count_table = obama[['word_count']].stack('word_count', new_column_name = ['word','count'])
Explanation: Sort the word counts for the Obama article
Turning dictonary of word counts into a table
End of explanation
obama_word_count_table.head()
obama_word_count_table.sort('count',ascending=False)
Explanation: Sorting the word counts to show most common words at the top
End of explanation
people['word_count'] = graphlab.text_analytics.count_words(people['text'])
people.head()
tfidf = graphlab.text_analytics.tf_idf(people['word_count'])
tfidf
people['tfidf'] = tfidf['docs']
Explanation: Most common words include uninformative words like "the", "in", "and",...
Compute TF-IDF for the corpus
To give more weight to informative words, we weigh them by their TF-IDF scores.
End of explanation
obama = people[people['name'] == 'Barack Obama']
obama[['tfidf']].stack('tfidf',new_column_name=['word','tfidf']).sort('tfidf',ascending=False)
Explanation: Examine the TF-IDF for the Obama article
End of explanation
clinton = people[people['name'] == 'Bill Clinton']
beckham = people[people['name'] == 'David Beckham']
Explanation: Words with highest TF-IDF are much more informative.
Manually compute distances between a few people
Let's manually compare the distances between the articles for a few famous people.
End of explanation
graphlab.distances.cosine(obama['tfidf'][0],clinton['tfidf'][0])
graphlab.distances.cosine(obama['tfidf'][0],beckham['tfidf'][0])
Explanation: Is Obama closer to Clinton than to Beckham?
We will use cosine distance, which is given by
(1-cosine_similarity) 
and find that the article about president Obama is closer to the one about former president Clinton than that of footballer David Beckham.
End of explanation
knn_model = graphlab.nearest_neighbors.create(people,features=['tfidf'],label='name')
Explanation: Build a nearest neighbor model for document retrieval
We now create a nearest-neighbors model and apply it to document retrieval.
End of explanation
knn_model.query(obama)
Explanation: Applying the nearest-neighbors model for retrieval
Who is closest to Obama?
End of explanation
swift = people[people['name'] == 'Taylor Swift']
knn_model.query(swift)
jolie = people[people['name'] == 'Angelina Jolie']
knn_model.query(jolie)
arnold = people[people['name'] == 'Arnold Schwarzenegger']
knn_model.query(arnold)
Explanation: As we can see, president Obama's article is closest to the one about his vice-president Biden, and those of other politicians.  
Other examples of document retrieval
End of explanation
people.head(2)
elton_john = people[people['name'] == 'Elton John']
elton_john.head()
elton_john[['word_count']].stack('word_count', new_column_name = ['word','count']).sort('count',ascending=False)
elton_john[['tfidf']].stack('tfidf', new_column_name = ['word','tfidf']).sort('tfidf',ascending=False)
Explanation: Exercise
1) Compare top words according to word counts to TF-IDF
In the notebook we covered in the module, explored two document representations: word counts and TF-IDF. Now, take a particular famous person, 'Elton John'. What are the 3 words in his articles with highest word counts? What are the 3 words in his articles with highest TF-IDF? These results illustrate why TF-IDF is useful for finding important words. Save these results to answer the quiz at the end.
End of explanation
victoria_beckham = people[people['name'] == 'Victoria Beckham']
paul_mccartney = people[people['name'] == 'Paul McCartney']
graphlab.distances.cosine(elton_john['tfidf'][0],victoria_beckham['tfidf'][0])
graphlab.distances.cosine(elton_john['tfidf'][0], paul_mccartney['tfidf'][0])
Explanation: 2) Measuring distance
Elton John is a famous singer; let’s compute the distance between his article and those of two other famous singers. In this assignment, you will use the cosine distance, which one measure of similarity between vectors, similar to the one discussed in the lectures. You can compute this distance using the graphlab.distances.cosine function. What’s the cosine distance between the articles on ‘Elton John’ and ‘Victoria Beckham’? What’s the cosine distance between the articles on ‘Elton John’ and Paul McCartney’? Which one of the two is closest to Elton John? Does this result make sense to you?
End of explanation
word_count_cosine_model = graphlab.nearest_neighbors.create(people,features=['word_count'],label='name',distance='cosine')
tfidf_cosine_model = graphlab.nearest_neighbors.create(people,features=['tfidf'],label='name',distance='cosine')
Explanation: 3) Building nearest neighbors models with different input features and setting the distance metric
In the sample notebook, we built a nearest neighbors model for retrieving articles using TF-IDF as features and using the default setting in the construction of the nearest neighbors model. Now, you will build two nearest neighbors models:
Using word counts as features
Using TF-IDF as features
In both of these models, we are going to set the distance function to cosine similarity. 
Here is how: when you call the function
add the parameter:
End of explanation
word_count_cosine_model.query(elton_john)
Explanation: Now we are ready to use our model to retrieve documents. Use these two models to collect the following results:
What’s the most similar article, other than itself, to the one on ‘Elton John’ using word count features?
End of explanation
tfidf_cosine_model.query(elton_john)
Explanation: What’s the most similar article, other than itself, to the one on ‘Elton John’ using TF-IDF features?
End of explanation
word_count_cosine_model.query(victoria_beckham)
Explanation: What’s the most similar article, other than itself, to the one on ‘Victoria Beckham’ using word count features?
End of explanation
tfidf_cosine_model.query(victoria_beckham)
Explanation: What’s the most similar article, other than itself, to the one on ‘Victoria Beckham’ using TF-IDF features?
End of explanation
<END_TASK>