<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
Step1:  Unsupervised Clustering
calculate relationships between various Wikipedia articles
Step2:  First use word count to try to gauge article content
Step3:  The most common words are "the", "in", etc. which are not important. We'll use TF-IDF to fix this problem.
TF-IDF
TF = Term frequency
Number of times a word appears in the article
IDF = Inverse Document Frequency
Log of total # of articles divided by # of articles containing the word.
log(Total # articles / # articles containing word)
This applies discount to words that appear in a lot of articles, because they are likely less important, like "the", "in", "and" etc.
Step4:  These words are much more relevant than the ones from before.
Step5:  Build nearest neighbor model which returns closest articles by cosine distance
<ASSISTANT_TASK:>
Python Code:
import os
from urllib import urlretrieve
import graphlab
URL = 'https://d396qusza40orc.cloudfront.net/phoenixassets/people_wiki.csv'
def get_data(filename='people_wiki.csv', url=URL, force_download=False):
    
    Download and cache the fremont data
    
    Parameters
    ----------
    filename: string (optional)
        location to save the data
        
    url: string (optional)
    
    force_download: bool (optional)
        if True, force redownload of data
    
    Returns
    -------
    data: graphlab SFrame. Similer to a pandas DataFrame,
        but with capacity for faster analysis of larger data sets
    
       
    
    if force_download or not os.path.exists(filename):
        urlretrieve(url, filename)
        
    sf = graphlab.SFrame('people_wiki.csv')
    
    return sf
people = get_data()
people.head()
# Look at some individual article text
obama = people[people['name'] == 'Barack Obama']
obama['text']
clooney = people[people['name'] == 'George Clooney']
clooney['text']
Explanation: Unsupervised Clustering
calculate relationships between various Wikipedia articles
End of explanation
obama['word_count'] = graphlab.text_analytics.count_words(obama['text'])
obama['word_count']
# stack the word_count field (dict) in order to create a table for sorting. This will show the most common words
obama_wordcount_table = obama[['word_count']].stack('word_count', new_column_name=['word', 'count'])
obama_wordcount_table.sort('count', ascending=False)
Explanation: First use word count to try to gauge article content
End of explanation
# add word count field
people['word_count'] = graphlab.text_analytics.count_words(people['text'])
#use graphlab's built in tfidf, which uses calculation above
people['tfidf'] = graphlab.text_analytics.tf_idf(people['word_count'])
people.head()
# look at tfidf for Obama
obama = people[people['name']=='Barack Obama']
obama[['tfidf']].stack('tfidf', new_column_name=['word', 'tfidf']).sort('tfidf', ascending=False)
Explanation: The most common words are "the", "in", etc. which are not important. We'll use TF-IDF to fix this problem.
TF-IDF
TF = Term frequency
Number of times a word appears in the article
IDF = Inverse Document Frequency
Log of total # of articles divided by # of articles containing the word.
log(Total # articles / # articles containing word)
This applies discount to words that appear in a lot of articles, because they are likely less important, like "the", "in", "and" etc.
End of explanation
# look at two more people
clinton = people[people['name'] == 'Bill Clinton']
beckham = people[people['name'] == 'David Beckham']
Tswift = people[people['name'] == 'Taylor Swift']
# calculate distance between two articles using cosine distance. Min is 0, max distance is 1.
graphlab.distances.cosine(obama['tfidf'][0], clinton['tfidf'][0])
graphlab.distances.cosine(obama['tfidf'][0], Tswift['tfidf'][0])
Explanation: These words are much more relevant than the ones from before.
End of explanation
knn_model = graphlab.nearest_neighbors.create(people, features=['tfidf'], label='name')
# closest Wikipedia articles to Obama
knn_model.query(obama)
knn_model.query(Tswift)
Explanation: Build nearest neighbor model which returns closest articles by cosine distance
End of explanation
<END_TASK>