<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 <h1>Constructing Ego Networks from Retweets</h1>
Yotam Shmargad<br>
University of Arizona<br>
Email
Step1:  <h2>2. Pulling ego tweets</h2>
Step2:  <h2>3. Pulling retweeters</h2>
Step3:  <h2>4. Visualizing the network of retweeters</h2>
Step4:  <h2>5. Pulling retweeter tweets</h2>
Step5:  <h2>6. Pulling retweeters of retweeters</h2>
Step6:  <h2>7. Visualizing the full network of retweeters</h2>
<ASSISTANT_TASK:>
Python Code:
# Install tweepy
# !pip install tweepy
# Import the libraries we need
import tweepy
import json
import time
import networkx
import os
import matplotlib.pyplot as plt
from collections import Counter
# Authenticate!
auth = tweepy.OAuthHandler("Consumer Key", "Consumer Secret")
auth.set_access_token("Access Token", "Access Token Secret")
api = tweepy.API(auth)
# Check working directory
os.getcwd()
# Set working directory
os.chdir('FOLDER FOR SAVING FILES')
# Check working directory
os.getcwd()
Explanation: <h1>Constructing Ego Networks from Retweets</h1>
Yotam Shmargad<br>
University of Arizona<br>
Email: yotam@email.arizona.edu<br>
Web: www.yotamshmargad.com
<h2>Introduction</h2>
<br>
Twitter has become a prominent online social network, playing a major role in how people all over the world share and consume information. Moreover, while some social networks have made it difficult for researchers to extract data from their servers, Twitter remains relatively open for now. This tutorial will go through the details of how to construct a Twitter user’s ego network from retweets they have received on their tweets. Instead of focusing on who follows who on Twitter, the method instead conceptualizes edges as existing between users if they have recently retweeted each other.<br><br>
Conceptualizing edges as retweets has two primary benefits. First, it captures recent interactions between users rather than decisions that they may have made long ago (i.e. following each other) that may not translate into meaningful interaction today. Second, users often have many more followers than they do retweeters. The method proposed can thus be used to analyze even relatively popular users. The code goes through obtaining authorization from Twitter, taking into account the limits that Twitter imposes on data extraction, and handling errors generated from deleted tweets or users.
<h2>1. Importing libraries and getting Twitter's approval</h2>
End of explanation
# Keep track of API calls
# User timeline
callsUT = 0
# Retweeters
callsRT = 0
# Number of tweets to be pulled
# Ego
E = 10
# Alter
A = 10
# Existing user with tweets
ego = api.user_timeline(screen_name = "CUBoulder", count = E, include_rts = False, exclude_replies = True)
callsUT += 1
len(ego)
# Existing user with no tweets
ego = api.user_timeline(screen_name = "DeveloperYotam", count = E, include_rts = False, exclude_replies = True)
callsUT += 1
len(ego)
# Non-existing user
ego = api.user_timeline(screen_name = "fakeuserq4587937045", count = E, include_rts = False, exclude_replies = True)
callsUT += 1
# Handling errors
ego = []
egosn = "CUBoulder"
try:
    ego_raw = api.user_timeline(screen_name = egosn, count = E, include_rts = False, exclude_replies = True)
except tweepy.TweepError:
    print("fail!")
callsUT += 1
# Converting results to a list of json objects
ego = [egotweet._json for egotweet in ego_raw]
# Writing ego tweets to a json file
with open('egotweet.json', 'w') as file:
    json.dump(ego, file)
callsUT
# Looking at a json object
ego[0]
# Accessing an element of ego tweets
ego[0]["id_str"]
# Storing one of ego's tweet id
egoid = ego[0]["id_str"]
# Storing and printing ego tweet ids and retweet counts
tweetids = []
retweets = []
if len(ego) != 0:
    for egotweet in ego:
        tweetids.append(egotweet["id_str"])
        retweets.append(egotweet["retweet_count"])
        print(egotweet["id_str"],egotweet["retweet_count"])
Explanation: <h2>2. Pulling ego tweets</h2>
End of explanation
# Collecting Retweets
egort = api.retweets(ego[0]["id_str"])
callsRT += 1
len(egort)
callsRT
# Non-existing tweet
egort = api.retweets("garblegarble")
callsRT += 1
# Note: callsRT did not increase in the last command
callsRT
callsRT += 1
# Sleep for 10 seconds
time.sleep(10)
# Collecting retweeters of ego tweets
allretweeters = []
self = []
check = []
for egotweet in ego:
    retweeters = []
    try:
        selftweet = 0
        if callsRT >= 75:
            time.sleep(900)
        egort_raw = api.retweets(egotweet["id_str"])
        egort = [egoretweet._json for egoretweet in egort_raw]
        for retweet in egort:
            if retweet["user"]["id_str"]!=egoid:
                allretweeters.append((egoid,retweet["user"]["id_str"]))
                retweeters.append(retweet["user"]["id_str"])
            else:
                selftweet = 1
        check.append(len(retweeters))
        self.append(selftweet)                
    except tweepy.TweepError:
        check.append(0)
        self.append(0)
    
    callsRT += 1
# Writing results to files
with open('check.json', 'w') as file:
    json.dump(check, file)
with open('self.json', 'w') as file:
    json.dump(self, file)
    
with open('allretweeters.json', 'w') as file:
    json.dump(allretweeters, file)
# Printing tweet ids, retweet counts, 
# retweeters obtained, and whether a self tweet is included
for a, b, c, d in zip(tweetids,retweets,check,self):
    print(a, b, c, d)
len(allretweeters)
allretweeters
Explanation: <h2>3. Pulling retweeters</h2>
End of explanation
# Assigning edge weight to be number of tweets retweeted
weight = Counter()
for (i, j) in allretweeters:
    weight[(i, j)] +=1
weight
# Defining weighted edges
weighted_edges = list(weight.items())
weighted_edges
# Defining the network object
G = networkx.Graph()
G.add_edges_from([x[0] for x in weighted_edges])
# Visualizing the network
networkx.draw(G, width=[x[1] for x in weighted_edges])
Explanation: <h2>4. Visualizing the network of retweeters</h2>
End of explanation
# Defining the set of unique retweeters
unique = [x[0][1] for x in weighted_edges]
len(unique)
unique
callsUT
# Collecting and storing the tweets of retweeters
alter = []
alters = []
for retweeter in unique:
    try:
        if callsUT >= 900:
            time.sleep(900)
        alter_raw = api.user_timeline(retweeter, count = A, include_rts = False, exclude_replies = True)
        alter = [altertweet._json for altertweet in alter_raw]
        alters.append(alter)
    except tweepy.TweepError:
        print("fail!")
    callsUT += 1
with open('alters.json', 'w') as file:
    json.dump(alters, file)
callsUT
len(alters)
# Printing the number of tweets pulled for each retweeter
for alt in alters:
    print(len(alt))
# Storing and printing alter ids, tweet ids, and retweet counts
altids = []
alttweetids = []
altretweets = []
for alt in alters:
    for alttweet in alt:
        altids.append(alttweet["user"]["id_str"])
        alttweetids.append(alttweet["id_str"])
        altretweets.append(alttweet["retweet_count"])
        print(alttweet["user"]["id_str"],alttweet["id_str"],alttweet["retweet_count"])      
Explanation: <h2>5. Pulling retweeter tweets</h2>
End of explanation
# Collecting retweeters of alter tweets
allalt = []
altself = []
altcheck = []
for alt in alters:
    for alttweet in alt:
        altid = alttweet["user"]["id_str"]
        altretweeters = []
        try:
            selftweet = 0
            if callsRT >= 75:
                time.sleep(900)
            altrt_raw = api.retweets(alttweet["id_str"])
            altrt = [altretweet._json for altretweet in altrt_raw]
            for retweet in altrt:
                if retweet["user"]["id_str"]!=altid:
                    allalt.append((altid,retweet["user"]["id_str"]))
                    altretweeters.append(retweet["user"]["id_str"])
                else:
                    selftweet = 1
            altcheck.append(len(altretweeters))
            altself.append(selftweet)                
        except tweepy.TweepError:
            altcheck.append(0)
            altself.append(0)
    
        callsRT += 1
# Writing results to files
with open('altcheck.json', 'w') as file:
    json.dump(altcheck, file)
with open('altself.json', 'w') as file:
    json.dump(altself, file)
    
with open('altretweeters.json', 'w') as file:
    json.dump(altretweeters, file)
with open('allalt.json', 'w') as file:
    json.dump(allalt, file)
# Printing alter user ids, tweet ids, retweet counts, 
# retweeters obtained, and whether a self tweet is included
for a, b, c, d, e in zip(altids,alttweetids,altretweets,altcheck,altself):
    print(a, b, c, d, e)
len(allalt)
allalt
Explanation: <h2>6. Pulling retweeters of retweeters</h2>
End of explanation
weight = Counter()
for (i, j) in allalt:
    weight[(i, j)] +=1
weight
all_edges = weighted_edges + list(weight.items())
all_edges
# Defining the full network object
G = networkx.Graph()
G.add_edges_from([x[0] for x in all_edges])
# Visualizing the full network
networkx.draw(G, width=[x[1] for x in all_edges])
Explanation: <h2>7. Visualizing the full network of retweeters</h2>
End of explanation
<END_TASK>