<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Characterizing Attackers
How many attackers are vandals vs normal users vs power users ?
are victims also trolls ?
investigate anons, gender, tenure, status
Step1:  Attacker Specific Analysis
Q
Step2:  Attack
Step3:  Q
Step4:  Victim Specific Analysis
Q Are victims also trolls?
Step5:  Shared Analysis
Q
Step6:  Q
Step7:  Q
Step8:  Q
Step9:  Q
Step10:  Q
Step11:  Q
<ASSISTANT_TASK:>
Python Code:
%load_ext autoreload
%autoreload 2
%matplotlib inline
import warnings
warnings.filterwarnings('ignore')
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
import pandas as pd
from load_utils import *
from analysis_utils import compare_groups,get_genders
d = load_diffs()
df_events, df_blocked_user_text = load_block_events_and_users()
Explanation: Characterizing Attackers
How many attackers are vandals vs normal users vs power users ?
are victims also trolls ?
investigate anons, gender, tenure, status
End of explanation
d['blocked'].groupby('user_text')['pred_aggression_score']\
        .agg( {'aggressiveness': np.mean})\
        .hist(bins = 50)
plt.xlabel('user level aggression score')
plt.ylabel('num users')
plt.title('')
# lets exclude anons 
d['blocked'].query('not author_anon').groupby('user_text')['pred_aggression_score']\
    .agg( {'aggressiveness': np.mean})\
    .hist(bins = 50)
plt.xlabel('user level aggression score')
plt.ylabel('num users')
plt.title('')
# lets compare to non-blocked users
# NOTE: would be better to have taken a random sample of users
d['2015'].query('not author_anon').groupby('user_text')['pred_aggression_score']\
    .agg( {'aggressiveness': np.mean})\
    .hist(bins = 50)
plt.xlabel('user level aggression score')
plt.ylabel('num users')
plt.title('')
Explanation: Attacker Specific Analysis
Q: Are users blocked for personal attacks pure trolls/vandals?
Methodology: Consider users blocked for harassment. Compute histogram over mean user level aggression and attack scores.
Aggression
End of explanation
d['blocked'].groupby('user_text')['pred_recipient_score']\
                 .agg( {'aggressiveness': np.mean}).hist(bins = 30)
plt.xlabel('user level attack score')
plt.ylabel('num users')
plt.title('')
d['blocked'].query('not author_anon').groupby('user_text')['pred_recipient_score']\
                 .agg( {'aggressiveness': np.mean}).hist(bins = 30)
plt.xlabel('user level attack score')
plt.ylabel('num users')
plt.title('')
d['2015'].query('not author_anon').groupby('user_text')['pred_recipient_score']\
    .agg( {'aggressiveness': np.mean})\
    .hist(bins = 50)
plt.xlabel('user level attack score')
plt.ylabel('num users')
plt.title('')
Explanation: Attack
End of explanation
# TODO
Explanation: Q: Do attacks come from pure trolls or from users who are generally non-attacking?
For different thresholds, assign each attack its users aggression score. Plot cdfs over aggression scores.
End of explanation
# TODO
Explanation: Victim Specific Analysis
Q Are victims also trolls?
End of explanation
o = (False, True)
x = 'author_anon'
compare_groups(d['sample'][:100000], x, order = o)
Explanation: Shared Analysis
Q: How do comments made by registered and anonymous authors compare?
End of explanation
# don't count posts to own article
o = (False, True)
x = 'recipient_anon'
compare_groups(d['sample'][:100000].query('not own_page'), x, order = o)
Explanation: Q: How do comments received by registered and anonymous authors compare?
End of explanation
x = 'own_page'
o = (False, True)
compare_groups(d['sample'][:100000], x, order = o)
x = 'own_page'
compare_groups(d['sample'][:100000], x, order = o, hue = 'author_anon')
Explanation: Q: How do authors write differently on their own page than on other pages?
End of explanation
d_gender = get_genders(d['sample'])
o = ('unknown: registered', 'male', 'female')
x = 'author_gender'
compare_groups(d_gender, x, order = o)
Explanation: Q: What is the effect of the author's gender?
End of explanation
o = ('unknown: registered', 'male', 'female')
x = 'recipient_gender'
compare_groups(d_gender.query('not own_page'), x, order= o)
Explanation: Q: What is the effect of the recipient's gender?
End of explanation
o = ('unknown: registered', 'male', 'female')
x = 'author_gender'
compare_groups(d_gender.query("not own_page and recipient_gender != 'unknown:anon'"), x, order = o, hue = 'recipient_gender')
Explanation: Q: How does the effect change when you interact author and recipient gender?
End of explanation
thresholds = np.percentile(d['2015']['user_text'].value_counts(),  np.arange(0, 100.01,0.5 ))
thresholds = sorted(set(thresholds.astype(int)))
bins = []
for i in range(len(thresholds)-1):
    label = '%d-%d' % (thresholds[i], thresholds[i+1]-1)
    rnge = range(thresholds[i], thresholds[i+1])
    bins.append((label, rnge))
def map_count(x):
    for label, rnge in bins:
        if x in rnge:
            return label
        
        
d_temp = d['2015'].query('not author_anon')\
                 .groupby('user_text')['pred_aggression_score']\
                 .agg( {'aggressiveness': np.mean, 'count': len})\
                 .assign(num_comment_range = lambda x: x['count'].apply(map_count))
o = [e[0] for e in bins]
sns.pointplot(x='num_comment_range', y= 'aggressiveness', data= d_temp, order = o)
# TODO: extend to attacks, use long term user data, repeat for victims
Explanation: Q: How does tone depend on the frequency of commenting?
Methodology: let the "aggressiveness" of a user be the averge aggression_score of all their comments. Compare aggression scores across groups of users based on how much the post
End of explanation
<END_TASK>