<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Get draft data
First, get the sample drafts for a range of dates.
Step1:  Create a table for the group and affiliation links in particular.
Step2:  Entity resolution on the affiliations
Using containment distance, collapse the entities in the affilation column and removed suspected duplicates.
Step3:  Plot the network links between working groups and affiliations
Step4:  Look at categories of affiliations
<ASSISTANT_TASK:>
Python Code:
window = {
    'since' : "2018-01-01T00:00:00",
    'until' : "2018-01-10T00:00:00"
}
dt = DataTracker()
def extract_data(doc):
    data = {}
    ## TODO: Add document UID?
    data['title'] = doc.title
    data['time'] = doc.time
    data['group-acronym'] = dt.group(doc.group).acronym
    data['type'] = doc.type.uri
    affiliations = [
        doc_author.affiliation
        for doc_author
        in dt.document_authors(doc)
    ]
    
    return [
        {**data,
         'affiliation' : affiliation}
        for affiliation
        in affiliations]
drafts = dt.documents(doctype = dt.document_type(
                          DocumentTypeURI("/api/v1/name/doctypename/draft")),
                      **window
                     )
data = [item for sublist in [extract_data(x) for x in drafts] for item in sublist]
draft_df = pd.DataFrame(data=data)
draft_df.head()
Explanation: Get draft data
First, get the sample drafts for a range of dates.
End of explanation
link_df = draft_df[['group-acronym', 'affiliation','time']]
link_df[:5]
Explanation: Create a table for the group and affiliation links in particular.
End of explanation
all_affiliations = link_df.groupby('affiliation').size()
ents = process.resolve_entities(all_affiliations,
                                process.containment_distance,
                                threshold=.25)
replacements = {}
for r in [{name: ent for name in ents[ent]} for ent in ents]:
    replacements.update(r)
    
link_df = link_df.replace(to_replace=replacements)
Explanation: Entity resolution on the affiliations
Using containment distance, collapse the entities in the affilation column and removed suspected duplicates.
End of explanation
edges = [
 (row[1]['group-acronym'], row[1]['affiliation'])
 for row
 in link_df[['group-acronym','affiliation']].iterrows()
]
G = nx.Graph()
G.add_nodes_from([x[0]
                  for x
                  in link_df[['group-acronym']].drop_duplicates().values],
                 category=0)
G.add_nodes_from(all_affiliations.index,
                 category=1)
G.add_edges_from(edges)
## Clean the graph
G.remove_node('none')
for c in list(nx.connected_components(G)):
    if len(c) <= 1:
        for n in c:
            G.remove_node(n)
colors = ['r' if x[1]['category'] else 'g' for x in list(G.nodes(data=True))]
nx.draw(
    G,
    node_color = colors,
    with_labels = True
)
nx.write_gexf(G,"group-affiliations.gexf")
Explanation: Plot the network links between working groups and affiliations
End of explanation
import bigbang.datasets.organizations as organizations
cat = organizations.load_data()
replacements = {x[1]['name'] : x[1]['category']
                for x
                in cat.iterrows()
                if not pd.isna(x[1]['category'])}
link_cat_df = link_df.replace(to_replace=replacements)
link_cat_df.groupby('affiliation').size().sort_values(ascending=False)[:40]
edges = [
 (row[1]['group-acronym'], row[1]['affiliation'])
 for row
 in link_cat_df[['group-acronym','affiliation']].iterrows()
]
affils = link_cat_df.groupby('affiliation').size().index
G = nx.Graph()
G.add_nodes_from([x[0]
                  for x
                  in link_cat_df[['group-acronym']].drop_duplicates().values],
                 category=0)
G.add_nodes_from(affils,
                 category=1)
G.add_edges_from(edges)
## Clean the graph
G.remove_node('none')
for c in list(nx.connected_components(G)):
    if len(c) <= 1:
        for n in c:
            G.remove_node(n)
[x[1] for x in list(G.nodes(data=True)) if 'category' not in x[1]]
colors = ['r' if x[1]['category'] else 'g' for x in list(G.nodes(data=True))]
nx.draw(
    G,
    node_color = colors,
    with_labels = True
)
nx.write_gexf(G,"group-org-categories.gexf")
Explanation: Look at categories of affiliations
End of explanation
<END_TASK>