<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 The following dictionary contains hand-curated labeled domains.
Step1:  The following scripts gathers generic email hosts from a list provided on a public Gist.
Step2:  <hr>
<ASSISTANT_TASK:>
Python Code:
domain_categories = {
    "generic" : [
        "gmail.com",
        "hotmail.com",
        "gmx.de",
        "gmx.net",
        "gmx.at",
        "earthlink.net",
        "comcast.net",
        "yahoo.com",
        "email.com"
    ],
    "personal" : [
        "mnot.net",
        "henriknordstrom.net",
        "adambarth.com",
        "brianrosen.net",
        "taugh.com",
        "csperkins.org",
        "sandelman.ca",
        "lowentropy.net",
        "gbiv.com"
    ],
    "company" : [
        "apple.com",
        "cisco.com",
        "chromium.org",
        "microsoft.com",
        "oracle.com",
        "google.com",
        "facebook.com",
        "intel.com",
        "verizon.com",
        "verizon.net",
        "salesforce.com",
        "cloudflare.com",
        "broadcom.com",
        "juniper.net",
        "netflix.com",
        "akamai.com",
        "us.ibm.com",
        "qualcomm.com",
        "siemens.com",
        "boeing.com",
        "sandvine.com",
        "marconi.com",
        "trilliant.com",
        "huawei.com", # chinese
        "zte.com.cn" # chinese
        "chinamobile.com",
        "chinaunicom.cn",
        "chinatelecom.cn",
        "cnnic.cn" # registry
    ],
    # from R.N.
    "academic" : [
        "caict.ac.cn", # chinese
        "scu.edu.cn", # chinese
        "tongji.edu.cn", # chinese
        "mit.edu",
        "ieee.org",
        "acm.org",
        "berkeley.edu",
        "harvard.edu",
        "lbl.gov"
    ],
    "sdo" : [
        "isoc.org",
        "icann.org",
        "amsl.com",
        "iana.org",
        "tools.ietf.org",
        "w3.org"
    ]
}
df = pd.DataFrame.from_records(itertools.chain(
    *[[{'domain' : dom, 'category' : cat} 
      for dom in domain_categories[cat]] 
     for cat in domain_categories]
))
Explanation: The following dictionary contains hand-curated labeled domains.
End of explanation
!wget https://gist.githubusercontent.com/ammarshah/f5c2624d767f91a7cbdc4e54db8dd0bf/raw/660fd949eba09c0b86574d9d3aa0f2137161fc7c/all_email_provider_domains.txt
aepd = open("all_email_provider_domains.txt").read().split("\n")
df = df.append([{'domain' : d, 'category' : 'generic'} for d in aepd]).drop_duplicates(subset=['domain'])
df.to_csv("domain_categories.csv", index=False)
Explanation: The following scripts gathers generic email hosts from a list provided on a public Gist.
End of explanation
pd.read_csv("domain_categories.csv", index_col='domain')
Explanation: <hr>
End of explanation
<END_TASK>