<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Following NY Times article on Gaps in Earnings Stand Out in Release of College Data
I decided to do some exploration myself
Download data
Step1:  lets explore
Step2:  for some reason 2011 was the last year for which there is earning information
Step3:  the number of schools covered
Step4:  for each there are lots of columns to read
Step5:  but there is a dictionary just for exploring what each column is
Step6:  the columns are grouped
Step7:  there are many columns in the earning category
Step8:  and here is the field for male earning 10 years after finishing school. Yes. females are male0
Step9:  compute the difference between women and men
Step10:  results
<ASSISTANT_TASK:>
Python Code:
%%bash
cd ~/Downloads
wget https://s3.amazonaws.com/ed-college-choice-public/CollegeScorecard_Raw_Data.zip
unzip CollegeScorecard_Raw_Data.zip
Explanation: Following NY Times article on Gaps in Earnings Stand Out in Release of College Data
I decided to do some exploration myself
Download data
End of explanation
!ls ~/Downloads/CollegeScorecard_Raw_Data
Explanation: lets explore
End of explanation
import pandas as pd
df = pd.read_csv('~/Downloads/CollegeScorecard_Raw_Data/MERGED2011_PP.csv', na_values=['PrivacySuppressed'])
Explanation: for some reason 2011 was the last year for which there is earning information
End of explanation
len(df)
Explanation: the number of schools covered
End of explanation
len(df.columns)
Explanation: for each there are lots of columns to read
End of explanation
ddict = pd.read_csv('~/Downloads/CollegeScorecard_Raw_Data/CollegeScorecardDataDictionary-09-12-2015.csv')
Explanation: but there is a dictionary just for exploring what each column is
End of explanation
ddict['dev-category'].unique()
Explanation: the columns are grouped
End of explanation
pd.options.display.max_colwidth = 87
ddict[ddict['dev-category'] == 'earnings'].set_index('VARIABLE NAME')['NAME OF DATA ELEMENT']
Explanation: there are many columns in the earning category
End of explanation
ddict[ddict['VARIABLE NAME'] == 'mn_earn_wne_male1_p10'].values
Explanation: and here is the field for male earning 10 years after finishing school. Yes. females are male0
End of explanation
df['diffp10'] = df.mn_earn_wne_male1_p10 - df.mn_earn_wne_male0_p10
df = df.sort(columns=['diffp10'], ascending=False)
for name in ['Massachusetts Institute of Technology', 'Stanford University']:
    print name, df[df.INSTNM == name].diffp10.values[0]
Explanation: compute the difference between women and men
End of explanation
df[['INSTNM','diffp10','mn_earn_wne_male0_p10','mn_earn_wne_male1_p10']].set_index('INSTNM').head(30)
Explanation: results
End of explanation
<END_TASK>