<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 First, I made a mistake naming the data set! It's 2015 data, not 2014 data. But yes, still use 311-2014.csv. You can rename it.
Importing and preparing your data
Import your data, but only the first 200,000 rows. You'll also want to change the index to be a datetime based on the Created Date column - you'll want to check if it's already a datetime, and parse it if not.
Step1:  What was the most popular type of complaint, and how many times was it filed?
Step2:  Make a horizontal bar graph of the top 5 most frequent complaint types.
Step3:  Which borough has the most complaints per capita? Since it's only 5 boroughs, you can do the math manually.
Step4:  According to your selection of data, how many cases were filed in March? How about May?
Step5:  I'd like to see all of the 311 complaints called in on April 1st.
Surprise! We couldn't do this in class, but it was just a limitation of our data set
Step6:  What was the most popular type of complaint on April 1st?
What were the most popular three types of complaint on April 1st
Step7:  What month has the most reports filed? How many? Graph it.
Step8:  What week of the year has the most reports filed? How many? Graph the weekly complaints.
Step9:  Noise complaints are a big deal. Use .str.contains to select noise complaints, and make an chart of when they show up annually. Then make a chart about when they show up every day (cyclic).
Step10:  Which were the top five days of the year for filing complaints? How many on each of those days? Graph it.
Step11:  What hour of the day are the most complaints? Graph a day of complaints.
Step12:  One of the hours has an odd number of complaints. What are the most common complaints at that hour, and what are the most common complaints the hour before and after?
Step13:  So odd. What's the per-minute breakdown of complaints between 12am and 1am? You don't need to include 1am.
Step14:  Looks like midnight is a little bit of an outlier. Why might that be? Take the 5 most common agencies and graph the times they file reports at (all day, not just midnight).
Step16:  Graph those same agencies on an annual basis - make it weekly. When do people like to complain? When does the NYPD have an odd number of complaints?
Step17:  Maybe the NYPD deals with different issues at different times? Check the most popular complaints in July and August vs the month of May. Also check the most common complaints for the Housing Preservation Bureau (HPD) in winter vs. summer.
<ASSISTANT_TASK:>
Python Code:
#df = pd.read_csv("small-311-2015.csv")
df = pd.read_csv("311-2014.csv", nrows=200000)
df.head(2)
df.info()
def parse_date (str_date):
    return dateutil.parser.parse(str_date)
df['created_dt']= df['Created Date'].apply(parse_date)
df.head(3)
df.info()
Explanation: First, I made a mistake naming the data set! It's 2015 data, not 2014 data. But yes, still use 311-2014.csv. You can rename it.
Importing and preparing your data
Import your data, but only the first 200,000 rows. You'll also want to change the index to be a datetime based on the Created Date column - you'll want to check if it's already a datetime, and parse it if not.
End of explanation
df["Complaint Type"].value_counts().head(1)
Explanation: What was the most popular type of complaint, and how many times was it filed?
End of explanation
df["Complaint Type"].value_counts().head(5).sort_values().plot(kind='barh')
Explanation: Make a horizontal bar graph of the top 5 most frequent complaint types.
End of explanation
df["Borough"].value_counts()
people_bronx= 1438159
people_queens= 2321580
people_manhattan=1636268
people_brooklyn= 2621793
people_staten_island= 473279
complaints_per_capita_bronx= 29610/people_bronx
complaints_per_capita_bronx
complaints_per_capita_queens=46824/people_queens
complaints_per_capita_queens
complaints_per_capita_manhattan=42050/people_manhattan
complaints_per_capita_manhattan
complaints_per_capita_staten_island=473279/people_staten_island
complaints_per_capita_staten_island
complaints_per_capita_brooklyn=2621793/people_brooklyn
complaints_per_capita_brooklyn
Explanation: Which borough has the most complaints per capita? Since it's only 5 boroughs, you can do the math manually.
End of explanation
df.index = df['created_dt']
#del df['Created Date']
df.head()
print("There were", len(df['2015-03']), "cases filed in March")
print("There were", len(df['2015-05']), "cases filed in May")
Explanation: According to your selection of data, how many cases were filed in March? How about May?
End of explanation
df['2015-04-01']
Explanation: I'd like to see all of the 311 complaints called in on April 1st.
Surprise! We couldn't do this in class, but it was just a limitation of our data set
End of explanation
df['2015-04-01']['Complaint Type'].value_counts().head(3)
df.info()
Explanation: What was the most popular type of complaint on April 1st?
What were the most popular three types of complaint on April 1st
End of explanation
df.resample('M').count()
df.resample('M').index[0]
import numpy as np
np.__version__
df.resample('M').count().plot(y="Unique Key")
ax= df.groupby(df.index.month).count().plot(y='Unique Key', legend=False)
ax.set_xticks([1,2,3,4,5,6,7,8,9,10,11, 12])
ax.set_xticklabels(['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'])
ax.set_ylabel("Number of Complaints")
ax.set_title("311 complains in 2015")
Explanation: What month has the most reports filed? How many? Graph it.
End of explanation
#df.resample('W').count().head(5)
df.resample('W').count().plot(y="Unique Key", color= "purple")
Explanation: What week of the year has the most reports filed? How many? Graph the weekly complaints.
End of explanation
df[df['Complaint Type'].str.contains("Noise")].head()
noise_df= df[df['Complaint Type'].str.contains("Noise")]
noise_graph= noise_df.groupby(noise_df.index.month).count().plot(y='Unique Key', legend=False)
noise_graph.set_xticks([1,2,3,4,5,6,7,8,9,10,11, 12])
noise_graph.set_xticklabels(['Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec'])
noise_graph.set_ylabel("Number of Noise Complaints")
noise_graph.set_title("311 noise complains in 2015")
noise_df.groupby(by=noise_df.index.hour)['Unique Key'].count().plot()
noise_graph= noise_df.groupby(noise_df.index.dayofweek).count().plot(y='Unique Key', legend=False)
noise_graph.set_xticks([1,2,3,4,5,6,7])
noise_graph.set_xticklabels(['Mon', 'Tues', 'Wed', 'Thur', 'Fri', 'Sat', 'Sun'])
noise_graph.set_ylabel("Number of Noise Complaints")
noise_graph.set_title("311 noise complains in 2015")
Explanation: Noise complaints are a big deal. Use .str.contains to select noise complaints, and make an chart of when they show up annually. Then make a chart about when they show up every day (cyclic).
End of explanation
daily_count= df['Unique Key'].resample('D').count().sort_values(ascending=False)
top_5_days= daily_count.head(5)
top_5_days
ax = top_5_days.plot(kind='bar') # I dont know how to put names to the labels
ax.set_title("Top 5 days")
ax.set_xlabel("Day")
ax.set_ylabel("Complaints")
Explanation: Which were the top five days of the year for filing complaints? How many on each of those days? Graph it.
End of explanation
hour_graph= df.groupby(df.index.hour).count().plot(y='Unique Key', legend=False)
hour_graph.set_xticks([0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23])
hour_graph.set_title("A day of complaints")
hour_graph.set_xlabel("Hours")
hour_graph.set_ylabel("Complaints")
Explanation: What hour of the day are the most complaints? Graph a day of complaints.
End of explanation
twelve_am_complaints= df[df.index.hour <1]
twelve_am_complaints.head()
twelve_am_complaints['Complaint Type'].value_counts().head(5)
one_am_complaints= df[df.index.hour == 1]
one_am_complaints['Complaint Type'].value_counts().head(5)
eleven_pm_complaints= df[df.index.hour == 23]
eleven_pm_complaints['Complaint Type'].value_counts().head(5)
Explanation: One of the hours has an odd number of complaints. What are the most common complaints at that hour, and what are the most common complaints the hour before and after?
End of explanation
twelve_am_complaints.groupby(twelve_am_complaints.index.minute).count()
Explanation: So odd. What's the per-minute breakdown of complaints between 12am and 1am? You don't need to include 1am.
End of explanation
df['Agency'].value_counts().head(5)
df_NYPD = df[df['Agency'] == 'NYPD']
df_HPD = df[df['Agency'] == 'HPD']
df_DOT = df[df['Agency'] == 'DOT']
df_DPR= df[df['Agency'] == 'DPR']
df_DOHMH= df[df['Agency'] == 'DOHMH']
all_graph = df_NYPD.groupby(by= df_NYPD.index.hour).count().plot(y='Unique Key', label='NYPD complaints')
all_graph.set_xticks([0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23])
all_graph.set_title("A day of complaints by the top 5 agencies")
all_graph.set_xlabel("Hours")
all_graph.set_ylabel("Complaints")
df_HPD.groupby(by= df_HPD.index.hour).count().plot(y='Unique Key', ax=all_graph , label='HPD complaints')
df_DOT.groupby(by= df_DOT.index.hour).count().plot(y='Unique Key', ax=all_graph , label='DOT complaints')
df_DPR.groupby(by= df_DPR.index.hour).count().plot(y='Unique Key', ax=all_graph , label='DPR complaints')
df_DOHMH.groupby(by= df_DOHMH.index.hour).count().plot(y='Unique Key', ax=all_graph , label='DOHMH complaints')
Explanation: Looks like midnight is a little bit of an outlier. Why might that be? Take the 5 most common agencies and graph the times they file reports at (all day, not just midnight).
End of explanation
all_graph = df_NYPD.groupby(by= df_NYPD.index.weekofyear).count().plot(y='Unique Key', label='NYPD complaints')
#all_graph.set_xticks([1,50])
all_graph.set_title("A year of complaints by the top 5 agencies")
all_graph.set_xlabel("Weeks")
ax.legend(loc='center left', bbox_to_anchor=(1, 0.5))
df_HPD.groupby(by= df_HPD.index.week).count().plot(y='Unique Key', ax=all_graph , label='HPD complaints')
df_DOT.groupby(by= df_DOT.index.hour).count().plot(y='Unique Key', ax=all_graph , label='DOT complaints')
df_DPR.groupby(by= df_DPR.index.hour).count().plot(y='Unique Key', ax=all_graph , label='DPR complaints')
df_DOHMH.groupby(by= df_DOHMH.index.hour).count().plot(y='Unique Key', ax=all_graph , label='DOHMH complaints')
plt.legend(bbox_to_anchor=(0, 1), loc='best', ncol=1)
print(May and June are the months with more complaints, followed by October, November and December. 
In May the NYPD and HPD have an odd number of complaints)
Explanation: Graph those same agencies on an annual basis - make it weekly. When do people like to complain? When does the NYPD have an odd number of complaints?
End of explanation
August_July = df["2015-07":"2015-08"]
August_July_complaints = August_July['Complaint Type'].value_counts().head(5)
August_July_complaints
May = df['2015-05']
May_complaints= May['Complaint Type'].value_counts().head(5)
May_complaints
# August_July_vs_May= August_July_complaints.plot(y='Unique Key', label='August - July complaints')
# August_July_vs_May.set_ylabel("Number of Complaints")
# August_July_vs_May.set_title("August-July vs May Complaints")
# May['Complaint Type'].value_counts().head(5).plot(y='Unique Key', ax=August_July_vs_May, label='May complaints')
# August_July_vs_May.set_xticks([1,2,3,4,5])
# August_July_vs_May.set_xticklabels(['Illegal Parking', 'Blocked Driveway', 'Noise - Street/Sidewalk', 'Street Condition', 'Noise - Commercial'])
#Most popular complaints of the HPD
df_HPD['Complaint Type'].value_counts().head(5)
summer_complaints= df_HPD["2015-06":"2015-08"]['Complaint Type'].value_counts().head(5)
summer_complaints
winter_complaints= df_HPD["2015-01":"2015-02"]['Complaint Type'].value_counts().head(5)
winter_complaints
winter_complaints_dec= df_HPD["2015-12"]['Complaint Type'].value_counts().head(5)
winter_complaints_dec
winter_results= df_HPD["2015-12"]['Complaint Type'].value_counts() + df_HPD["2015-01":"2015-02"]['Complaint Type'].value_counts()
winter_results
Explanation: Maybe the NYPD deals with different issues at different times? Check the most popular complaints in July and August vs the month of May. Also check the most common complaints for the Housing Preservation Bureau (HPD) in winter vs. summer.
End of explanation
<END_TASK>