<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Homework 11
Step1:  1. I want to make sure my Plate ID is a string. Can't lose the leading zeroes!
Step2:  2. I don't think anyone's car was built in 0AD. Discard the '0's as NaN.
Step3:  3. I want the dates to be dates! Read the read_csv documentation to find out how to make pandas automatically parse dates.
Step4:  4. "Date first observed" is a pretty weird column, but it seems like it has a date hiding inside. Using a function with .apply, transform the string (e.g. "20140324") into a Python date. Make the 0's show up as NaN.
Step5:  5. "Violation time" is... not a time. Make it a time.
Step6:  6. There sure are a lot of colors of cars, too bad so many of them are the same. Make "BLK" and "BLACK", "WT" and "WHITE", and any other combinations that you notice.
Step7:  7.Join the data with the Parking Violations Code dataset from the NYC Open Data site.
Step8:  8. How much money did NYC make off of parking violations?
Step9:  9. What's the most lucrative kind of parking violation? The most frequent?
Step10:  10. New Jersey has bad drivers, but does it have bad parkers, too? How much money does NYC make off of all non-New York vehicles?
Step11:  11. Make a chart of the top few.
Step12:  12. What time of day do people usually get their tickets? You can break the day up into several blocks - for example 12am-6am, 6am-12pm, 12pm-6pm,6pm-12am.
Step13:  14. Make a graph of the number of tickets per day.
Step14:  15. Make a graph of the amount of revenue collected per day.
Step15:  16. Manually construct a dataframe out of https
Step16:  17. What's the parking-ticket-$-per-licensed-driver in each borough of NYC? Do this with pandas and the dataframe you just made, not with your head!
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
Explanation: Homework 11: PARKING TICKETS
End of explanation
#As the datatypes in all of the columns vary, I decided to to make all the values, except for the ones I specify
#into str. This also takes care of questions 1. -> dtype=str
Explanation: 1. I want to make sure my Plate ID is a string. Can't lose the leading zeroes!
End of explanation
# The syntax for this is really very nice and clear, an example na_values= {'Vehicle Year' : ['0']}
# A lot more here: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html
Explanation: 2. I don't think anyone's car was built in 0AD. Discard the '0's as NaN.
End of explanation
#parse_dates=True,keep_date_col = True
import datetime
import datetime as dt
dt.datetime.strptime('08/04/2013', '%m/%d/%Y')
datetime.datetime(2013, 8, 4, 0, 0)
parser = lambda date: pd.datetime.strptime(date, '%m/%d/%Y')
#taken from here: http://stackoverflow.com/questions/23797491/parse-dates-in-pandas
#adding this, column counts starts at 0, without index, parse_dates=[4], date_parser=parser
#nrows=100000,
df = pd.read_csv("violations.csv", na_values= {'Vehicle Year' : ['0']}, parse_dates=[4], date_parser=parser, dtype=str)
df.head()
#df.info()
Explanation: 3. I want the dates to be dates! Read the read_csv documentation to find out how to make pandas automatically parse dates.
End of explanation
#df['Date First Observed'].value_counts()
import dateutil.parser
def date_transfer(x):
    x = str(x)
    if x == '0':
        return 'NaN'
    try:
        date = dateutil.parser.parse(x)
        return str(date.strftime('%d/%m/%Y'))
    except:
        return 'NaN'    
    #    dt.datetime.strptime('20130719', '%Y%m%d')
    #    datetime.datetime(2013, 7, 19, 0, 0)
    #    year_month_tag = dateutil.parser.parse(x)
    return x
print(date_transfer(20151198))
df['Date First Observed'].apply(date_transfer).head()
Explanation: 4. "Date first observed" is a pretty weird column, but it seems like it has a date hiding inside. Using a function with .apply, transform the string (e.g. "20140324") into a Python date. Make the 0's show up as NaN.
End of explanation
#adding n_values = {'Violation Time' : ['0']}
#df = pd.read_csv("violations.csv", na_values= {'Vehicle Year' : ['0'], 'Violation Time' : ['0']}, parse_dates=[4], date_parser=parser, dtype=str)
df['Violation Time'].head(5)
#df['Violation Time'].value_counts()
import dateutil
def pmam(x):
    x = str(x)
    #x = (':'.join(a+b for a,b in zip(x[::2], x[1::2])))
    if x == 'NaN':
        pass
    try:
        x = str(x[:2] + ':' + x[2:])
        date = dateutil.parser.parse(x)
        return str(date.strftime('%H:%M %p'))
    except:
        return 'NaN'
print(pmam('NaN'))
print(pmam("0752A"))
print(pmam("1240P"))
print(pmam("0232P"))
df['Violation Time pmam'] = df['Violation Time'].apply(pmam)
Explanation: 5. "Violation time" is... not a time. Make it a time.
End of explanation
df['Vehicle Color'].head(10)
def black_and_white(car_color):
    car_color = str(car_color)
    if car_color == 'NaN':
        return 'NaN'
    elif car_color == 'WHITE':
        return car_color
    elif car_color == 'BLACK':
        return car_color
    else:
        return car_color.replace("BK","BLACK").replace("WH", "WHITE")                                                                                                                                
def rainbow(car_color):
    car_color = str(car_color)
    if car_color == 'NaN':
        return car_color
    elif car_color == 'BROWN':
        return car_color
    elif car_color == 'BLUE':
        return car_color
    elif car_color == 'RED':
        return car_color
    elif car_color == 'GREY':
        return car_color
    elif car_color == 'GREEN':
        return car_color
    else:
        return car_color.replace("BR","BROWN").replace("RD", "RED").replace("GY", "GRAY").replace("TN", "TAN").replace("BLU", "BLUE").replace("GN", "GREEN")
df['Vehicle Color'].apply(black_and_white).apply(rainbow).head(5)
Explanation: 6. There sure are a lot of colors of cars, too bad so many of them are the same. Make "BLK" and "BLACK", "WT" and "WHITE", and any other combinations that you notice.
End of explanation
DOF_Parking_Violation_Codes_df = pd.read_csv("DOF_Parking_Violation_Codes.csv")
DOF_Parking_Violation_Codes_df.head()
df_merged = df.merge(DOF_Parking_Violation_Codes_df, left_on='Violation Code', right_on='CODE')
Explanation: 7.Join the data with the Parking Violations Code dataset from the NYC Open Data site.
End of explanation
def money_to_int(money_str):
    if money_str == '200 (Heavy Tow plus violation fine)':
        money_str_Heavy_Tow = money_str[:3]
        return int(money_str_Heavy_Tow)
    if money_str == '100\n(Regular Tow, plus violation fine)':
        money_str_Heavy_Tow = money_str[:3]
        return int(money_str_Heavy_Tow)
    try:
        return int(money_str.replace("$","").replace(",",""))
    except:
        return None
df_merged['All Other Areas int'] = df_merged['All Other Areas'].apply(money_to_int)
df_merged['Manhattan\xa0 96th St. & below int'] = df_merged['Manhattan\xa0 96th St. & below'].apply(money_to_int)
Mininum_Amount = df_merged['All Other Areas int'].sum()
print("NYC made at least: ${:,.2f} on parking fines.".format(Mininum_Amount))
Explanation: 8. How much money did NYC make off of parking violations?
End of explanation
def def_strip(definition):
    return definition.strip()
df_merged['DEFINITION_stripped'] = df_merged['DEFINITION'].apply(def_strip)
#Most frequent kind of parking violation
df_merged['DEFINITION'].value_counts().head(1)
count_by_violation_code = df_merged['CODE'].value_counts()
count_by_violation_code.head()
df_merged.groupby('CODE')['All Other Areas int'].sum().sort_values(ascending=False).head()
Most_lucrative_amount = df_merged.groupby('CODE')['All Other Areas int'].sum().sort_values(ascending=False)[1]
Most_lucrative_fine_in_words = df_merged[df_merged['CODE'] == '14']
Most_lucrative_fine_in_words = Most_lucrative_fine_in_words['DEFINITION'].head(1)
print("The most lucrative fine regarded ${:,.2f} on parking fines.".format(Most_lucrative_amount)) 
print("It regarded " + Most_lucrative_fine_in_words)
Explanation: 9. What's the most lucrative kind of parking violation? The most frequent?
End of explanation
df_merged['Registration State'].value_counts().head(10)
Fines_for_New_York_Cars = df_merged.groupby('Registration State')['All Other Areas int'].sum().sort_values(ascending=False).head(1)
Fines_for_New_York_Cars
Mininum_Amount #Total fines
print("New York City makes ${:,.2f} off non-New-York cars.".format(Mininum_Amount - 220943620.0)) 
#Double Check:
df_merged[df_merged['Registration State'] != 'NY'].sum()
Explanation: 10. New Jersey has bad drivers, but does it have bad parkers, too? How much money does NYC make off of all non-New York vehicles?
End of explanation
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use("ggplot")
df_merged['Registration State'].value_counts().sort_values(ascending=True).tail(10).plot(kind='barh')
Explanation: 11. Make a chart of the top few.
End of explanation
import dateutil
def Time_of_day(x):
    x = str(x)
    #x = (':'.join(a+b for a,b in zip(x[::2], x[1::2])))
    if x == 'NaN':
        pass
    try:
        x = str(x[:2] + ':' + x[2:])
        date = dateutil.parser.parse(x)
        time = int(str(date.strftime('%H')))
        if time <= 6:
            return '12am - 6am'
        elif time <= 12:
            return '6am - 12pm'
        elif time <= 18:
            return '12pm - 6pm'
        elif time <= 24:
            return '6pm - 12am'
    except:
        return 'NaN'
df_merged['Time of Day'] = df['Violation Time'].apply(Time_of_day)
df_merged['Time of Day'].value_counts()
Explanation: 12. What time of day do people usually get their tickets? You can break the day up into several blocks - for example 12am-6am, 6am-12pm, 12pm-6pm,6pm-12am.
End of explanation
df_merged['Issue Date'].head()
#Filtering out all the dates that were in the future and before 2013, obviously wrong dates.
df_merged_only_2013_dates = df_merged[(df_merged['Issue Date'] > '2013-01-01') & (df_merged['Issue Date'] < '2013-12-31')]
#More on histograms
#http://stackoverflow.com/questions/27365467/python-pandas-plot-histogram-of-dates
#http://stackoverflow.com/questions/29672375/histogram-in-matplotlib-time-on-x-axis
df_merged_only_2013_dates.groupby('Issue Date')['Issue Date'].count().plot(kind="bar", figsize=(15, 6))
Explanation: 14. Make a graph of the number of tickets per day.
End of explanation
df_merged['Issue Date'].head()
df_merged_only_2013_dates.groupby('Issue Date')['All Other Areas int'].sum().plot(kind="bar", figsize=(15, 6))
Explanation: 15. Make a graph of the amount of revenue collected per day.
End of explanation
df_boroughs = pd.read_csv("NYC_Boroughs.csv")
df_boroughs.head()
Explanation: 16. Manually construct a dataframe out of https://dmv.ny.gov/statistic/2015licinforce-web.pdf (only NYC boroughts - bronx, queens, manhattan, staten island, brooklyn), having columns for borough name, abbreviation, and number of licensed drivers.
End of explanation
df_merged['Violation County'].value_counts()
df_merged.groupby('Violation County')['All Other Areas int'].sum()
def boroughs_trans(borough):
    borough = str(borough)
    if borough == 'NaN':
        pass
    elif borough == 'RICH':
        return 'R'
    elif borough == 'BRONX':
        return 'BX'
    else:
        return None
Explanation: 17. What's the parking-ticket-$-per-licensed-driver in each borough of NYC? Do this with pandas and the dataframe you just made, not with your head!
End of explanation
<END_TASK>