<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Three Pandas Tips For Noobs
For those new to Pandas, you'll learn a number of tips that will help with your data engineering and analysis tasks. You may find these buried in the documentation or StackOverflow posts, but I'm consolidating them here for you.
Here's what's covered
Step1:  1. Ensuring Your Changes Stick
There are many ways to fill in missing (NaN) values in a DataFrame; some people use the mean of the column, others enter 0. You can do whatever you want. However, just because you tell Pandas to fill in the missing values doesn't mean the change will stick.
Let's use the fillna() method of the DataFrame and see what happens.
Step2:  Hrm, it looks like the DataFrame is updated, but is it? I think not!
Step3:  What the heck?! The missing values haven't actually been updated. So how do we make the change stick? Using the inplace=True argument like so...
Step4:  Success! The DataFrame has now been updated.
2. Applying a Function With No Arguments to a DataFrame
One of the reasons Pandas rocks is that you can apply a function to either a single column of a DataFrame or an entire DataFrame, using the apply() function. You'll be using this often, so here's how.
Step6:  According to Pandas, the Date is an object, meaning it doesn't actually see it as a date. Let's change that.
Step8:  Voila! Our data column is now a datetime.
3. Applying a Function With Arguments to a DataFrame
Along with applying a function to a single column, another common task is to create an additional column based on the values in two or more columns. In order to do that, we need create a function that takes multiple parameters, and then apply it to the DataFrame.
We'll be using the same apply() function we used in the previous tip, plus a little lambda magic.
<ASSISTANT_TASK:>
Python Code:
# Import the Python libraries we need
import pandas as pd
# Define a variable for the accidents data file
f = './data/accidents1k.csv'
# Use read_csv() to import the data
accidents = pd.read_csv(f,
                        sep=',',
                        header=0,
                        index_col=False,
                        parse_dates=True,
                        tupleize_cols=False,
                        error_bad_lines=False,
                        warn_bad_lines=True,
                        skip_blank_lines=True,
                        low_memory=False
                        )
# Run the head() command to see the top 5 rows of the data
accidents.head()
Explanation: Three Pandas Tips For Noobs
For those new to Pandas, you'll learn a number of tips that will help with your data engineering and analysis tasks. You may find these buried in the documentation or StackOverflow posts, but I'm consolidating them here for you.
Here's what's covered:
Ensuring changes you make to DataFrames stick
Applying a function with no arguments to a DataFrame
Applying a function with arguments to a DataFrame
Here's the link to the original dataset we're using:
UK Road Safety Data
Additional Resources
This and much more is covered in my upcoming book: Python Business Intelligence Cookbook, now available for pre-order from Packt Publishing.
Import The Data
The first thing we need to do is import the data into a DataFrame. I suggest using the read_csv() method from Pandas for this.
End of explanation
# Fill in the NaN values and check the DataFrame
accidents.fillna(value=0).head()
Explanation: 1. Ensuring Your Changes Stick
There are many ways to fill in missing (NaN) values in a DataFrame; some people use the mean of the column, others enter 0. You can do whatever you want. However, just because you tell Pandas to fill in the missing values doesn't mean the change will stick.
Let's use the fillna() method of the DataFrame and see what happens.
End of explanation
accidents.head()
Explanation: Hrm, it looks like the DataFrame is updated, but is it? I think not!
End of explanation
# Fill the NaN values and ensure the DataFrame is indeed updated.
accidents.fillna(value=0,
                 inplace=True)
accidents.head()
Explanation: What the heck?! The missing values haven't actually been updated. So how do we make the change stick? Using the inplace=True argument like so...
End of explanation
# Let's take a look at the Date column
accidents['Date'].head()
Explanation: Success! The DataFrame has now been updated.
2. Applying a Function With No Arguments to a DataFrame
One of the reasons Pandas rocks is that you can apply a function to either a single column of a DataFrame or an entire DataFrame, using the apply() function. You'll be using this often, so here's how.
End of explanation
# Define a function to convert a string to a date.
def convert_string_to_date(s):
    
    Given a string, use the to_datetime function of Pandas to convert
    it to a datetime, and then return it.
    
    return pd.to_datetime(s)
# Apply the function to the Data column using the apply() function.
# Note: we do not have to explicitly pass in the value in the row being processed.
accidents['Date'] = accidents['Date'].apply(convert_string_to_date)
# Let's check it out.
accidents['Date'].head()
Explanation: According to Pandas, the Date is an object, meaning it doesn't actually see it as a date. Let's change that.
End of explanation
# Create a few dicts and a DataFrame to hold the mappings for the accident data
# Accident severity
severity = {
    1: 'fatal',
    2: 'serious',
    3: 'fairly serious'
}
# Day of Week
days_of_week = {
    1: 'Sunday',
    2: 'Monday',
    3: 'Tuesday',
    4: 'Wednesday',
    5: 'Thursday',
    6: 'Friday',
    7: 'Saturday',
    0: 'Earlier this week'
}
# Road surfaces, updated to fit the sensationalism of a news broadcast
road_surfaces = {
    1: 'dry',
    2: 'wet',
    3: 'snow-covered',
    4: 'frosty',
    5: 'flooded',
    6: 'oily',
    7: 'muddy',
    -1: 'Data missing or out of range',
}
# Local Authority (District) - create a DataFrame from the CSV file
f = './data/accidents1k.csv'
# Use read_csv() to create a DataFrame from the local_authority_district mapping tab of the data dictionary.
# There are almost 1000 districts, hence I put them into a CSV file.
districts = pd.read_csv('./data/local_authority_district.csv',
                        sep=',',
                        header=0,
                        index_col=0,
                        parse_dates=False,
                        tupleize_cols=False,
                        error_bad_lines=False,
                        warn_bad_lines=True,
                        skip_blank_lines=True,
                        low_memory=False
                        )
# Define a function to create a one-sentence summary of the record.
def create_summary(day_of_week, accident_severity, road_surface, local_authority_district):
    
    Create a one-sentence summary of the record.
    Parameters: integer values for the Day_of_Week, Accident_Severity,
                Road_Surface_Conditions and Local_Authority_(District) columns
    
    
    # Perform the value lookups in the dicts and DataFrame
    dow = days_of_week[day_of_week]
    sev = severity[accident_severity]
    road = road_surfaces[road_surface]
    lad = districts.loc[local_authority_district].label
    
    # If the day of week was specified use the first sentence variation, otherwise use the second
    # Yes, this is redundant and we could optimize it. I leave that to you!
    if day_of_week != 0:
        return "On {} a {} accident occured on a {} road in {}".format(dow, sev, road, lad)
    else:
        return "{} a {} accident occured on a {} road in {}".format(dow, sev, road, lad)
# Create a new column in the DataFrame and fill it with the summary produced by the create_summary function
# Pass in the parameters needed to create the summary
accidents['summary'] = accidents.apply(lambda x: create_summary(x['Day_of_Week'],
                                                                x['Accident_Severity'],
                                                                x['Road_Surface_Conditions'],
                                                                x['Local_Authority_(District)']), axis=1)
# Let's see some results!
accidents['summary'].head()
# Let's view an entire summary
accidents['summary'][0]
Explanation: Voila! Our data column is now a datetime.
3. Applying a Function With Arguments to a DataFrame
Along with applying a function to a single column, another common task is to create an additional column based on the values in two or more columns. In order to do that, we need create a function that takes multiple parameters, and then apply it to the DataFrame.
We'll be using the same apply() function we used in the previous tip, plus a little lambda magic.
End of explanation
<END_TASK>