<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Basics of Pandas
The last few weeks we have been using low-level methods to read data in to Python and manipulate it.  This week we will be exploring pandas to accelerate this process.
Pandas is based around the notion that arrays can be indexed in a flexible manner, and that we can structure our data access around the indexing labels.
We will start out, as we often do, by applying our boilerplate setup.
Step1:  Pandas provides a number of read_* options, including read_csv, which we will use here.
One important note about read_csv in particular is that there are over 50 possible arguments to it.  This allows for intensely flexible specification of how to read data in, how to parse it, and very detailed control over things like encoding of files and so forth.  This flexibility is designed to eliminate the need to pre-process any data files before importing, but it can also make for a complex import process if you only have to adjust a few columns.  We will use this in some of its more simple ways here.
Below, we read the building inventory file into an object called df (for Data Frame).
Step2:  One of the first things we can do is examine the columns that the dataframe has identified.
<ASSISTANT_TASK:>
Python Code:
%matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt
Explanation: Basics of Pandas
The last few weeks we have been using low-level methods to read data in to Python and manipulate it.  This week we will be exploring pandas to accelerate this process.
Pandas is based around the notion that arrays can be indexed in a flexible manner, and that we can structure our data access around the indexing labels.
We will start out, as we often do, by applying our boilerplate setup.
End of explanation
df = pd.read_csv("data-readonly/IL_Building_Inventory.csv")
Explanation: Pandas provides a number of read_* options, including read_csv, which we will use here.
One important note about read_csv in particular is that there are over 50 possible arguments to it.  This allows for intensely flexible specification of how to read data in, how to parse it, and very detailed control over things like encoding of files and so forth.  This flexibility is designed to eliminate the need to pre-process any data files before importing, but it can also make for a complex import process if you only have to adjust a few columns.  We will use this in some of its more simple ways here.
Below, we read the building inventory file into an object called df (for Data Frame).
End of explanation
df.columns
df.head()
df.tail()
df.describe()
df.dtypes
df.groupby(["Agency Name"])["Square Footage"].sum()
df["Agency Name"].value_counts()
df.describe()
df["Total Floors"].median()
df.median()
df.quantile([0.1, 0.2, 0.9])
df["Agency Name"].apply(lambda a: a.upper()).head()
df["Agency Name"].apply(lambda a: a).head()
"This is my string".lower()
"this is my string. here is another.".capitalize()
df = pd.read_csv("data-readonly/IL_Building_Inventory.csv", na_values={'Year Acquired': 0, 'Year Constructed': 0})
df.count()
df.iloc[10]
df.iloc[10]
df.loc[10, ["County", "Senate Dist"]]
year = df.groupby("Year Acquired")
df.index = df["Year Acquired"]
df.head()
df.loc[1970].head()
df.head()
df.loc[1974]
df.loc[0]
df = pd.read_csv("data-readonly/IL_Building_Inventory.csv",
                 na_values={'Year Acquired': 0, 'Year Constructed': 0})
df.index
df2 = df.set_index("Year Acquired")
df2.index
df2.loc[1975].head()
df2.iloc[[1974, 1975]]
keith = df.set_index("City")
keith.loc["Kinmundy"].describe()
names = ["date", "city", "state", "country", "shape", "duration_seconds", "duration_reported", "description", "report_date", "latitude", "longitude"]
ufo = pd.read_csv("data-readonly/ufo-scrubbed-geocoded-time-standardized.csv",
                 names = names, parse_dates = ["date", "report_date"])
ufo.dtypes
ufo.describe()
sum_seconds = ufo.groupby("state")["duration_seconds"].sum()
sum_seconds.sort_values() / (365*24*3600)
states = ufo.groupby("state")
states["duration_seconds"].mean()
ufo.loc[ ufo["duration_seconds"] > 900 , ["state", "duration_seconds", "shape"] ].groupby("state")["duration_seconds"].sum()
ufo["date"].min()
ufo["date"].max()
first_sighting = ufo.groupby("state")["date"].min()
last_sighting = ufo.groupby("state")["date"].max()
last_sighting - first_sighting
first_sighting.index
ufo["state"].nunique()
ufo["country"].unique()
ufo["country"] = ufo["country"].astype("category")
ufo["shape"] = ufo["shape"].astype("category")
ufo["state"] = ufo["state"].astype("category")
ufo["city"].nunique()
ufo.shape
ufo.groupby("city").count().nlargest(10, "date")
ufo.dtypes
shape_times = ufo.groupby("shape")["duration_seconds"].sum()
shape_times.index
shape_times.plot()
shape_times.sort_values().plot()
shape_times.nlargest(5)
shape_state = ufo.groupby(["state", "shape"])
times = shape_state["duration_seconds"].sum()
times.loc[ ["il", "mi", "oh"], ["sphere", "unknown"] ]
times.loc["il":"ok", "sphere":"unknown"]
unsorted_nonsense = times.sort_index()
unsorted_nonsense.loc["il":"ok"]
unsorted_nonsense.loc["il":"ok"].plot()
ufo.set_index("date", inplace=True)
ufo.resample("A")["duration_seconds"].sum()
myplot = ufo.resample("10A")["duration_seconds"].sum().plot()
myplot.set_yscale('log')
r = ufo.resample("10A")
r["duration_seconds"].sum()
ufo.resample("W")["duration_seconds"].sum()
day_of_week = ufo.index.dayofweek
ufo["day_of_week"] = day_of_week
ufo.groupby("day_of_week")["duration_seconds"].sum().plot()
ufo.groupby("state").sum().loc["tx"]
ufo.reset_index()
week = ufo.set_index( ["day_of_week", "state", "shape"] )
week.loc[, 'il', 'cigar']
new_week = week.sort_index()
ufo = ufo.reset_index()
ufo.index = ufo.date
ufo.index.dayofweek
Explanation: One of the first things we can do is examine the columns that the dataframe has identified.
End of explanation
<END_TASK>