<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 First look at Data
We will look at the location data-set from the vast challenge 2015.
This initial exploration will be accomplished using the following tools
Step1:  1. Read the data
Step2:  Let's look at the first five rows
Step3:  What is the size of the table?
Step4:  What are the types of the data?
Step5:  What are the values of type ?
Step6:  How many different ids are there?
Step7:  What is the type of the timestamps?
Step8:  They are strings, it would be better if they were dats, lets fix that
with the to_datetime function
Step9:  Now the time column contains datetime objects
2. Looking at location data
First, take a random subsample to speed up exploration
Step10:  We will now create a simple scatter plot with all the X and Y values in our subsample
Step11:  It looks very similar to the pats in the map
Now lets look at just the check-in samples
Step12:  Lets look at the range of the location data
Step13:  Now lets create a 2d histogram to see which locations are more popular. We will use the hist2d function
Step14:  We can increase the number of bins
Step15:  3. Single guest
Now lets plot the locations for a single random person
Step16:  Now lets try to use the time information
Step17:  At what time did he arrive?
Step18:  At what time did he leave?
Step19:  So how long did he stay?
Step20:  4. Single time frame
Where were the guests between 12
Step21:  lets add some jitter
Step22:  5. Time analysis
Now lets try to ask some simple questions about time data
At what time do guests arrive?
At what time do they leave?
How long they stay?
How does park ocupacy vary during the day?
To answer the first questions we needd to transform the data
Step23:  Now for the question of park occupacy, we need to group the dataframe by time
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
%matplotlib inline
import seaborn as sns
from matplotlib import pyplot as plt
import numpy as np
sns.set_style("darkgrid")
%cd C:\Users\Profesor\Documents\curso_va_2015\va_course_2015
Explanation: First look at Data
We will look at the location data-set from the vast challenge 2015.
This initial exploration will be accomplished using the following tools:
ipython notebook
Pandas
Seaborn 
numpy
0. Setup environment
End of explanation
df = pd.read_csv("../MC1 2015 Data/park-movement-Fri.csv")
Explanation: 1. Read the data
End of explanation
df.head()
Explanation: Let's look at the first five rows
End of explanation
df.shape
Explanation: What is the size of the table?
End of explanation
df.dtypes
Explanation: What are the types of the data?
End of explanation
df["type"].unique()
df.groupby("type")["type"].count()
Explanation: What are the values of type ?
End of explanation
df["id"].unique().shape
pd.pivot_table(df,columns="type", values="X", index="id", aggfunc=len).head()
pd.pivot_table(df,columns="type", values="X", index="id", aggfunc=len).mean()
Explanation: How many different ids are there?
End of explanation
type(df.Timestamp[0])
Explanation: What is the type of the timestamps?
End of explanation
df["time"] = pd.to_datetime(df.Timestamp, format="%Y-%m-%d %H:%M:%S")
df.tail()
df.dtypes
Explanation: They are strings, it would be better if they were dats, lets fix that
with the to_datetime function
End of explanation
df_small = df.sample(10000)
df_small.shape
Explanation: Now the time column contains datetime objects
2. Looking at location data
First, take a random subsample to speed up exploration
End of explanation
df_small.plot("X","Y","scatter")
Explanation: We will now create a simple scatter plot with all the X and Y values in our subsample
End of explanation
df_small.loc[df_small["type"]=="check-in"].plot("X","Y","scatter")
Explanation: It looks very similar to the pats in the map
Now lets look at just the check-in samples
End of explanation
df["X"].min()
df["X"].max()
df["Y"].min()
df["Y"].max()
Explanation: Lets look at the range of the location data
End of explanation
cnts, xe, ye, img = plt.hist2d(df_small["X"], df_small["Y"],range=((0,100),(0,100)),normed=True)
Explanation: Now lets create a 2d histogram to see which locations are more popular. We will use the hist2d function
End of explanation
cnts, xe, ye, img = plt.hist2d(df_small["X"], df_small["Y"],range=((0,100),(0,100)),normed=True, bins=20)
df_small.plot("X","Y","hexbin")
Explanation: We can increase the number of bins
End of explanation
guest_id = np.random.choice(df["id"])
guest_df = df.loc[df["id"]==guest_id]
guest_df.shape
guest_df.plot("X","Y","scatter")
Explanation: 3. Single guest
Now lets plot the locations for a single random person
End of explanation
plt.scatter(guest_df["X"],guest_df["Y"],c=guest_df["time"])
Explanation: Now lets try to use the time information
End of explanation
guest_df["time"].min()
Explanation: At what time did he arrive?
End of explanation
guest_df["time"].max()
Explanation: At what time did he leave?
End of explanation
guest_df["time"].max() - guest_df["time"].min()
Explanation: So how long did he stay?
End of explanation
noon_dates = (df["time"] < '2014-06-06 12:05:00') & (df["time"] >= '2014-06-06 12:00:00')
noon_df = df.loc[noon_dates]
noon_df.shape
plt.scatter(noon_df["X"], noon_df["Y"], alpha=0.01, marker="o", s=30)
Explanation: 4. Single time frame
Where were the guests between 12:00 and 12:05 ?
End of explanation
plt.scatter(noon_df["X"] +5*np.random.random(len(noon_df))
           ,noon_df["Y"]+5*np.random.random(len(noon_df)),
            alpha=0.01, marker="o", s=30)
Explanation: lets add some jitter
End of explanation
grouped_times = df.groupby("id")["time"]
arrivals = grouped_times.min()
departures = grouped_times.max()
duration = departures - arrivals
sns.distplot(arrivals.dt.hour+arrivals.dt.minute/60)
sns.distplot(departures.dt.hour+departures.dt.minute/60)
h_duration = duration.dt.seconds/60/60
sns.distplot(h_duration)
Explanation: 5. Time analysis
Now lets try to ask some simple questions about time data
At what time do guests arrive?
At what time do they leave?
How long they stay?
How does park ocupacy vary during the day?
To answer the first questions we needd to transform the data
End of explanation
time_groups = df.groupby(df.time.dt.hour)
occupancy = time_groups["id"].aggregate(lambda x:len(np.unique(x)))
occupancy.plot()
Explanation: Now for the question of park occupacy, we need to group the dataframe by time
End of explanation
<END_TASK>