<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Loading data
Simple stuff. We're loading in a CSV here, and we'll run the describe function over it to get the lay of the land.
Step1:  In journalism, we're primarily concerned with using data analysis for two purposes
Step2:  One record isn't super useful, so we'll do 10
Step3:  If we want, we can keep drilling down. Maybe we should also limit our inquiry to, say, La Guardia.
Step4:  Huh, does LGA struggle more than usual to get its planes to Atlanta on time? Let's live dangerously and make a boxplot.
(Spoiler alert
Step5:  And so on.
Of course data journalists are also in the business of finding trends, so let's do some of that.
Describing trends
Being good, accountability-minded reporters, one thing we might be interested in is each airline's on-time performance throughout our sample. Here's one way to check that
Step6:  Huh. Looks like the median flight from most of these carriers tends to show up pretty early. How does that change when we look at the mean?
Step7:  A little less generous. We can spend some time debating which portrayal is more fair, but the large difference between the two is still worth noting.
We can, of course, also drill down by destination
Step8:  And if we want a more user-friendly display ...
Step9:  BONUS! Correlation
Up until now, we've spent a lot of time seeing how variables act in isolation -- mainly focusing on arrival delays. But sometimes we might also want to see how two variables interact. That's where correlation comes into play.
For example, let's test one of my personal suspicions that longer flights (measured in distance) tend to experience longer delays.
Step10:  And now we'll make a crude visualization, just to show off
<ASSISTANT_TASK:>
Python Code:
df = pd.read_csv('data/ontime_reports_may_2015_ny.csv')
df.describe()
Explanation: Loading data
Simple stuff. We're loading in a CSV here, and we'll run the describe function over it to get the lay of the land.
End of explanation
df.sort('ARR_DELAY', ascending=False).head(1)
Explanation: In journalism, we're primarily concerned with using data analysis for two purposes:
Finding needles in haystacks
And describing trends
We'll spend a little time looking at the first before we move on to the second.
Needles in haystacks
Let's start with the longest delays:
End of explanation
df.sort('ARR_DELAY', ascending=False).head(10)
Explanation: One record isn't super useful, so we'll do 10:
End of explanation
la_guardia_flights = df[df['ORIGIN'] == 'LGA']
la_guardia_flights.sort('ARR_DELAY', ascending=False).head(10)
Explanation: If we want, we can keep drilling down. Maybe we should also limit our inquiry to, say, La Guardia.
End of explanation
lga_to_atl = df[df['DEST'] == 'ATL']
lga_to_atl.boxplot('ACTUAL_ELAPSED_TIME', by='ORIGIN')
Explanation: Huh, does LGA struggle more than usual to get its planes to Atlanta on time? Let's live dangerously and make a boxplot.
(Spoiler alert: JFK is marginally worse)
End of explanation
df.groupby('CARRIER').median()['ARR_DELAY']
Explanation: And so on.
Of course data journalists are also in the business of finding trends, so let's do some of that.
Describing trends
Being good, accountability-minded reporters, one thing we might be interested in is each airline's on-time performance throughout our sample. Here's one way to check that:
End of explanation
df.groupby('CARRIER').mean()['ARR_DELAY']
Explanation: Huh. Looks like the median flight from most of these carriers tends to show up pretty early. How does that change when we look at the mean?
End of explanation
df.groupby(['CARRIER', 'ORIGIN']).median()['ARR_DELAY']
Explanation: A little less generous. We can spend some time debating which portrayal is more fair, but the large difference between the two is still worth noting.
We can, of course, also drill down by destination:
End of explanation
df.boxplot('ARR_DELAY', by='CARRIER')
Explanation: And if we want a more user-friendly display ...
End of explanation
df.corr()
Explanation: BONUS! Correlation
Up until now, we've spent a lot of time seeing how variables act in isolation -- mainly focusing on arrival delays. But sometimes we might also want to see how two variables interact. That's where correlation comes into play.
For example, let's test one of my personal suspicions that longer flights (measured in distance) tend to experience longer delays.
End of explanation
import matplotlib.pyplot as plt
plt.matshow(df.corr())
Explanation: And now we'll make a crude visualization, just to show off:
End of explanation
<END_TASK>