<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Scientific programming with the SciPy stack
Pandas
Import libraries and check versions.
Step1:  Read the data and get a row count.  Data source
Step2:  SymPy
SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible.
Step3:  This example was gleaned from
Step4:  What is the probability that the temperature is actually greater than 33 degrees?
<img src="eq1.png">
We can use Sympy's integration engine to calculate a precise answer.
Step5:  Assume we now have a thermometer and can measure the temperature.  However, there is still uncertainty involved.
Step6:  We now have two measurements -- 30 +- 3 degrees and 26 +- 1.5 degrees.  How do we combine them?  30 +- 3 was our prior measurement.  We want to cacluate a better estimate of the temperature (posterior) given an observation of 26 degrees.
<img src="eq2.png">
<ASSISTANT_TASK:>
Python Code:
import pandas as pd
import numpy as np
import sys
print('Python version ' + sys.version)
print('Pandas version ' + pd.__version__)
print('Numpy version ' + np.__version__)
Explanation: Scientific programming with the SciPy stack
Pandas
Import libraries and check versions.
End of explanation
file_path = r'data\T100_2015.csv.gz'
df = pd.read_csv(file_path, header=0)
df.count()
df.head(n=10)
df = pd.read_csv(file_path, header=0, usecols=["PASSENGERS", "ORIGIN", "DEST"])
df.head(n=10)
print('Min: ', df['PASSENGERS'].min())
print('Max: ', df['PASSENGERS'].max())
print('Mean: ', df['PASSENGERS'].mean())
df = df.query('PASSENGERS > 10000')
print('Min: ', df['PASSENGERS'].min())
print('Max: ', df['PASSENGERS'].max())
print('Mean: ', df['PASSENGERS'].mean())
OriginToDestination = df.groupby(['ORIGIN', 'DEST'], as_index=False).agg({'PASSENGERS':sum,})
OriginToDestination.head(n=10)
OriginToDestination = pd.pivot_table(OriginToDestination, values='PASSENGERS', index=['ORIGIN'], columns=['DEST'], aggfunc=np.sum)
OriginToDestination.head()
OriginToDestination.fillna(0)
Explanation: Read the data and get a row count.  Data source: U.S. Department of Transportation, TranStats database.  Air Carrier Statistics Table T-100 Domestic Market (All Carriers):  "This table contains domestic market data reported by both U.S. and foreign air carriers, including carrier, origin, destination, and service class for enplaned passengers, freight and mail when both origin and destination airports are located within the boundaries of the United States and its territories." -- 2015
End of explanation
import sympy
from sympy import *
from sympy.stats import *
from sympy import symbols
from sympy.plotting import plot
from sympy.interactive import printing
printing.init_printing(use_latex=True)
print('Sympy version ' + sympy.__version__)
Explanation: SymPy
SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS) while keeping the code as simple as possible in order to be comprehensible and easily extensible.
End of explanation
T = Normal('T', 30, 3)
Explanation: This example was gleaned from:
Rocklin, Matthew, and Andy R. Terrel. "Symbolic Statistics with SymPy." Computing in Science & Engineering 14.3 (2012): 88-93.
Problem: Data assimilation -- we want to assimilate new measurements into a set of old measurements.  Both sets of measurements have uncertainty.  For example, ACS estimates updated with local data.
Assume we've estimated that the temperature outside is 30 degrees.  However, there is certainly uncertainty is our estimate.  Let's say +- 3 degrees.  In Sympy, we can model this with a normal random variable.
End of explanation
P(T > 33)
N(P(T > 33))
Explanation: What is the probability that the temperature is actually greater than 33 degrees?
<img src="eq1.png">
We can use Sympy's integration engine to calculate a precise answer.
End of explanation
noise = Normal('noise', 0, 1.5)
observation = T + noise
Explanation: Assume we now have a thermometer and can measure the temperature.  However, there is still uncertainty involved.
End of explanation
T_posterior = given(T, Eq(observation, 26))
Explanation: We now have two measurements -- 30 +- 3 degrees and 26 +- 1.5 degrees.  How do we combine them?  30 +- 3 was our prior measurement.  We want to cacluate a better estimate of the temperature (posterior) given an observation of 26 degrees.
<img src="eq2.png">
End of explanation
<END_TASK>