<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Reading Data
Python has a large number of different ways to read data from external files. 
Python supports almost any type of file you can think of, from simple text files to complex binary formats.
In this class we are going to mainly use the pakages Astropy and Pandas to load extrnal files.
Both of these packages create a python object called a Table.
Tables are very useful, since there are lots of built-in methods that allow us to easily manipulate the data.
Step1:  The AstroPy package - QTable
Step2:  Renaming columns
Step3:  Sorting
Step4:  Masking
Step5:  Adding a column to the Table
Step6:  Saving a table
Step7:  The Pandas package - DataFrame
Step8:  Renaming columns
Step9:  Sorting
Step10:  Masking
Step11:  Adding a column to the Table
Step12:  Saving a table
Step13:  QTables vs. DataFrames
As you can see, the astropy QTable and the pandas DataFrame are very similar.
There are some important differences that we will discover this quarter.
Astronomers use both packages, depending on the situation.
Pandas is the dominate packages outside astronomy.
Part I - Advantage Pandas
Pandas is really good for working with dates!
Step14:  Messy Data
Pandas is a good choice when working with messy data files.
In the "real world" all data is messy.
For example, here is the contents of the file Mess.csv
Step15:  Skip the header
Step16:  NaN = Not_A_Number, python's null value
Column names are messed up
Option 1 - Turn off the header
Step17:  Option 2 - Add the column names
Step18:  Deal with the missing data with fillna()
<ASSISTANT_TASK:>
Python Code:
import os
Explanation: Reading Data
Python has a large number of different ways to read data from external files. 
Python supports almost any type of file you can think of, from simple text files to complex binary formats.
In this class we are going to mainly use the pakages Astropy and Pandas to load extrnal files.
Both of these packages create a python object called a Table.
Tables are very useful, since there are lots of built-in methods that allow us to easily manipulate the data.
End of explanation
import numpy as np
from astropy.table import QTable
os.listdir()
planet_table = QTable.read('Planets.csv', format='ascii.csv')
planet_table
print(planet_table)
Explanation: The AstroPy package - QTable
End of explanation
planet_table.rename_column('col2', 'ecc')
print(planet_table)
planet_table['Name']
planet_table['Name'][0]
Explanation: Renaming columns
End of explanation
planet_table.sort(['ecc'])
planet_table
Explanation: Sorting
End of explanation
planet_table.sort(['a'])    # re-sort our table
mask1 = np.where(planet_table['a'] > 5)
mask1
planet_table[mask1]
mask2 = ((planet_table['a'] > 5) &
         (planet_table['ecc'] < 0.05))
planet_table[mask2]
Explanation: Masking
End of explanation
perihelion = planet_table['a'] * (1.0 - planet_table['ecc'])
perihelion
planet_table['Peri'] = perihelion
planet_table
Explanation: Adding a column to the Table
End of explanation
planet_table.write('NewPlanets.csv', format='ascii.csv')
os.listdir()
Explanation: Saving a table
End of explanation
import pandas as pd
planet_table2 = pd.read_csv('Planets.csv')
planet_table2
print(planet_table2)
Explanation: The Pandas package - DataFrame
End of explanation
planet_table2.rename(columns={'Unnamed: 2': 'ecc'}, inplace=True)
planet_table2
planet_table2['Name']
planet_table['Name'][0]
Explanation: Renaming columns
End of explanation
planet_table2.sort_values(['ecc'])
planet_table2
planet_table2.sort_values(['ecc'], ascending=False)
Explanation: Sorting
End of explanation
mask3 = planet_table['a'] > 5
mask3
planet_table2[mask3]
mask4 = ((planet_table2['a'] > 5) &
         (planet_table2['ecc'] < 0.05))
planet_table2[mask4]
Explanation: Masking
End of explanation
perihelion = planet_table2['a'] * (1.0 - planet_table2['ecc'])
perihelion
planet_table2['Peri'] = perihelion
planet_table2
Explanation: Adding a column to the Table
End of explanation
planet_table2.to_csv('NewPlanets2.csv', index=False)
os.listdir()
Explanation: Saving a table
End of explanation
import datetime
doctor_table = pd.read_csv('Doctor.csv')
doctor_table
doctor_table.sort_values(['BirthDate'])
doctor_table['BirthDate'] = pd.to_datetime(doctor_table['BirthDate'])
doctor_table.sort_values(['BirthDate'])
today = datetime.date.today()
today
age = today - doctor_table['BirthDate']
age
doctor_table['AgeToday'] = age / np.timedelta64(1, 'Y')
doctor_table
doctor_table.describe()
Explanation: QTables vs. DataFrames
As you can see, the astropy QTable and the pandas DataFrame are very similar.
There are some important differences that we will discover this quarter.
Astronomers use both packages, depending on the situation.
Pandas is the dominate packages outside astronomy.
Part I - Advantage Pandas
Pandas is really good for working with dates!
End of explanation
messy_table = pd.read_csv('Mess.csv')
Explanation: Messy Data
Pandas is a good choice when working with messy data files.
In the "real world" all data is messy.
For example, here is the contents of the file Mess.csv:
This is not going to end well ...
End of explanation
messy_table = pd.read_csv('Mess.csv', skiprows = 6)
messy_table
Explanation: Skip the header
End of explanation
messy_table = pd.read_csv('Mess.csv', skiprows = 6, header= None)
messy_table
Explanation: NaN = Not_A_Number, python's null value
Column names are messed up
Option 1 - Turn off the header
End of explanation
cols = ["Name", "Size"]
messy_table = pd.read_csv('Mess.csv', skiprows = 6, names = cols)
messy_table
Explanation: Option 2 - Add the column names
End of explanation
messy_table['Name'].fillna("unknown", inplace=True)
messy_table['Size'].fillna(999.0, inplace=True)
messy_table
Explanation: Deal with the missing data with fillna()
End of explanation
<END_TASK>