<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Physique
Mini Table of Contents
Using Physique from a working directory not containing Physique itself
NIST Fundamental Constants
NIST Official Conversions (to metric) 
Webscraping example
Step1:  Get the current directory
Step2:  Then append the directory containing the Physique package/library (it's just a folder) with sys.path.append; the absolute path for where I placed it just happened to be "/Users/ernestyeung/Documents/TeslaModelSP85D"
Step3:  Programming note
Step4:  Find a Fundamental Constant you are interested in using the usual panda modules
Step5:  NIST Official Conversions (to metric)
This is the pandas DataFrame containing all the NIST Official Conversions to SI.
Step6:  From the list of columns, search for the quantity you desired by trying out different search terms
Step7:  Or we can look up the SI unit we want to convert to.
Step8:  Look at what you want and see the index; it happens to be 340 in this example.
Step9:  Then the attributes can accessed by the column names.
Step10:  So for example, the reusable SSME delivers a vacuum thrust of 470000 lb or
Step11:  To obtain the conversion for pressure in psia, which we search for with "psi"
Step12:  So for a chamber pressure of 3028 psia for the SSME,
Step13:  Also, get the conversion for atmospheres (atm)
Step14:  Webscraping example
Step15:  Take a look at it with the usual BeautifulSoup modules (i.e. functions).
Now, as the Udacity Data Wrangling instructor said, Shannon Bradshaw, taught, we're going to need to use the Inspect Element (Firefox), or Develop -> Web Inspector (Mac OS X Safari) functions on your web browswer to see what the relevant html codes are.  
Now in this particular case (webpage formats are all different; assume the worst), there are no distinguishing classes for the tables (they're just nested tables on tables).  cf. stackoverflow.com BeautifulSoup scraping nested tables  I'm using the solution from this stackoverflow answer.
Step16:  Let's just focus on the Physical Data subpage for today.  This is the way to find a specific tag (in this case img) with a specific attribute (in this case alt="PHYSICAL DATA"), and then the parent module gets its parent.  Then the href index in the square brackets [] gets the web address we desire.
Step17:  At this point, I wish there was a rational and civilized manner to scrape all the relevant quantitative data from here for all the links (using Scrapy?) but I need help at this point for that endeavor.  Otherwise, I manually look at the webpage itself and manually use Inspect Element to find what I want and then use BeautifulSoup accordingly.
Step18:  Time to scrape the actual html code for the table we desire
Step19:  I'll add back the units as part of the data (I don't know a sane and civilized way of attaching to each of the column names in pandas, a pandas DataFrame, the units, as extra information)
Step20:  Time to save our work as a "pickle'd" pandas DataFrame.
Step21:  And so to access this, to use in Python, do the following, using .read_pickle of pandas
<ASSISTANT_TASK:>
Python Code:
import os, sys
Explanation: Physique
Mini Table of Contents
Using Physique from a working directory not containing Physique itself
NIST Fundamental Constants
NIST Official Conversions (to metric) 
Webscraping example: JPL Solar System Dynamics (JPL SSD) - Planets and Pluto
Using Physique from a working directory not containing Physique itself
End of explanation
currentdir = os.getcwd(); os.getcwd();
Explanation: Get the current directory
End of explanation
currentdir # I'm on a different computer now
sys.path.append('/home/topolo/PropD/Propulsion/')
import Physique
Explanation: Then append the directory containing the Physique package/library (it's just a folder) with sys.path.append; the absolute path for where I placed it just happened to be "/Users/ernestyeung/Documents/TeslaModelSP85D": substitute that for the absolute path you find (look at your Finder or File organizing program)
End of explanation
from Physique import FundConst
print Physique.FundConst.columns
Physique.FundConst
Explanation: Programming note: __init__.py in the main directory uses os.path.dirname(__file__) with __file__ (literally that, it's not a placeholder name) being the string with the absolute pathname of the "file from which the module was loaded, if it was loaded from a file" (cf. stackoverflow  Python file attribute absolute or relative?), i.e. "When a module is loaded in Python, __file__ is set to its name. You can then use that with other functions to find the directory that the file is located in." (cf. stackoverflow  what does the file wildcard mean/do?)  
NIST Fundamental Constants
End of explanation
g_0pd = FundConst[ FundConst["Quantity"].str.contains("gravity") ] 
# standard acceleration of gravity as a panda DataFrame
g_0pd
# access the values you're interested in 
print g_0pd.Quantity
print g_0pd.Value.get_values()[0] 
print g_0pd.Unit.get_values()[0] 
# you can also grab just the 1 entry from this DataFrame using the .loc module
FundConst[FundConst["Quantity"].str.contains("Boltzmann")].loc[49,:]
g_0pd.loc[303,:]
Explanation: Find a Fundamental Constant you are interested in using the usual panda modules
End of explanation
convDF = Physique.conv
convDF.columns
Explanation: NIST Official Conversions (to metric)
This is the pandas DataFrame containing all the NIST Official Conversions to SI.
End of explanation
convDF[convDF['Toconvertfrom'].str.contains("pound-force ")]
Explanation: From the list of columns, search for the quantity you desired by trying out different search terms: e.g. I'm reading Huzel and Huang's Modern Engineering for Design of Liquid-Propellant Rocket Engines and I want to know how to convert from
* lb (pound or pound-force) for thrust into force in Newton (N)
* psia (pounds per square inch absolute) for (chamber) pressure into pressure in Pascal (Pa)
We can try to look up the U.S. or Imperial units from the Toconvertfrom column.
End of explanation
convDF[convDF['to'].str.contains("newton ")]
Explanation: Or we can look up the SI unit we want to convert to.
End of explanation
lbf2N = convDF.loc[340,:]; lbf2N
Explanation: Look at what you want and see the index; it happens to be 340 in this example.
End of explanation
print lbf2N.Toconvertfrom, lbf2N.to, lbf2N.Multiplyby
Explanation: Then the attributes can accessed by the column names.
End of explanation
print 470000*lbf2N.Multiplyby, lbf2N.to 
Explanation: So for example, the reusable SSME delivers a vacuum thrust of 470000 lb or
End of explanation
convDF[convDF['Toconvertfrom'].str.match("psi")]
Explanation: To obtain the conversion for pressure in psia, which we search for with "psi"
End of explanation
psi2Pa = convDF.loc[372,:]
print 3028*psi2Pa.Multiplyby, psi2Pa.to
Explanation: So for a chamber pressure of 3028 psia for the SSME,
End of explanation
convDF[convDF['Toconvertfrom'].str.match("atm")]
atm2Pa = convDF.loc[15,:]
print 3028*psi2Pa.Multiplyby/atm2Pa.Multiplyby, atm2Pa.Toconvertfrom
Explanation: Also, get the conversion for atmospheres (atm):
End of explanation
JPL_SSD_URL = "http://ssd.jpl.nasa.gov/" # JPL NASA Solar System Dynamics webpage
jpl_ssd_BS = Physique.scrape_BS.scraped_BS(JPL_SSD_URL)
Explanation: Webscraping example: JPL Solar System Dynamics (JPL SSD) - Planets and Pluto
Take a look at the file scrape_BS.py in this Physique folder.  BS stands for the BeautifulSoup python module that's extensively used here.  Start at the class called scraped_BS which will use the python module requests to put the html out from a url address into a BeautifulSoup object.
End of explanation
# for table in jpl_ssd_BS.soup.find_all("table"):
#    for subtable in table.find_all("table"):
#        print subtable.find("table") # uncomment this out and run it to see the whole darn thing
Explanation: Take a look at it with the usual BeautifulSoup modules (i.e. functions).
Now, as the Udacity Data Wrangling instructor said, Shannon Bradshaw, taught, we're going to need to use the Inspect Element (Firefox), or Develop -> Web Inspector (Mac OS X Safari) functions on your web browswer to see what the relevant html codes are.  
Now in this particular case (webpage formats are all different; assume the worst), there are no distinguishing classes for the tables (they're just nested tables on tables).  cf. stackoverflow.com BeautifulSoup scraping nested tables  I'm using the solution from this stackoverflow answer.
End of explanation
jpl_ssd_BS.soup.find('img',{"alt":"PHYSICAL DATA"}).parent['href']
JPL_SSD_PHYS_DATA_URL = JPL_SSD_URL + jpl_ssd_BS.soup.find('img',{"alt":"PHYSICAL DATA"}).parent['href'][1:]
JPL_SSD_PHYS_DATA_URL
jpl_ssd_phys_data_BS = Physique.scrape_BS.scraped_BS(JPL_SSD_PHYS_DATA_URL)
Explanation: Let's just focus on the Physical Data subpage for today.  This is the way to find a specific tag (in this case img) with a specific attribute (in this case alt="PHYSICAL DATA"), and then the parent module gets its parent.  Then the href index in the square brackets [] gets the web address we desire.
End of explanation
jpl_ssd_phys_data_BS.soup.find('h2',text="Planets").find_next('a')
JPL_SSD_PLANET_PHYS_PAR_URL = JPL_SSD_URL + jpl_ssd_phys_data_BS.soup.find('h2',text="Planets").find_next('a')['href']
jpl_ssd_planet_phys_par_BS = Physique.scrape_BS.scraped_BS(JPL_SSD_PLANET_PHYS_PAR_URL)
jpl_ssd_planet_phys_parTBL = jpl_ssd_planet_phys_par_BS.soup.find("div", {"class":"page_title"}).find_next("table")
Explanation: At this point, I wish there was a rational and civilized manner to scrape all the relevant quantitative data from here for all the links (using Scrapy?) but I need help at this point for that endeavor.  Otherwise, I manually look at the webpage itself and manually use Inspect Element to find what I want and then use BeautifulSoup accordingly.
End of explanation
data = []
for row in jpl_ssd_planet_phys_parTBL.find_all('tr', recursive=False):
    cols = row.find_all('td', recursive=False)
    cols = [ele.text if ele.text != u'\xa0' else u'' for ele in cols]
    data.append(cols) 
hdrs = data[:2]    # get the headers first
jpl_ssd_planet_phys_parTBL.find_all('tr')[2].find_all('td')[18].text
data = [[row[0].replace(u'\xa0',''),]+row[1:] for row in data[2:]]  # remove the space, \xa0 from each of the planet's names
data = [[row[0],]+[ele.replace('\n','') for ele in row[1:]] for row in data] # remove the '\n' strings
data = [[row[0],]+[ele.split(u'\xb1')[0] for ele in row[1:]] for row in data] # let's just get the values 
data = [[row[0],]+[ele.split(u'\xa0')[0] for ele in row[1:]] for row in data] # let's just get the values
Explanation: Time to scrape the actual html code for the table we desire: jpl_ssd_planet_phys_parTBL.  Take a look at the function make_conv_lst in scrape_BS.py and take a look at that first for loop.  That's the procedure we'll take (and I confirmed that this is in practice what's done on stackoverflow).  But wait: the data values are themselves tables.  So again, there is no rhyme or reason for the logic or rationale for the html tables for data, in general, for any websites.  So I'll get the headers first (which makes sense) and do an ugly hack for the data values (also notice the recursive=False option).
End of explanation
data = [hdrs[1],] + data
import pandas as pd
data = pd.DataFrame( data )
data.columns = hdrs[0]
data
Explanation: I'll add back the units as part of the data (I don't know a sane and civilized way of attaching to each of the column names in pandas, a pandas DataFrame, the units, as extra information)
End of explanation
data.to_pickle('./rawdata/JPL_NASA_SSD_Planet_Phys_Par_values.pkl') # values only
Explanation: Time to save our work as a "pickle'd" pandas DataFrame.
End of explanation
PlanetParDF = pd.read_pickle('./rawdata/JPL_NASA_SSD_Planet_Phys_Par_values.pkl')
Explanation: And so to access this, to use in Python, do the following, using .read_pickle of pandas:
End of explanation
<END_TASK>