<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Labos en Python
Tous les labos, version Python. (Pas de labo 1.)
Step1:  Labo 2
Import data
Step2:  Dataframe manipulation
show var (columns)
delete var
rename var
create var
head
Step3:  Normality
Skewness
Step4:  Kurtosis
Step5:  Kolmogorov-Smirnov
Lilliefors 
Step6:  Shapiro-Wilk
http
Step7:  Transformations
Square root
Step8:  Logarithmic
Step9:  Centrage et réduction
Step10:  Descriptive statistics
Step11:  Labo 3
Histograms
Classic histograms
Histograms with normal curve
Scatterplots
Scatterplots avec droite de régression
Scatterplots matrix
3D scatterplots
Correlation matrix
Step12:  Pearson
Spearman
Simple linear regression
Contingency table
Modalités variables nominales
Contingency table
Labo 4
T-Test
F Test
Step13:  Satterthwaite method
Pooled method
Results analysis
ANOVA
Variance analysis
http
<ASSISTANT_TASK:>
Python Code:
# jupyter magic
%matplotlib inline
# python scientific stack
import numpy as np
import pandas as pd
import scipy.stats as scs
import statsmodels
import statsmodels.api as sm
import statsmodels.formula.api as smf
import statsmodels.stats as sms
# fileformat
from simpledbf import Dbf5
from sas7bdat import SAS7BDAT
Explanation: Labos en Python
Tous les labos, version Python. (Pas de labo 1.)
End of explanation
# excel
#df = pd.read_excel('data/labo2/SR_Data.xls')
# DBF (Dbase)
dbf = Dbf5('data/labo2/SR_Data.dbf')
df = dbf.to_dataframe()
# SPSS
# savReaderWriter error with pip install
# SAS
sas = SAS7BDAT('data/labo2/tableau1.sas7bdat')
Explanation: Labo 2
Import data
End of explanation
# show vars
df.columns
# delete var
df = df.drop('Shape_Leng', 1) # 1 = column axis
# df.drop('Shape_Leng', 1, inplace=True) # same as previous, inplace impacts this dataframe instead of the returned one
# rename var
df = df.rename(columns={'POPTOT_FR':'POPTOT'})
# create var
df['km'] = df['Shape_Area'] / 1000000
df['HabKm2'] = df['POPTOT'] / df['km']
# show data head
df.head()
Explanation: Dataframe manipulation
show var (columns)
delete var
rename var
create var
head
End of explanation
#scs.skew(df)
df.skew()
Explanation: Normality
Skewness
End of explanation
df.kurt()  # or df.kurtosis()
Explanation: Kurtosis
End of explanation
# scs.kstest(df['HabKm2'], 'norm')
statsmodels.stats.diagnostic.kstest_normal(df['HabKm2'])
Explanation: Kolmogorov-Smirnov
Lilliefors : Kolmogorov Smirnov test with estimated mean and variance
End of explanation
scs.shapiro(df['HabKm2'])
Explanation: Shapiro-Wilk
http://www.science-emergence.com/NormalityTests/
End of explanation
df['SqrtDens'] = np.sqrt(df['HabKm2'])
df['SqrtImg'] = np.sqrt(df['IMMREC_PCT'])
Explanation: Transformations
Square root
End of explanation
# log(0) = error
df['LogDens'] = np.log(df['HabKm2'])
df['LogImg'] = np.log(df['IMMREC_PCT'] + 1)
Explanation: Logarithmic
End of explanation
#df[['INDICE_PAU', 'Dist_Min', 'N_1000', 'Dist_Moy_3']]
zscores = df.ix[:,11:15]
# scaling from machine learning
from sklearn.preprocessing import scale
zscores = pd.DataFrame(scale(zscores), index=zscores.index, columns=zscores.columns)
zscores.mean()
zscores.std()
Explanation: Centrage et réduction
End of explanation
df.describe()
df.mean()
df.std()
df.min()
df.max()
df.median()
#df.range() : min, max
df.quantile(0.75) # param : 0.25, 0.75... default 0.5 
Explanation: Descriptive statistics
End of explanation
df.corr()
Explanation: Labo 3
Histograms
Classic histograms
Histograms with normal curve
Scatterplots
Scatterplots avec droite de régression
Scatterplots matrix
3D scatterplots
Correlation matrix
End of explanation
scs.ttest_ind?
Explanation: Pearson
Spearman
Simple linear regression
Contingency table
Modalités variables nominales
Contingency table
Labo 4
T-Test
F Test
End of explanation
df.cov()
#statsmodels.stats.anova.anova_lm
statsmodels.stats.anova.anova_lm?
Explanation: Satterthwaite method
Pooled method
Results analysis
ANOVA
Variance analysis
http://stackoverflow.com/questions/25537399/anova-in-python-using-pandas-dataframe-with-statsmodels-or-scipy
End of explanation
<END_TASK>