<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Regression Modeling in Practice
Assignment
Step1:  General information on the Gapminder data
Step2:  Variables distribution
Before computing the linear regression between the icome per person and the residential electricity consumption, let's have a look at the distributions of the variables.
Step3:  Income per person
Step4:  From the distribution graph, we can see that the distribution is skewed-right and bimodal. There is also a singular case with high income per capita. That country is 
Step5:  For this assignment the explanatory variable income per person will have to be centered as its mean is 8784.5.
Residential electricity consumption
Step6:  The residential electricity consumption is also skewed-right. And there are also a couple of countries presenting unusual higher values. Those countries are 
Step7:  Bivariate distribution
If we look now at the distribution of the two variables in a scatter plot. We can see that our singular countries for the two variables of study result in outlier points. Nothing special was done about them as there are no managment error on them and no reason to discard them.
Moreover, the distribution of the data along the regression line shows that the homoscedasticity hypothesis is not well met as the spread of the data along the line is bigger for higher values of income.
Step8:  Centering the explanatory variable
For the assignment, the explanatory variable (here the income per person) has to be centered by subracting its mean.
Step9:  OLS regression model
<ASSISTANT_TASK:>
Python Code:
# Magic command to insert the graph directly in the notebook
%matplotlib inline
# Load a useful Python libraries for handling data
import pandas as pd
import numpy as np
import statsmodels.formula.api as smf
import scipy.stats as stats
import seaborn as sns
import matplotlib.pyplot as plt
from IPython.display import Markdown, display
# Read the data
data_filename = r'gapminder.csv'
data = pd.read_csv(data_filename, low_memory=False)
data = data.set_index('country')
Explanation: Regression Modeling in Practice
Assignment: Test a Basic Linear Regression Model
Following is the Python program I wrote to fulfill the second assignment of the Regression Modeling in Practice online course.
I decided to use Jupyter Notebook as it is a pretty way to write code and present results.
Assignment research question
Using the Gapminder database, I would like to see if there is a linear relationship between the income per person (explanatory variable) and the residential consumption of electricity (response variable).
Data management
For the question I'm interested in, the countries for which data are missing will be discarded. As missing data in Gapminder database are replace directly by NaN no special data treatment is needed.
End of explanation
display(Markdown("Number of countries: {}".format(len(data))))
display(Markdown("Number of variables: {}".format(len(data.columns))))
Explanation: General information on the Gapminder data
End of explanation
subdata2 = (data[['incomeperperson', 'relectricperperson']]
                .assign(income=lambda x: pd.to_numeric(data['incomeperperson'], errors='coerce'),
                        electricity=lambda x: pd.to_numeric(data['relectricperperson'], errors='coerce'))
                .dropna())
Explanation: Variables distribution
Before computing the linear regression between the icome per person and the residential electricity consumption, let's have a look at the distributions of the variables.
End of explanation
sns.distplot(subdata2['income'], kde=False)
plt.xlabel('Income per person (2000 US$)');
Explanation: Income per person
End of explanation
subdata2.loc[subdata2['income'] > 45000]
subdata2['income'].describe()
Explanation: From the distribution graph, we can see that the distribution is skewed-right and bimodal. There is also a singular case with high income per capita. That country is :
End of explanation
sns.distplot(subdata2['electricity'], kde=False)
plt.xlabel('Residential electricity consumption (kWh)');
subdata2['electricity'].describe()
Explanation: For this assignment the explanatory variable income per person will have to be centered as its mean is 8784.5.
Residential electricity consumption
End of explanation
subdata2.loc[subdata2['electricity'] > 6000]
Explanation: The residential electricity consumption is also skewed-right. And there are also a couple of countries presenting unusual higher values. Those countries are :
End of explanation
sns.regplot(x='income', y='electricity', data=subdata2)
plt.xlabel('Income per person (2000 US$)')
plt.ylabel('Residential electricity consumption (kWh)')
plt.title('Scatterplot for the association between the income and the residential electricity consumption');
Explanation: Bivariate distribution
If we look now at the distribution of the two variables in a scatter plot. We can see that our singular countries for the two variables of study result in outlier points. Nothing special was done about them as there are no managment error on them and no reason to discard them.
Moreover, the distribution of the data along the regression line shows that the homoscedasticity hypothesis is not well met as the spread of the data along the line is bigger for higher values of income.
End of explanation
subdata3 = subdata2.assign(income_centered=lambda x: x['income']-subdata2['income'].mean())
display(Markdown("Income mean after centereing : {:3g}".format(subdata3['income_centered'].mean())))
sns.regplot(x='income_centered', y='electricity', data=subdata3)
plt.xlabel('Centered income per person (2000 US$)')
plt.ylabel('Residential electricity consumption (kWh)')
plt.title('Scatterplot for the association between the income and the residential electricity consumption');
Explanation: Centering the explanatory variable
For the assignment, the explanatory variable (here the income per person) has to be centered by subracting its mean.
End of explanation
reg1 = smf.ols('electricity ~ income_centered', data=subdata3).fit()
reg1.summary()
Explanation: OLS regression model
End of explanation
<END_TASK>