<SYSTEM_TASK:>
Given the following text description, write Python code to implement the functionality described below step by step
<END_TASK>
<USER_TASK:>
Description:
 Titanic kaggle competition with SVM
Step1:  Let's load and examine the titanic data with pandas first.
Step2:  So we have 891 training examples with 10 information columns given. Of course it is not straight forward to use all of them at this point. 
In this example, we will just explore two simple SVM models that only use two features. 
Our choice of models are
Step3:  Here is how we examine how the selection works.
Step4:  Gender-Age model
We will introduce a few concepts while including the Age feature. 
missing values
identify missing values
Step5:  How many missing values are there?
Step6:  SVM does not allow features with missing values, what do we do? 
One idea would be to fill in them with a number we think is reasonable. 
Let's try to use the average age first.
Step7:  can you think of better ways to do this?
Step8:  feature rescaling
Step9:  Let's examine the selection function of the model.
Step10:  Create a submission file with the Gender-Age model
First we want to read in the test data set and add in the gender features as what we did with the training data set.
Step11:  We notice again that some of the age value is missing in the test data, and want to fill in the same way as what we did with the training data.
Step12:  Note here we give the missing values the mean age of the training data. 
What's the pros and cons of doing this? 
We want to get the features from the test data, and scale our age feature the same way as what we did in the training data.
Step13:  We use the model above to predict the survive of our test data. 
The model is fitted with the entire training data.
Step14:  create a file that can be submit to kaggle
We read in the example submission file provided by kaggle, and then replace the "Survived" column with our own prediction.
We use the to_csv method of panda dataframe, now we can check with kaggle on how well we are doing.
<ASSISTANT_TASK:>
Python Code:
#import all the needed package
import numpy as np
import scipy as sp
import re
import pandas as pd
import sklearn
from sklearn.cross_validation import train_test_split,cross_val_score
from sklearn.preprocessing import StandardScaler
from sklearn import metrics
import matplotlib 
from matplotlib import pyplot as plt
%matplotlib inline
from sklearn.svm import SVC
Explanation: Titanic kaggle competition with SVM
End of explanation
data = pd.read_csv('data/train.csv')
print data.head()
# our target is the survived column
y= data['Survived']
print data.shape
Explanation: Let's load and examine the titanic data with pandas first.
End of explanation
#add in Sex_male features
data['Sex_male']=data.Sex.map({'female':0,'male':1})
data.head()
#get the features we indented to use 
feature_cols=['Pclass','Sex_male']
X=data[feature_cols]
X.head()
#use the default SVM rbf model
model=SVC()
scores=cross_val_score(model,X,y,cv=10,scoring='accuracy')
print scores, np.mean(scores),np.std(scores)
Explanation: So we have 891 training examples with 10 information columns given. Of course it is not straight forward to use all of them at this point. 
In this example, we will just explore two simple SVM models that only use two features. 
Our choice of models are:
- gender, class
- gender, feature
Gender-Class model
Recall how we generated features from catagories last session. We use the same method to generate an additional feature called Sex_male.
End of explanation
xmin,xmax=X['Pclass'].min()-0.5,X['Pclass'].max()+0.5
ymin,ymax=X['Sex_male'].min()-0.5,X['Sex_male'].max()+0.5
print xmin,xmax,ymin,ymax
xx, yy = np.meshgrid(np.linspace(xmin, xmax, 200), np.linspace(ymin, ymax, 200))
model.fit(X,y)
Z = model.decision_function(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
fig=plt.figure(figsize=(20,10))
ax=fig.add_subplot(111)
ax.pcolormesh(xx, yy, -Z, cmap=plt.cm.RdBu,alpha=0.5)
ax.scatter(X['Pclass']+np.random.randn(len(X['Pclass']))*0.1, X['Sex_male']+np.random.randn(len(X['Pclass']))*0.05, c=y,s=40, cmap=plt.cm.RdBu_r)
ax.set_xlabel("Pclass")
ax.set_ylabel("Sex_male")
ax.set_xlim([0.5,3.5])
ax.set_ylim([-0.5,1.5])
plt.show()
Explanation: Here is how we examine how the selection works.
End of explanation
#use the isnull function to check if there is any missing value in the Age column. 
pd.isnull(data['Age']).any()
Explanation: Gender-Age model
We will introduce a few concepts while including the Age feature. 
missing values
identify missing values
End of explanation
print len(data['Age'][pd.isnull(data['Age'])])
Explanation: How many missing values are there?
End of explanation
data['Age'][pd.isnull(data['Age'])]=data['Age'].mean()
Explanation: SVM does not allow features with missing values, what do we do? 
One idea would be to fill in them with a number we think is reasonable. 
Let's try to use the average age first.
End of explanation
#generate our new feature
feature_cols=['Age','Sex_male']
X=data[feature_cols]
X.head()
#use the default SVM rbf model
scores=cross_val_score(model,X,y,cv=10,scoring='accuracy')
print scores, np.mean(scores),np.std(scores)
Explanation: can you think of better ways to do this?
End of explanation
X['Age']=(X['Age']-X['Age'].median())/X['Age'].std()
#X = StandardScaler().fit_transform(X)
scores=cross_val_score(model,X,y,cv=10,scoring='accuracy')
print scores, np.mean(scores),np.std(scores)
Explanation: feature rescaling
End of explanation
xmin,xmax=X['Age'].min()-0.5,X['Age'].max()+0.5
ymin,ymax=X['Sex_male'].min()-0.5,X['Sex_male'].max()+0.5
print xmin,xmax,ymin,ymax
xx, yy = np.meshgrid(np.linspace(xmin, xmax, 200), np.linspace(ymin, ymax, 200))
model.fit(X,y)
Z = model.decision_function(np.c_[xx.ravel(), yy.ravel()])
Z = Z.reshape(xx.shape)
fig=plt.figure(figsize=(20,10))
ax=fig.add_subplot(111)
ax.pcolormesh(xx, yy, -Z, cmap=plt.cm.RdBu,alpha=0.5)
ax.scatter(X['Age'], X['Sex_male']+np.random.randn(len(X['Age']))*0.05, c=y,s=40, cmap=plt.cm.RdBu_r)
ax.set_xlabel("Normalized Age")
ax.set_ylabel("Sex_male")
ax.set_ylim([-0.5,1.5])
ax.set_xlim([-3,4.5])
plt.show()
Explanation: Let's examine the selection function of the model.
End of explanation
test_data = pd.read_csv('data/test.csv')
#print test_data.head()
#add in Sex_male features
test_data['Sex_male']=test_data.Sex.map({'female':0,'male':1})
Explanation: Create a submission file with the Gender-Age model
First we want to read in the test data set and add in the gender features as what we did with the training data set.
End of explanation
#use the isnull function to check if there is any missing value in the Age column. 
pd.isnull(test_data['Age']).any()
print len(test_data['Age'][pd.isnull(test_data['Age'])])
test_data['Age'][pd.isnull(test_data['Age'])]=data['Age'].mean()
Explanation: We notice again that some of the age value is missing in the test data, and want to fill in the same way as what we did with the training data.
End of explanation
#generate our new feature
X_test=test_data[feature_cols]
X_test['Age']=(X_test['Age']-data['Age'].median())/data['Age'].std()
Explanation: Note here we give the missing values the mean age of the training data. 
What's the pros and cons of doing this? 
We want to get the features from the test data, and scale our age feature the same way as what we did in the training data.
End of explanation
y_pred=model.predict(X_test)
X_test.head()
Explanation: We use the model above to predict the survive of our test data. 
The model is fitted with the entire training data.
End of explanation
samplesubmit = pd.read_csv("data/titanic_submit_example.csv")
#samplesubmit.head()
samplesubmit["Survived"]=y_pred
#samplesubmit.to_csv
samplesubmit.to_csv("data/titanic_submit_gender_age.csv",index=False)
samplesubmit.head()
Explanation: create a file that can be submit to kaggle
We read in the example submission file provided by kaggle, and then replace the "Survived" column with our own prediction.
We use the to_csv method of panda dataframe, now we can check with kaggle on how well we are doing.
End of explanation
<END_TASK>