Trouble getting PolynomialFeatures to work while fitting a logistic model in Scikit Learn
<p>Kaggle has an introductory data science problem on the Titanic, where the goal is to predict the chances of survival of a passenger, given some information about him/her (eg. sex, age, cabin class etc.). I implemented a simple logistic regression model for this in Python using Scikit Learn, and I was exploring adding higher degree factors of the "Age" variable in particular. I used PolynomialFeatures following the instructions on Scikit Learn's site:</p>

<pre><code>import pandas as pd
from sklearn import linear_model
import numpy as np
from sklearn import preprocessing
from sklearn.preprocessing import PolynomialFeatures

# Import titanic data
titanic = pd.read_csv("train.csv")

# Set the training set as 70% of the dataset and cross_validation set as remaining 30%
predictors = ["Age"]
training_set = titanic[predictors].iloc[range(0,int(titanic.shape[0]*0.7)),:]
cv_set = titanic[predictors].iloc[range(int(titanic.shape[0]*0.7),titanic.shape[0]),:]
training_actuals = titanic["Survived"].iloc[range(0,int(titanic.shape[0]*0.7))]
cv_actuals = titanic["Survived"].iloc[range(int(titanic.shape[0]*0.7),titanic.shape[0])]

# Create polynomial features
poly = PolynomialFeatures(degree=3)
training_set = poly.fit_transform(training_set)
cv_set = poly.fit_transform(cv_set)

# Fit a logistic regression model, predict values for training and cross-validation sets
alg = linear_model.LogisticRegression()
alg.fit(training_set, training_actuals)
cv_predictions = alg.predict(cv_set)
training_predictions = alg.predict(training_set)

# Measure and print accuracy of prediction over both training and cross-validation sets
cv_accuracy = len(cv_predictions[cv_predictions == np.array(cv_actuals)])/float(len(cv_predictions))
print "Prediction accuracy on cross-validation set is %s%%" % (cv_accuracy * 100)
training_accuracy = len(training_predictions[training_predictions == np.array(training_actuals)])/float(len(training_predictions))
print "Prediction accuracy on training set is %s%%" % (training_accuracy * 100)
</code></pre>

<p>My accuracy of predicted values over the training set improves 1-2 percentage points when I add a square feature for the age (i.e. polynomial degree 2), but when I make the degree 3, as in the code above, the accuracy actually reverts to the same as for the linear case (i.e. degree = 1). Theoretically it should improve slightly or stay the same as for degree = 2. This behavior continues for all higher degrees as well. I am very new to Scikit Learn, and I would appreciate any insight on what I am doing wrong. </p>