How to make Keras Neural Net outperforming Logistic Regression on Iris data
<p>I am comparing Keras Neural-Net with simple <a href="http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegressionCV.html" rel="nofollow">Logistic Regression from Scikit-learn</a> on IRIS data. I expect that Keras-NN will perform better, as suggested by <a href="http://blog.fastforwardlabs.com/post/139921712388/hello-world-in-keras-or-scikit-learn-versus" rel="nofollow">this post</a>.</p>

<p>But why by mimicking the code there, the result of Keras-NN is lower than 
Logistic regression?</p>

<pre><code>import seaborn as sns
import numpy as np
from sklearn.cross_validation import train_test_split
from sklearn.linear_model import LogisticRegressionCV
from keras.models import Sequential
from keras.layers.core import Dense, Activation
from keras.utils import np_utils

# Prepare data
iris = sns.load_dataset("iris")
X = iris.values[:, 0:4]
y = iris.values[:, 4]

# Make test and train set
train_X, test_X, train_y, test_y = train_test_split(X, y, train_size=0.5, random_state=0)

################################
# Evaluate Logistic Regression
################################
lr = LogisticRegressionCV()
lr.fit(train_X, train_y)
pred_y = lr.predict(test_X)
print("Test fraction correct (LR-Accuracy) = {:.2f}".format(lr.score(test_X, test_y)))



################################
# Evaluate Keras Neural Network
################################

# Make ONE-HOT
def one_hot_encode_object_array(arr):
    '''One hot encode a numpy array of objects (e.g. strings)'''
    uniques, ids = np.unique(arr, return_inverse=True)
    return np_utils.to_categorical(ids, len(uniques))


train_y_ohe = one_hot_encode_object_array(train_y)
test_y_ohe = one_hot_encode_object_array(test_y)

model = Sequential()
model.add(Dense(16, input_shape=(4,)))
model.add(Activation('sigmoid'))
model.add(Dense(3))
model.add(Activation('softmax'))
model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='adam')

# Actual modelling
model.fit(train_X, train_y_ohe, verbose=0, batch_size=1)
score, accuracy = model.evaluate(test_X, test_y_ohe, batch_size=16, verbose=0)
print("Test fraction correct (NN-Score) = {:.2f}".format(score))
print("Test fraction correct (NN-Accuracy) = {:.2f}".format(accuracy))
</code></pre>

<p>I'm using this version of Keras</p>

<pre><code>In [2]: keras.__version__
Out[2]: '1.0.1'
</code></pre>

<p>The result shows:</p>

<pre><code>Test fraction correct (LR-Accuracy) = 0.83
Test fraction correct (NN-Score) = 0.75
Test fraction correct (NN-Accuracy) = 0.60
</code></pre>

<p>According to <a href="http://blog.fastforwardlabs.com/post/139921712388/hello-world-in-keras-or-scikit-learn-versus" rel="nofollow">that post</a>, the accuracy of Keras should be 0.99. What went wrong?</p>