Tensorflow DNNClassifier loss goes low but terrible accuracy precision recall
<p>I am using 0.10rc0 Tensorflow's contrib.learn.DNNClassifier to learn a binary classification problem.  The loss drops from initially being in the thousands to around 0.7, but accuracy and precision hover around 50% while recall has dropped to 0.0001.  Using another optimizer loss dropped to around 70, accuracy/precision hovered around 50% and recall was around 20%.</p>

<p>Question: how is it possible for loss to drop while accuracy/precision/recall remain terrible?</p>

<p>I can understand if the loss function was bad, but I'm using TF's DNNClassifier and so they've specified the loss function.  I imagine they know what they are doing in that department.  Here are some additional facts that may be relevant:</p>

<ul>
<li>training data size is 1.3 million, validation data size is 0.3 million, and there are 150 features per record</li>
<li>batch size is 50,000</li>
<li>the mean of the target column for the validation set is 0.517 so the data is very evenly split between positive (1) and negative (0) samples of the class</li>
<li>the loss for both training and validation data is around 0.7 so I think it's not an overfitting problem since they are the same (correct me if that assumption is invalid)</li>
</ul>

<p>Here's the latest validation step output:</p>

<pre><code>INFO:tensorflow:Validation (step 9280): labels/actual_target_mean = 0.517459, labels/prediction_mean = 0.498576, recall/positive_threshold_0.500000_mean = 0.000114476, recall = 0.000114476, auc = 0.499985, precision/positive_threshold_0.500000_mean = 0.487179, precision = 0.487179, loss = 0.721832, accuracy/threshold_0.500000_mean = 0.482538, global_step = 9280, accuracy/baseline_target_mean = 0.517459, accuracy = 0.482538
</code></pre>

<p>What am I missing???</p>