R and Random Forest: How caret and pROC deal with positive and negative class?
<p>In the past days, I've been analyzing the performance of R's implementation of Random Forest and the different tools available in order to obtain:</p>

<ul>
<li>AUC</li>
<li>Sensitivity</li>
<li>Specificity</li>
</ul>

<p>Thus, I've used two different methods:</p>

<ul>
<li>mroc and coords from <strong>pROC</strong> library in order to obtain the performance of the model at different cutoff points.</li>
<li>confusionMatrix from <strong>caret</strong> library in order to obtain the optimal performance of the model (AUC, Accuracy, Sensitivity, Specificity, ...)</li>
</ul>

<p>The point is that I've realized that there is some differences between both approaches.</p>

<p>I've developed the following code:</p>

<pre><code>suppressMessages(library(randomForest))
suppressMessages(library(pROC))
suppressMessages(library(caret))

set.seed(100)

t_x &lt;- as.data.frame(matrix(runif(100),ncol=10))
t_y &lt;- factor(sample(c("A","B"), 10, replace = T), levels=c("A","B"))

v_x  &lt;- as.data.frame(matrix(runif(50),ncol=10))
v_y &lt;- factor(sample(c("A","B"), 5, replace = T), levels=c("A","B"))

model &lt;- randomForest(t_x, t_y, ntree=1000, importance=T);
prob.out &lt;- predict(model, v_x, type="prob")[,1];
prediction.out &lt;- predict(model, v_x, type="response");

mroc &lt;- roc(v_y,prob.out,plot=F)

results &lt;- coords(mroc,seq(0, 1, by = 0.01),input=c("threshold"),ret=c("sensitivity","specificity","ppv","npv"))

accuracyData &lt;- confusionMatrix(prediction.out,v_y)
</code></pre>

<p>If you compare the <strong>results</strong> and <strong>accuracyData</strong> variables, you can see that the relationship between sensitivity and specificity is the opposite.</p>

<p>That is, the confusionMatrix results are:</p>

<pre><code>Confusion Matrix and Statistics

          Reference
Prediction A B
         A 1 1
         B 2 1

               Accuracy : 0.4             
                 95% CI : (0.0527, 0.8534)
    No Information Rate : 0.6             
    P-Value [Acc &gt; NIR] : 0.913           

                  Kappa : -0.1538         
 Mcnemar's Test P-Value : 1.000           

            Sensitivity : 0.3333          
            Specificity : 0.5000          
         Pos Pred Value : 0.5000          
         Neg Pred Value : 0.3333          
             Prevalence : 0.6000          
         Detection Rate : 0.2000          
   Detection Prevalence : 0.4000          
      Balanced Accuracy : 0.4167          

       'Positive' Class : A 
</code></pre>

<p>But if I look for such Sensitivity and Specificity in the coords calculation, I find them swapped:</p>

<pre><code>     sensitivity specificity       ppv       npv
0.32         0.5   0.3333333 0.3333333 0.5000000
</code></pre>

<p>Apparently, Sensitivity and Specificity is are opposite in coords and confusionMatrix.</p>

<p>Taking into account that confusionMatrix identifies correctly the positive class, I assume that this good interpretation of Sensitivity and Specificity.</p>

<p>My question is: <em>Is there any way of forcing coords to interpret the positive and negative classes in the way I want to?</em></p>