Matlab Machine Learning Train, Validate, Test Partitions
<p>I'm using Matlab's Statistics and Machine Learning Toolbox to create decision trees, ensembles, Knn models, etc. I would like to separate my data into training/testing partitions, then have the models train and cross validate using the training data (essentially splitting the training data into training and validation data) while preserving my testing data for error metrics. It is important that the models not be trained in any way using the testing data. For my decision tree, I have something like the following code:</p>

<pre><code>chess = csvread(filename);
predictors = chess(:,1:6);
class = chess(:,7);

cvpart = cvpartition(class,'holdout', 0.3);
Xtrain = predictors(training(cvpart),:);
Ytrain = class(training(cvpart),:);
Xtest = predictors(test(cvpart),:);
Ytest = class(test(cvpart),:);


% Fit the decision tree
tree = fitctree(Xtrain, Ytrain, 'CrossVal', 'on');

% Error Metrics
testingLoss = loss(tree,Xtest,Ytest,'Subtrees','all'); % Testing
resubcost = resubLoss(tree,'Subtrees','all'); % Training
[cost,secost,ntermnodes,bestlevel] = cvloss(tree,'Subtrees','all'); % Cross Val
</code></pre>

<p>However, this returns </p>

<pre><code>Undefined function 'loss' for input arguments of
type 'classreg.learning.partition.ClassificationPartitionedModel'.
</code></pre>

<p>when attempting to find the testing error. I have tried several combinations of similar methods using different types of classification algorithms, but keep coming back to not being able to apply test data to a cross validated model due to partitioned data. How am I supposed to apply test data to a cross validated model?</p>