Spark, ML, StringIndexer: handling unseen labels
<p>My goal is to build a multicalss classifier.</p>

<p>I have built a pipeline for feature extraction and it includes as a first step a StringIndexer transformer to map each class name to a label, this label will be used in the classifier training step.</p>

<p>The pipeline is fitted the training set.</p>

<p>The test set has to be processed by the fitted pipeline in order to extract the same feature vectors.</p>

<p>Knowing that my test set files have the same structure of the training set. The possible scenario here is to face an unseen class name in the test set, in that case the StringIndexer will fail to find the label, and an exception will be raised.</p>

<p>Is there a solution for this case? or how can we avoid that from happening?</p>