Multi-label feature selection using sklearn
<p>I'm looking to perform feature selection with a multi-label dataset using sklearn. I want to get the final set of features <strong>across</strong> labels, which I will then use in another machine learning package. I was planning to use the method I saw <a href="https://stackoverflow.com/questions/16400722/feature-selection-for-multilabel-classification-scikit-learn/16421186#16421186">here</a>, which selects relevant features for each label separately.</p>

<pre><code>from sklearn.svm import LinearSVC
from sklearn.feature_selection import chi2, SelectKBest
from sklearn.multiclass import OneVsRestClassifier
clf = Pipeline([('chi2', SelectKBest(chi2, k=1000)),
                ('svm', LinearSVC())])
multi_clf = OneVsRestClassifier(clf)
</code></pre>

<p>I then plan to extract the indices of the included features, per label, using this:</p>

<pre><code>selected_features = []
for i in multi_clf.estimators_:
    selected_features += list(i.named_steps["chi2"].get_support(indices=True))
</code></pre>

<p>Now, my question is, how do I choose which selected features to include in my final model? I could use every unique feature (which would include features that were only relevant for one label), or I could do something to select features that were relevant for more labels.  </p>

<p>My initial idea is to create a histogram of the number of labels a given feature was selected for, and to identify a threshold based on visual inspection. My concern is that this method is subjective. Is there a more principled way of performing feature selection for multilabel datasets using sklearn?</p>