How can I use sklearn.naive_bayes with (multiple) categorical features?
<p>I want to learn a Naive Bayes model for a problem where the class is boolean. Some of the features are boolean, but other features are categorical and can take on a small number of values (~5).</p>
<p>If all my features were boolean then I would want to use <code>sklearn.naive_bayes.BernoulliNB</code>. It seems clear that <code>sklearn.naive_bayes.MultinomialNB</code> is <em>not</em> what I want.</p>
<p>One solution is to split up my categorical features into boolean features. For instance, if a variable &quot;X&quot; takes on values &quot;red&quot;, &quot;green&quot;, &quot;blue&quot;, I can have three variables: &quot;X is red&quot;, &quot;X is green&quot;, &quot;X is blue&quot;. That violates the assumption of conditional independence of the variables given the class, so it seems totally inappropriate.</p>
<p>Another possibility is to encode the variable as a real-valued variable where 0.0 means red, 1.0 means green, and 2.0 means blue. That also seems totally inappropriate to use GaussianNB (for obvious reasons).</p>
<p>I don't understand how to fit what I am trying to do into the Naive Bayes models that sklearn gives me.</p>
<p>[Edit to explain why I don't think multinomial NB is what I want]:</p>
<p>My understanding is that in multinomial NB the feature vector consists of counts of how many times a token was observed in <code>k</code> iid samples.</p>
<p>My understanding is that this is a fit for document of classification where there is an underlying class of document, and then each word in the document is assumed to be drawn from a categorical distribution specific to that class. A document would have <code>k</code> tokens, the feature vector would be of length equal to the vocabulary size, and the sum of the feature counts would be <code>k</code>.</p>
<p>In my case, I have a number of bernoulli variables, plus a couple categorical ones. But there is no concept of the &quot;counts&quot; here.</p>
<p>Example: classes are people who like or don't like math. Predictors are college major (categorical) and whether they went to graduate school (boolean).</p>
<p>I don't think this fits multinomial since there are no counts here.</p>