PySpark count values by condition
<p>I have a DataFrame, a snippet here:</p>

<pre><code>[['u1', 1], ['u2', 0]]
</code></pre>

<p>basically a string field named <code>f</code> and either a 1 or a 0 for second element (<code>is_fav</code>).</p>

<p>What I need to do is grouping on the first field and counting the occurrences of 1s and 0s. I was hoping to do something like</p>

<pre><code>num_fav = count((col("is_fav") == 1)).alias("num_fav")

num_nonfav = count((col("is_fav") == 0)).alias("num_nonfav")

df.groupBy("f").agg(num_fav, num_nonfav)
</code></pre>

<p>It does not work properly, I get in both cases the same result which amounts to the count for the items in the group, so the filter (whether it is a 1 or a 0) seems to be ignored. Does this depend on how <code>count</code> works?</p>