Spark: Add column to dataframe conditionally
<p>I am trying to take my input data:</p>

<pre><code>A    B       C
--------------
4    blah    2
2            3
56   foo     3
</code></pre>

<p>And add a column to the end based on whether B is empty or not:</p>

<pre><code>A    B       C     D
--------------------
4    blah    2     1
2            3     0
56   foo     3     1
</code></pre>

<p>I can do this easily by registering the input dataframe as a temp table, then typing up a SQL query.</p>

<p>But I'd really like to know how to do this with just Scala methods and not having to type out a SQL query within Scala.</p>

<p>I've tried <code>.withColumn</code>, but I can't get that to do what I want.</p>