Spark 1.5.2: NaN while calculating stddev
<p>I am getting NaN while calculating standard deviation (stddev). This is a very simple use case as described below:</p>

<pre class="lang-scala prettyprint-override"><code> val df = Seq(("1",19603176695L),("2", 26438904194L),("3",29640527990L),("4",21034972928L),("5", 23975L)).toDF("v","data")
</code></pre>

<p>I have the stddev defined as an UDF: </p>

<pre class="lang-scala prettyprint-override"><code>def stddev(col: Column) = {
        sqrt(mean(col*col) - mean(col)*mean(col))
 }
</code></pre>

<p>I'm getting <code>NaN</code> when I call the UDF as shown below:</p>

<pre class="lang-scala prettyprint-override"><code>df.agg(stddev(col("data")).as("stddev")).show() 
</code></pre>

<p>It produces the following: </p>

<pre><code>+------+
|stddev|
+------+
|   NaN|
+------+
</code></pre>

<p>What am I doing wrong?</p>