Applying a function (mkString) to an entire column in Spark dataframe, error if column name has "."
<p>I'm attempting to apply a function over a column of a Spark dataframe in Scala. The column is a String type, and I'd like to concatenate each token in the string with an "_" delimiter (e.g. "A B" --> "A_B"). I'm doing this with:</p>

<pre><code>val converter: (String =&gt; String) = (arg: String) =&gt; {arg.split(" ").mkString("_")}
val myUDF = udf(converter)
val newDF = oldDF
  .withColumn("TEST", myUDF(oldDF("colA.B")) )
display(newDF)
</code></pre>

<p>This works for columns in the dataframe with names without a dot ("."). However, the dot in the column name "colA.B" seems to be breaking the code and throws the error:</p>

<pre><code>  org.apache.spark.sql.AnalysisException: Cannot resolve column name "colA.B" among (colA.B, col1, col2);
</code></pre>

<p>I suppose a work around would be to rename the column (similar to <a href="https://stackoverflow.com/questions/30359539/accessing-column-names-with-periods-spark-sql-1-3">this</a>), but I'd prefer not to do this. </p>