Converting pipe-delimited file to spark dataframe to CSV file
<p>I have a CSV file with one single column and the rows are defined as follows : </p>

<pre><code>123 || food || fruit
123 || food || fruit || orange 
123 || food || fruit || apple
</code></pre>

<p>I want to create a csv file with a single column and distinct row values as : </p>

<pre><code>orange
apple
</code></pre>

<p>I tried using the following code :  </p>

<pre><code> val data = sc.textFile("fruits.csv")
 val rows = data.map(_.split("||"))
 val rddnew = rows.flatMap( arr =&gt; {
 val text = arr(0) 
 val words = text.split("||")
 words.map( word =&gt; ( word, text ) )
 } )
</code></pre>

<p>But this code is not giving me the correct result as wanted. <br>
Can anyone please help me with this ?</p>