How to set partition for Window function for PySpark?
<p>I'm running a PySpark job, and I'm getting the following message:</p>

<pre><code>WARN org.apache.spark.sql.execution.Window: No Partition Defined for Window operation! Moving all data to a single partition, this can cause serious performance degradation.
</code></pre>

<p>What does the message indicate, and how do I define a partition for a Window operation?</p>

<p>EDIT:</p>

<p>I'm trying to rank on an entire column.</p>

<p>My data is organized as:</p>

<pre><code>A
B
A
C
D
</code></pre>

<p>And I want:</p>

<pre><code>A,1
B,3
A,1
C,4
D,5
</code></pre>

<p>I don't think there should by a .partitionBy() for this, only .orderBy(). The trouble is, this appears to cause performance degradation. Is there another way to achieve this without a Window function?</p>

<p>If I partition by the first column, the result would be:</p>

<pre><code>A,1
B,1
A,1
C,1
D,1
</code></pre>

<p>Which I do not want.</p>