why hbase KeyValueSortReducer need to sort all KeyValue
<p>I am learning Phoenix CSV Bulk Load recently and I found that the source code of <code>org.apache.phoenix.mapreduce.CsvToKeyValueReducer</code> will cause OOM ( java heap out of memory ) when columns are large in one row (In my case, 44 columns in one row and the avg size of one row is 4KB). </p>

<p>What's more, this class is similar with the hbase bulk load reducer class - <code>KeyValueSortReducer</code>. It means that OOM may happen when using <code>KeyValueSortReducer</code> in my case. </p>

<p>So, I have a question of <code>KeyValueSortReducer</code> - <strong>why it need to sort all kvs in treeset first and then write all of them to context? If I remove the treeset sorting code and wirte all kvs directly to the context, the result will be different or be wrong ?</strong></p>

<p>I am looking forward to your reply. Best wish to you!</p>

<p>here is the source code of KeyValueSortReducer:</p>

<pre><code>public class KeyValueSortReducer extends Reducer&lt;ImmutableBytesWritable, KeyValue, ImmutableBytesWritable, KeyValue&gt; {
  protected void reduce(ImmutableBytesWritable row, java.lang.Iterable&lt;KeyValue&gt; kvs,
      org.apache.hadoop.mapreduce.Reducer&lt;ImmutableBytesWritable, KeyValue, ImmutableBytesWritable, KeyValue&gt;.Context context)
  throws java.io.IOException, InterruptedException {
    TreeSet&lt;KeyValue&gt; map = new TreeSet&lt;KeyValue&gt;(KeyValue.COMPARATOR);
    for (KeyValue kv: kvs) {
      try {
        map.add(kv.clone());
      } catch (CloneNotSupportedException e) {
        throw new java.io.IOException(e);
      }
    }
    context.setStatus("Read " + map.getClass());
    int index = 0;
    for (KeyValue kv: map) {
      context.write(row, kv);
      if (++index % 100 == 0) context.setStatus("Wrote " + index);
    }
  }
}
</code></pre>