loading Redshift from S3 (with partitions)
<p>My S3 file (<code>s3://MY_s3/stage/my_s3_file</code>) has a partition on <code>load_dt</code>:</p>

<p>S3 Structure:</p>

<pre><code>-MY_S3
 --stage
  ---my_s3_file
    ----load_dt=2016-02-01
         ----000024
    ----load_dt=2016-02-02
         ----000036
</code></pre>

<p>The actual files are under load_dt partitions like 000024 &amp; 000036.</p>

<p>The COPY command works fine if I am not keeping load_dt in Redshift table, but when I am adding load_dt in Redshift table the COPY command fails due to data error as the input layout &amp; target layout are not matching (with extra load_dt on target).</p>

<p>The hive table which creates the S3 file shows partitioned load_dt at the end.</p>

<p>How to make this COPY command work with load_dt being on target Redshift?</p>

<p>I even tried changing S3 locations to <code>s3://MY_s3/stage/my_s3_file/load_dt</code> but no luck.</p>