Auto-scaling ECS Cluster to/from zero instances
<p>I have implemented the <a href="http://en.clouddesignpattern.org/index.php/CDP:Job_Observer_Pattern" rel="noreferrer">Job Observer Pattern</a> using SQS and ECS. Job descriptions are pushed to the SQS queue for processing. The job processing run on an ECS Cluster within an Auto-Scaling Group running ECS Docker Tasks. </p>

<p>Each ECS Task does:</p>

<ol>
<li>Read message from SQS queue  </li>
<li>Execute job on data (~1 hour)</li>
<li>Delete message</li>
<li>Loop while there are more messages</li>
</ol>

<p>I would like to scale down the cluster when there is no more work for each Instance, eventually to zero instances.</p>

<p>Looking at <a href="https://stackoverflow.com/questions/10886471/amazon-auto-scaling-api-for-job-servers">this similar post</a>, the answers suggest scale-in would need to be handled outside of ASG in some way. Instances would self-scale-in, either by explicitly self-terminating or by toggling ASG Instance Protection off when there are no more messages.</p>

<p>This also doesn't handle the case of running multiple ECS Tasks on a single instance, as an individual task shouldn't terminate if other Tasks are running in parallel.</p>

<p>Am I limited to self scale-in and only one Task per Instance? Any way to only terminate once all ECS Tasks on an instance have exited? Any other scale-in alternatives?</p>