Chain automated builds in the same Docker Hub repository
<p>Due to build <a href="https://stackoverflow.com/a/34588866/865719">time restrictions</a> on Docker Hub, I decided to split the <code>Dockerfile</code> of a time-consuming automated build into three files.
Each one of those &quot;sub-builds&quot; finishes within Docker Hub's time limits.</p>
<p>I have now the following setup within the same repository:</p>
<pre><code>| branch | dockerfile         | tag    |
| ------ | ------------------ | ------ |
| master | /step-1.Dockerfile | step-1 |
| master | /step-2.Dockerfile | step-2 |
| master | /step-3.Dockerfile | step-3 |
</code></pre>
<p>The images build on each other in the following order:</p>
<ul>
<li><code>step-1.Dockerfile</code> : <code>FROM ubuntu</code></li>
<li><code>step-2.Dockerfile</code> : <code>FROM me/complex-image:step-1</code></li>
<li><code>step-3.Dockerfile</code> : <code>FROM me/complex-image:step-2</code></li>
</ul>
<p>A separate web application triggers the building of <code>step-1</code> using the &quot;build trigger&quot; URL provided by Docker Hub (to which the <code>{&quot;docker_tag&quot;: &quot;step-1&quot;}'</code> payload is added). However, Docker Hub doesn't provide a way to automatically trigger <code>step-2</code> and then <code>step-3</code> afterwards.</p>
<p>How can I automatically trigger the following build steps in their respective order?** (i.e., trigger <code>step-2</code> after <code>step-1</code> finishes. Then, trigger <code>step-3</code> after <code>step-2</code> finishes).</p>
<p><strong>NB: I don't want to set up separate repositories for each of <code>step-i</code> then link them using Docker Hub's &quot;Repository Links.&quot;</strong> I just want to link <strong>tags</strong> in the same repository.</p>
<p><strong>Note:</strong> Until now, my solution is to attach a Docker Hub Webhook to a web application that I've made. When <code>step-n</code> finishes, (i.e., calls my web application's URL with a JSON file containing the tag name of <code>step-n</code>) the web application uses the &quot;build trigger&quot; to trigger <code>step-n+1</code>. It works as expected, however, I'm wondering whether there's a &quot;better&quot; way of doing things.</p>
<p><a href="https://stackoverflow.com/a/36949444/865719">As requested by Ken Cochrane</a>, here are the <a href="https://github.com/maddouri/cling-ubuntu-docker/blob/175dcc183312e93f7b6b9da826a973bf1c3d880c/Dockerfile" rel="nofollow noreferrer">initial <code>Dockerfile</code></a> as well as the <a href="https://github.com/maddouri/cling-ubuntu-docker/blob/175dcc183312e93f7b6b9da826a973bf1c3d880c/build-cling.sh" rel="nofollow noreferrer">&quot;build script&quot;</a> that it uses. I was just trying to dockerize <a href="https://root.cern.ch/cling" rel="nofollow noreferrer">Cling</a> (a C++ interpreter). It needs to compile <a href="https://en.wikipedia.org/wiki/LLVM" rel="nofollow noreferrer">LLVM</a>, <a href="https://en.wikipedia.org/wiki/Clang" rel="nofollow noreferrer">Clang</a> and <a href="https://en.wikipedia.org/wiki/CINT" rel="nofollow noreferrer">Cling</a>. As you might expect, depending on the machine, it needs a few hours to do so, and Docker Hub allows &quot;only&quot; 2-hour builds at most :) The &quot;sub build&quot; images that I added later (still in the <code>develop</code> branch) build a part of the whole thing each. I'm not sure that there is any further optimization to be made here.</p>
<p>Also, in order to test various ideas (and avoid waiting h-hours for the result) I have setup <a href="https://hub.docker.com/r/maddouri/void-docker" rel="nofollow noreferrer">another repository</a> with a similar structure (the only difference is that its <code>Dockerfile</code>s don't do as much work).</p>
<p><strong>UPDATE 1:</strong> On <a href="https://stackoverflow.com/a/36949444/865719">Option 5</a>: as expected, the <code>curl</code> from <code>step-1.Dockerfile</code> has been ignored:</p>
<p><em>Settings</em> → <em>Build Triggers</em> → <em>Last 10 Trigger Logs</em></p>
<pre><code>| Date/Time                 | IP Address      | Status  | Status Description       | Request Body               | Build Request |
| ------------------------- | --------------- | ------- | ------------------------ | -------------------------- | ------------- |
| April 30th, 2016, 1:18 am | &lt;my.ip.v4.addr&gt; | ignored | Ignored, build throttle. | {u'docker_tag': u'step-2'} | null          |
</code></pre>
<p>Another problem with this approach is that it requires me to put the build trigger's (secret) token in the <code>Dockerfile</code> for everyone to see :) (hopefully, Docker Hub has an option to invalidate it and regenerate another one)</p>
<p><strong>UPDATE 2:</strong> Here is <a href="https://github.com/maddouri/dockerhub-trigger-propagate" rel="nofollow noreferrer">my current attempt</a>:
It is basically a <a href="https://www.heroku.com/" rel="nofollow noreferrer">Heroku</a>-hosted application that has an <a href="https://apscheduler.readthedocs.io/en/v3.0.5/index.html" rel="nofollow noreferrer">APScheduler</a> periodic &quot;trigger&quot; that starts the initial build step, and a <a href="http://flask.pocoo.org/" rel="nofollow noreferrer">Flask</a> webhook handler that &quot;propagates&quot; the build (i.e., it has the ordered list of build tags. Each time it is called by the webhook, it triggers the next build step).</p>