CMake - run test as part of the build process and capture stdout output to file
<p>We have several unit tests which we would like to be run as part of our build process.</p>

<p>To achieve this I have a helper script which creates a custom command that runs the test, and if successful, creates a file <code>"test_name.passed"</code>.</p>

<p>I then add a custom target <code>"test_name.run"</code> which depends on <code>"test_name.passed"</code>.</p>

<p>The idea is that if <code>"test_name.passed"</code> doesn't exist or is older than <code>"test_name"</code>, the custom command will be run.</p>

<p>Builds will continue to run the custom command until the test passes. Once it passes, subsequent builds won't call the custom command, so the test won't be run when it doesn't need to.</p>

<p><strong>So far this all works exactly as described</strong></p>

<p>Here is the script:</p>

<pre><code># create command which runs the test and creates a sentinel file if it passes
add_custom_command(
    OUTPUT  ${TEST_NAME}.passed
    COMMAND $&lt;TARGET_FILE:${TEST_NAME}&gt;
    COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
    DEPENDS ${TEST_NAME}
    )

# create test.run module which depends on test.passed
add_custom_target(${TEST_NAME}.run
    ALL
    DEPENDS ${TEST_NAME}.passed
    )
</code></pre>

<p><strong>The problem - noise on <code>stdout</code></strong></p>

<p>The problem is that our tests often log a tonne of information to <code>stdout</code>, and it makes for a very noisy build.</p>

<p>I'm trying to now capture <code>stdout</code> to a file, and only in the event of a failure, display the test output.</p>

<p>My first attempt was to try a Bash shell scripting syntax - capture <code>stdout</code> into a file and when the exit status is an error, cat the file.</p>

<pre><code>add_custom_command(
    OUTPUT  ${TEST_NAME}.passed
    COMMAND $&lt;TARGET_FILE:${TEST_NAME}&gt; &gt; ${TEST_NAME}.output || cat ${TEST_NAME}.output
    COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
    DEPENDS ${TEST_NAME}
    )
</code></pre>

<p>This doesn't work, as even if the test fails I am getting the sentinal <code>"test_name.passed"</code> file created, which means the next time I try to build it thinks the test passed.</p>

<p><strong>Possible sub-standard fix</strong></p>

<p>By integrating with <code>ctest</code>, I can run each test through ctest and use the command-line option <code>--output-on-failure</code></p>

<pre><code>add_custom_command(
    OUTPUT  ${TEST_NAME}.passed
    COMMAND ctest --build-config $&lt;CONFIGURATION&gt; --tests-regex ${TEST_NAME} --output-on-failure
    COMMAND ${CMAKE_COMMAND} -E touch ${TEST_NAME}.passed
    DEPENDS ${TEST_NAME}
    )
</code></pre>

<p>The problems with this are two-fold.</p>

<ol>
<li>It <em>vastly</em> increases the build time. Each and every test has to be executed through a separate ctest process, all the registered test names parsed against the regex, etc. With the number of individual tests we have the extra time adds up considerably.</li>
<li>ctest outputs a lot of noise by default. Specifying the <code>--quiet</code> flag suppresses the <code>--output-on-failure</code> flag, so you can either have noisy output or no output - there is no way to only get the failures.</li>
</ol>

<p><strong>Question</strong></p>

<p>Is there a way to achieve what I want? </p>

<p>ie:</p>

<ul>
<li>Run the test manually (ie: not through ctest)</li>
<li>Capture the output to a file</li>
<li>Only output that file in the event of the test exit status indicating failure. </li>
<li>Touch a sentinel file in the event of the test exit status indicating success.</li>
</ul>

<p>Bonus points for a cross-platform method, but if it has to be Linux only, then so be it.</p>