Channels ▼


Static Testing C++ Code

Code Snippet Markers

More Insights

White Papers

More >>


More >>


More >>

Assume you want to test the legality of 10 statements. If you put them all in the same program, you face the following difficulty: If the program can be compiled with no errors, then you infer that all 10 statements are syntactically correct. But if the compilation generates errors, it is difficult to ascertain which of the statements are syntactically incorrect. It could even be that all the statements are correct when considered one at a time, but their coexistence in the same program is incorrect.

A better method would be to:

  • Prepare the source code of a syntactically correct program that doesn't contain the statements to test.
  • Create 10 copies of this program, inserting into every copy one of the statements to test, obtaining in this way 10 different programs.
  • Separately compile the 10 programs. The wrong statements are the ones that made the original program no more suitable to compilation.

This approach is effective, but laborious for users. Therefore, it is convenient to automate it.

For such purposes, you can insert the 10 statements in the same program, and submit the source code to a utility that generates the 10 versions of the source code, one for every statement to test. Of course, to let the utility know which statements to test, those statements should be adequately marked. In fact, they generally aren't simple statements but code snippets containing one or more statements.

In addition, such marking must specify whether the pointed-out code snippets are declared as legal or as illegal. Listing One is an example of a static-check program containing four code snippets. Three markers are used (nesting is not allowed):

  • @LEGAL marks the beginning of a code snippet declared as legal.
  • @ILLEGAL marks the beginning of a code snippet declared as illegal.
  • @END marks the end of the current code snippet.

The example program includes statements common to every test. After these statements, there is a block of two statements declared as legal, the declaration of a vector of complex numbers, and a call to the min function with two integer numbers as parameters. These statements are actually legal.

Listing One

#include <complex>
#include <vector>
#include <algorithm>
#include <iostream>
int main() {
    std::vector<std::complex<double> > v;
    std::min(2, 3);
    sort(v.begin(), v.end());
    std::min(2, 3.);
    int i;
    std::cout << i;
    std::vector<int> vi;
    sort(vi.begin(), vi.end());

Afterwards, there is a block of two statements declared as illegal. In fact, only the second statement is illegal, as it tries to sort complex numbers. The v variable is declared twice. This is not an error, as when the first code snippet is active, the second code snippet is inactive, and vice versa; therefore, the two declarations do not coexist.

Next, there is a code snippet containing a single statement that is declared as illegal. It is really illegal because it tries to compute the lesser between an int value and a double value.

Finally, there is a code snippet of seven statements, declared as legal. Actually, this code snippet does not contain syntax errors, but the second line prints the value of a variable that has never received a value. In such a situation, many compilers emit warnings if the needed compiling options are set.

This raises another issue: If a code snippet declared as legal causes the compiler to emit a warning, is the test to be considered successful or failed? And if a code snippet declared as illegal causes the compiler to emit a warning but no errors, is the test to be considered successful or failed?

In a way, in both cases the test is partially successful. Consequently, there is a third possible outcome of a test: In addition to failure and success, there can be a warning.

A Static Testing Utility

I present here a utility that implements static testing automation. The utility is a standard C++ program that retrieves from the command line the paths of the files containing the static testing programs, and accesses some configuration environment variables. (The source code is available here.)

The program reads the files to process, and for each one (and for every marked snippet contained in the file), it generates a new source file whose syntactic correctness is then checked.

To check the syntactic validity of a statement, the most efficient technique would be to use a library providing a compilation service or a static-analysis service. Such libraries do not exist for C++ (or they have little popularity). Alternatively, a compiler can be launched. If a compilation is successful, the process usually sets the return code to zero; if it fails, it sets the return code to another value. So, it is enough to execute the compiler, passing to it the desired options and the source code to check, wait for the completion of the compilation, and check the return code of the process. This is what this utility does.

Some compilers allow an option that lets you perform only a syntax check on the code without generating the executable code, thereby speeding up the compilation. A further speedup can be obtained by disabling code optimization. Yet, some warnings are not output if a simple syntax check is performed or if code optimizations are disabled. Therefore, it is advisable to actually generate the optimized executable program, even if it is never executed. Additionally, there isn't a standard technique to launch a compiler—every compiler has its incompatible options. Therefore, I put the dependencies from the compiler and the platform into environment variables.

Included with the utility's download package are two scripts—one for the Windows command interpreter (CMD.EXE) that tests the program with the Visual C++ and GCC compilers, and the other for UNIX/Linux shells that tests the same program with the GCC compiler. Figure 1 is the execution output of the script for Windows.

==== Visual C++ test ====
Processing test file 'example.cpp'
Running "cl /nologo /GX /Za /Ox /W4 /WX _test1.cpp 2>NUL >NUL"
OK: Compilation with no errors nor warnings, as expected.
Running "cl /nologo /GX /Za /Ox _test2.cpp 2>NUL >NUL"
OK: Compilation with errors, as expected.
Running "cl /nologo /GX /Za /Ox _test3.cpp 2>NUL >NUL"
OK: Compilation with errors, as expected.
Running "cl /nologo /GX /Za /Ox /W4 /WX _test4.cpp 2>NUL >NUL"
Running "cl /nologo /GX /Za /Ox _test4.cpp 2>NUL >NUL"
*** WARNING: Compilation with no errors but some warnings, while a compilation 
*** with no errors nor warnings was expected.
Failures: 0, warnings: 1, successes: 3

One test file processed.
Total failures: 0, total warnings: 1, total successes: 3.

Figure 1: Execution output of the script for Windows.

Carlo is a CAD/CAM software developer in Bergamo, Italy. You can contact him at

Related Reading

Currently we allow the following HTML tags in comments:

Single tags


<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags


<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

