x64 inline assembly in c to align instructions
<p>I've got a very hot instruction loop which needs to be properly aligned on 32-bytes boundaries to maximize <a href="http://www.anandtech.com/show/6355/intels-haswell-architecture/6" rel="noreferrer">Intel's Instruction Fetcher</a> effectiveness.</p>

<p>This issue is specific to Intel not-too-old line of CPU (from Sandy Bridge onward). Failure to align properly the beginning of the loop results in up to 20 % speed loss, which is definitely too noticeable.
This issue is pretty rare, one needs a highly optimized set of instructions for the instruction fetcher to become the bottleneck. But fortunately, it's not a unique case. Here is a <a href="http://pzemtsov.github.io/2014/05/12/mystery-of-unstable-performance.html" rel="noreferrer">nice article explaining in details</a> how such a problem can be detected.</p>

<p>The problem is, gcc nor clang would care aligning properly this instruction loop. It makes compiling this code a nightmare producing random outcome, depending on how "good" the hot loop is aligned by chance. It also means that modifying a totally unrelated function can nonetheless highly impact performance of the hot loop.</p>

<p>Already tried several compiler flags, none of them gives satisfying result.</p>

<p><em>[Edit]</em> More detailed description of tried compilation flags :</p>

<ul>
<li><code>-falign-functions=32</code> : no impact or negative impact</li>
<li><code>-falign-jumps=32</code> : no impact</li>
<li><code>-falign-loops=32</code> : works fine when the hot loop is isolated into a tiny piece of test code. But in normal build, the compilation flag is applied across the entire source, and in this case it is detrimental : aligning <em>all</em> loops on 32-bytes is bad for performance. Only the very hot ones benefit from it.</li>
<li>Also attempted to use <code>__attribute__((optimize("align-loops=32")))</code> in the function declaration. Doesn't produce any effect (identical binary generated, as if the the statement wasn't there). Later confirmed by <code>gcc</code> support team to be effectively ignored. <em>Edit</em> : @Jester indicates in comment that the statement works with gcc 5+. Unfortunately, my dev station uses primarily gcc 4.8.4, and this is more a problem of portability, since I don't control the final compiler used in the build process.</li>
</ul>

<p>Only building using PGO can reliably produce expected performance, but PGO cannot be accepted as a solution since this piece of code will be integrated into other programs using their own build chain.</p>

<p>So, I'm considering inline assembly.
This would be specific to x64 instruction set, so no portability required.</p>

<p>If my understanding is correct, assembly like NASM allows the use of statements such as : <code>ALIGN 32</code> which would force the next instruction to be aligned on 32 bytes boundaries.</p>

<p>Since the target source code is in C, it would be necessary to include this statement. For example, something like <code>asm("ALIGN 32");</code>
(which of course doesn't work).</p>

<p>I hope it's mostly a matter of knowing the right instruction to write, and not something deeper such as "it's impossible".</p>