Effect of location of MPI_Init on the speed of serial code
<p>I have a code on cluster which is managed by SLURM. The code consists of   serial and parallel functions. Serial functions are called right after main and then parallel functions follow. I realized that when I initialize MPI at the beginning, right after main, the code slows down enormously for serial functions. But initializing MPI right before parallel functions and only after serials finish makes the code run at normal speed. I tried several options of SLURM as follows but they do not change the result. What I thought was if there are 5 nodes and 5 tasks then the speed should be more or less the same for serial functions. I would like to know why this happens.</p>

<pre><code>int main(int argc, char** argv)
{
    MPI_Init (&amp;argc, &amp;argv);   

    int rank, n_procs;
    MPI_Comm world = MPI_COMM_WORLD;
    MPI_Comm_rank (world, &amp;rank);
    MPI_Comm_size (world, &amp;n_procs);

    // serial functions
    // parallel functions
}
</code></pre>

<p>Script 1:</p>

<pre><code>#!/bin/bash
#SBATCH -M truba 
#SBATCH -p mid1
#SBATCH -A a
#SBATCH -J a-test
#SBATCH -N 5
#SBATCH -t 3:00:0
#SBATCH --output=slurm-%j.out
#SBATCH --error=slurm-%j.err

module load centos6.4/lib/openmpi/1.8.8-gcc-5.1.0

mpirun ./out
exit
</code></pre>

<p>Script 2:</p>

<pre><code>#!/bin/bash
#SBATCH -M truba 
#SBATCH -p mid1
#SBATCH -A a
#SBATCH -J a-test
#SBATCH -N 5
#SBATCH -n 5
#SBATCH -c 16
#SBATCH -t 3:00:0
#SBATCH --output=slurm-%j.out
#SBATCH --error=slurm-%j.err

module load centos6.4/lib/openmpi/1.8.8-gcc-5.1.0

mpirun ./out
exit
</code></pre>