How to Run a JMH Benchmark in Maven Using exec:java Instead of exec:exec?
<p>This <a href="http://javapapers.com/java/java-micro-benchmark-with-jmh/" rel="noreferrer">post on javapapers.com</a> shows how to run a <a href="http://openjdk.java.net/projects/code-tools/jmh/" rel="noreferrer">JMH</a> benchmark in Maven by typing <code>mvn exec:exec</code>. Running JMH within Maven is pretty handy, since you can easily run it from an Eclipse Run Configuration or even in a Maven phase.</p>

<p>However, there are two problems with this setup:</p>

<ol>
<li><p>When you kill Maven, JMH will continue running in the background, as <code>exec:exec</code> starts it in a separate VM.</p></li>
<li><p>Usually, JMH will start <em>yet another</em> VM to run the benchmarks, so you will end up with at least 3 VMs running at the same time.</p></li>
</ol>

<p>Fortunately, the <a href="http://www.mojohaus.org/exec-maven-plugin/" rel="noreferrer">Exec Maven Plugin</a> comes with a second goal, <code>exec:java</code>, which executes a main class directly in the VM Maven runs. However, when I tried to configure Maven to run JMH using <code>exec:java</code>, the benchmark crashes because of missing classes:</p>

<pre><code># JMH 1.11.3 (released 40 days ago)
# VM version: Error: Could not find or load main class org.openjdk.jmh.runner.VersionMain
# VM invoker: C:\Program Files\Java\jdk1.7.0\jre\bin\java.exe
[...]
# Run progress: 0.00% complete, ETA 00:02:40
# Fork: 1 of 1
Error: Could not find or load main class org.openjdk.jmh.runner.ForkedMain
&lt;forked VM failed with exit code 1&gt;
</code></pre>

<p>Here is the relevant part of the <code>pom.xml</code>:</p>

<pre><code>&lt;plugin&gt;
    &lt;groupId&gt;org.codehaus.mojo&lt;/groupId&gt;
    &lt;artifactId&gt;exec-maven-plugin&lt;/artifactId&gt;
    &lt;version&gt;1.4.0&lt;/version&gt;
    &lt;configuration&gt;
        &lt;mainClass&gt;my.Benchmark&lt;/mainClass&gt;
    &lt;/configuration&gt;
&lt;/plugin&gt;
</code></pre>

<p>And here is how I run JMH from <code>my.Benchmark</code>:</p>

<pre><code>public static void main(String[] args) throws RunnerException {
    Options options = new OptionsBuilder().include(my.Benchmark.class.getSimpleName())
            .forks(1).build();
    new Runner(options).run();
}
</code></pre>

<p>I realize that JMH uses the <code>java.class.path</code> system property to determine the classpath for the forked VMs and that this property does not contain Maven's project dependencies. But what is the preferred way to deal with this?</p>