Various TimeOut Methods on TestNG are not working
<p>A bit of a broad problem so bear with me.</p>

<p>I've recently inherited a testing suite (java, selenium, testng) that is completely external to the product it is testing.  The suite has a major issue with tests running over time (one case hit 9 hours recently on what should be a 90 second test case).</p>

<p>To remedy this issue, I have been trying to enforce time-outs on the tests, but I have had no luck.  A quick overview of the build:</p>

<p>We build and test using maven, </p>

<pre><code>mvn clean test -Dsuite=debug -Dbrowser=firefox -Dmakescreen=NO
</code></pre>

<p>and designate the test suite(s) to be run using an xml doc like so:</p>

<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE suite SYSTEM 'http://testng.org/testng-1.0.dtd'&gt;
&lt;suite name="Test Debugging" verbose="1"&gt;
    &lt;suite-files&gt;
             &lt;suite-file path="suites/DebugTest.xml"/&gt;
    &lt;/suite-files&gt;
&lt;/suite&gt;
</code></pre>

<p>DebugTest.xml looks something like:</p>

<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;!DOCTYPE suite SYSTEM 'http://testng.org/testng-1.0.dtd'&gt;
&lt;suite name="SUITENAME" verbose="1"&gt;
    &lt;test name="TESTNAME"&gt;
        &lt;parameter name="FileName" value="filename.csv" /&gt;
        &lt;classes&gt;
            &lt;class name="tests.testpath" /&gt;
        &lt;/classes&gt;
    &lt;/test&gt;
&lt;/suite&gt;
</code></pre>

<p>Before changes, I've confirmed the tests I am working with run in roughly 10 seconds and pass.</p>

<p>Each test follows the following format:</p>

<pre><code>public class TestName extends BaseTest{

@Override
public void runTest(){
    MainPage mainPage = new MainPage();
    //test does various things
    mainPage.assertCoreElements();
}
</code></pre>

<p>You'll see I am extending a Basetest Class and overriding the runtest method.  The BaseTest Looks like:</p>

<pre><code>public abstract class BaseTest extends BaseEntity implements IAnalys {


/**
 * To override.
 */
public abstract void runTest();

/**
 * Test
 *
 * @throws Throwable Throwable
 */
@Test
public void xTest(ITestContext ctx) throws Throwable {
    System.setProperty("location", "");
    Class&lt;? extends BaseTest&gt; currentClass = this.getClass();
    String testRailTestId = TestRailOperations.getTestIdByTitleInRun(testRailHostAddress,  testRailLogin,  testRailPassword,  testRailRunId,  ctx.getName());
    checkJiraIssue(currentClass); //not an jira extended method
    checkJiraIssue(currentClass.getMethod("runTest", null)); //not an jira extended method

    try {
        processBugAnnotation();
        logger.logTestName(currentClass.getName());
        zephyrExecution();
        runTest();
        logger.logTestEnd(currentClass.getName());
        TestRailOperations.setResultToTest(lastStep, testRailHostAddress,  testRailLogin,  testRailPassword, testRailTestId, TestRailOperations.TestResult.PASSED, "");
        zephyrTestPass();
    } catch (Exception | AssertionError e) {
        logger.debug(this, e);
        if (shouldAnalys()) {
            invokeAnalys(e, getBrowser().getDriver().getPageSource());
        } else {
            TestRailOperations.setResultToTest(lastStep, testRailHostAddress,  testRailLogin,  testRailPassword, testRailTestId, TestRailOperations.TestResult.FAILED, Logger.fatalMessage);
            Logger.fatalMessage = "No info";
            zephyrTestFail();
            logger.warn("");
            logger.warnRed(getLoc("loc.test.failed"));
            logger.warn("");
            throw e;
        }
    } finally {
        if (Browser.getAnalyzeTraffic()) {
            getBrowser().assertAnalyzedTrafficResponseCode();
            getBrowser().assertAnalyzedTrafficLoadTime();
            ProxyServ.saveToFile(this.getClass().getName());
        }
        makeScreen(currentClass);
        printAllFormAppearingTimes();
        clearFormAppearingTimes();
        setIsFirstTimeLoadMainPage(true);
    }
}
</code></pre>

<p>I also have a before method in the parent class that gets the browser and moves us to the starting url, and an after class that quits the browser.</p>

<p><strong>My Goal</strong></p>

<p>I want to configure these tests to timeout at a given limit (initially something like 10ms to prove it works).</p>

<p><strong>What I've Tried</strong></p>

<ol>
<li>@Test(timeout=10000) at the xTest method declaration:  </li>
</ol>

<p>This was the first thing I tried and it worked...sort of.  The test in question did time out at the right point, but it would also spawn a separate thread running independently of the original test.  This thread would open a browser, then terminate without closing the browser, leaving whatever machine I was working on with hundreds of open windows after a given run.  Not ideal!  I've checked my after class method, and it clearly tries to close all browsers (unsuccessfully here).</p>

<p>Some of my reading indicates that a second thread of control is being spawned independent of the before and after methods, which is responsible for my hanging windows.</p>

<ol start="2">
<li>@Test(timeout=10000) on runtest method in the test class</li>
</ol>

<p>this is not ideal because I'd have to copy paste to every single test in the suite, but gave it a try anyway.  This does work, but again seems to have issues with a second thread.  If the original test completes in time a second browser window opens, but nothing happens in it because it doesn't seem to be hitting any code in the before method, then fails when I assert the page loaded.</p>

<ol start="3">
<li>@Rule Timeout at both Basetest and test level</li>
</ol>

<p>To get around the issues mentioned above I followed the documentation here: <a href="https://github.com/junit-team/junit/wiki/Timeout-for-tests" rel="nofollow">https://github.com/junit-team/junit/wiki/Timeout-for-tests</a></p>

<p>This was completely ineffective, using both</p>

<pre><code>`@Rule public Timeout globalTimeout = Timeout.seconds(10);`
</code></pre>

<p>and</p>

<pre><code>`@ClassRule public Timeout globalTimeout = Timeout.seconds(10);`
</code></pre>

<p>Would not cause tests to timeout, event when I set the time to 1 second on a 30 second run.</p>

<ol start="4">
<li><p>I've also tried setting the timeout value in the xml suite:</p>

<p><code>&lt;suite name="Test Debugging" verbose="1" time-out="200"&gt;</code></p></li>
</ol>

<p>Again with no measurable impact even at smallest possible time slice.</p>

<p>At this point I am truly at a loss -- I either need a way to handle the extra threads that are spawning with the @test annotations, or I need to figure out why instructions from the other methods are not being followed.</p>

<p>Thanks for the help,</p>

<p>Joe</p>

<p>EDIT:  Supplying Code for before and after functions</p>

<pre><code>@BeforeClass
public void before() {
    Browser browser = getBrowser();
    browser.windowMaximise();
    long before = new Date().getTime();
    browser.navigate(browser.getStartBrowserURL());
    long openTimeFirstPage = new Date().getTime() - before;
    listFormAppearingTime.add("Main Page=" + String.valueOf(openTimeFirstPage));
    SikuliElement allowGeolocationChrome = new SikuliElement("ALLOW_CHROME.png", 5);
    SikuliElement allowGeolocationIE = new SikuliElement("ALLOW_IE.png", 5);
    if (Browser.currentBrowser.equals(Browsers.CHROME)) {
        if (allowGeolocationChrome.isPresent()) {
            allowGeolocationChrome.click();
        }
    }
    if (Browser.currentBrowser.equals(Browsers.IEXPLORE)) {
        if (allowGeolocationIE.isPresent()) {
            allowGeolocationIE.click();
        }
    }
    stepNumber = 1;
    if (Browser.getAnalyzeTraffic()) {
        browser.startAnalyzeTraffic();
    }
    if ("true".equalsIgnoreCase(System.getProperty(ZEPHYR_CONNECTION_KEY, "false"))) {
        try {
            reporter = new ZephyrJiraReporter();
            newClass = true;
            if (reporter.report()) {
                reporter.createTestCycle();
            }
        } catch (Exception e) {
            logger.debug(this, e);
            logger.error(e.getMessage());
        }
    }
}


/**
 * Close browser after each test Class
 * And update Jira-Zephyr test cycle
 */
@AfterClass
public void after() {
    ProxyServ.stopProxyServer();
    if (getBrowser().isBrowserAlive()) {
        if ("true".equalsIgnoreCase(System.getProperty(ZEPHYR_CONNECTION_KEY, "false"))) {
            updateTestCycle();
        }
        getBrowser().exit();
    }
}
</code></pre>