vstset.console.exe assembly resolution fails
<p>I have a <code>UnitTests.dll</code>, to which <code>Common.dll</code> is referenced (both built with VS2015).</p>

<p>I have following directory structure:</p>

<pre><code>C:\Test\
    - UnitTests.dll
    - UnitTests.runsettings    
C:\Bin\
    - Common.dll
</code></pre>

<p><code>UnitTests.runsettings</code> content is as follows:</p>

<pre><code>&lt;?xml version="1.0" encoding="utf-8"?&gt;
&lt;RunSettings&gt;
    &lt;RunConfiguration&gt;
        &lt;TargetPlatform&gt;x64&lt;/TargetPlatform&gt;
    &lt;/RunConfiguration&gt;
    &lt;MSTest&gt;
        &lt;MapInconclusiveToFailed&gt;True&lt;/MapInconclusiveToFailed&gt;
        &lt;CaptureTraceOutput&gt;False&lt;/CaptureTraceOutput&gt;
        &lt;DeleteDeploymentDirectoryAfterTestRunIsComplete&gt;False&lt;/DeleteDeploymentDirectoryAfterTestRunIsComplete&gt;
        &lt;DeploymentEnabled&gt;False&lt;/DeploymentEnabled&gt;
        &lt;AssemblyResolution&gt;
            &lt;Directory Path="C:\Bin\" includeSubDirectories="true" /&gt;
        &lt;/AssemblyResolution&gt;
    &lt;/MSTest&gt;    
&lt;/RunSettings&gt;
</code></pre>

<hr>

<p>I invoke the tests:</p>

<pre><code>C:\Test&gt; vstest.console.exe UnitTests.dll /settings:UnitTests.runsettings /inIsolation
</code></pre>

<p><code>vstest.console.exe</code> refers to <em>C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe</em>.</p>

<hr>

<p>I receive following error:</p>

<pre><code>Starting test execution, please wait... 
Failed   TestMethod1 
Error Message:
    Test method UnitTests.UnitTest1.TestMethod1 threw exception: System.IO.FileNotFoundException: Could not load file or assembly 'Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.

Stack Trace:
    at UnitTests.UnitTest1.TestMethod1()
</code></pre>

<p>More, with Fusion Log enabled:</p>

<pre><code>Starting test execution, please wait... 
Failed   TestMethod1 
Error Message:
    Test method UnitTests.UnitTest1.TestMethod1 threw exception: System.IO.FileNotFoundException: Could not load file or assembly 'Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.
=== Pre-bind state information === 
LOG: DisplayName = Common, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
  (Fully-specified)
LOG: Appbase = file:///C:/Test 
LOG: Initial PrivatePath = NULL 
Calling assembly : UnitTests, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.
=== LOG: This bind starts in default load context. 
LOG: Using application configuration file: 
C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.executionengine.exe.Config 
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework64\v4.0.30319\config\machine.config. 
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind). 
LOG: Attempting download of new URL file:///C:/Test/Common.DLL. 
LOG: Attempting download of new URL file:///C:/Test/Common/Common.DLL. 
LOG: Attempting download of new URL file:///C:/Test/Common.EXE. 
LOG: Attempting download of new URL file:///C:/Test/Common/Common.EXE.

Stack Trace:
    at UnitTests.UnitTest1.TestMethod1()
</code></pre>

<p>Do I face some kind of a caching issue? How to persuade <code>vstest.console.exe</code> to look for the dependencies inside <code>C:\Bin\</code> (as theoretically pointed by <code>AssemblyResolution</code> element)?</p>

<hr>

<p><strong>UPDATE:</strong></p>

<p><a href="https://connect.microsoft.com/VisualStudio/feedback/details/3105814/vstset-console-exe-assembly-resolution-fails" rel="nofollow">Submitted as a bug to MSFT on connect</a>
(with repro steps - under the DETAILS tab at the bottom).</p>

<p>Existing limitations of assembly resolution, and forcing the usage of <code>DeploymentItem</code> attribute, doesn't scale at all. Redundant maintenance cost (where developers are forced to manually keep track of which dependencies are required for the unit tests to run) introduces friction to the tests. Any issues, which appear as a result of that friction, are very hard to troubleshoot.</p>