How to get .NET Core MSTest tests working locally and in Azure DevOps (previously Visual Studio Team Services or VSTS)

mstest_fixture_steps.md

These steps are for getting .NET Core 2.1 MSTest tests to support building up the environment variables that an Azure Function uses both in a local environment and in Azure DevOps CI.

Assumptions

• Need to use live resources for integration testing because Event Hub doesn't have an emulator
  • This means I have connection strings that must be treated like secrets (can't use UseDevelopment=true for everything)
• Can't easily build up my own configuration using ConfigurationBuilder in the application because I'm using Azure Functions (and Webjobs)
  • Can't use something like this
• Need to be able to run locally and in Azure DevOps (VSTS)

appsetting.test.json workaround

Code

// Azure Function
public static class Endpoint
{
    private static string EhConnectionString { get; } =
        Environment.GetEnvironmentVariable("EventHubConnectionString");

    [FunctionName("Endpoint")]
    public static async Task<IActionResult> RunAsync(
        [HttpTrigger(AuthorizationLevel.Function, "post")]HttpRequest request, ILogger log)
    { /* do cool stuff */ }
}

// test fixture
[TestClass]
public class EndpointTests : FunctionTest
{
    [TestInitialize]
    public void Setup()
    {
        var settings = new ConfigurationBuilder()
            .AddJsonFile("appsettings.test.json", false)
            .Build();
        foreach (var item in settings.AsEnumerable())
        {
            Environment.SetEnvironmentVariable(item.Key, item.Value);
        }
    }
    
    // test methods
}

// local appsettings.test.json that is also gitignored
{
  "EventHubConnectionString": "<ConnectionString>"
}

Azure DevOps

1. Added Variable Group that included secrets
2. Add dotnet build task to build definition (after dotnet restore task) with:
   1. "Arguments" field: --no-restore --output $(Build.BinariesDirectory)
3. Add File Creator task to build definition (after dotnet build task) with:
   1. "File path" field: $(Build.BinariesDirectory)\appsettings.test.json
   2. "File content" field:

   {
       "EventHubConnectionString": "$(EventHubConnectionString)",
   }

4. Add dotnet test task to run only integration test project within solution with:
   1. "Arugments" field: --no-restore --no-build --output $(Build.BinariesDirectory)

So final task list was:

1. dotnet restore
2. dotnet build
3. File Creator
4. dotnet test (integration test)

Runsettings argument attempt (FAILED)

RUNSETTING ARGUMENTS DO NOT WORK AT ALL WITH DOTNET TEST (see open issue)

1. Add *.runsettings file containing non-sensitive config to add to environment variables
2. Add test setup code that iterates through TestContext#Properties and sets environment variables

<!--mysettings.runsettings-->
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <TestRunParameters>  
    <Parameter name="AzureWebJobsStorage" value="UseDevelopmentStorage=true" />
    <Parameter name="MyNonSensitiveEnvVarSetting" value="foobar" />  
  </TestRunParameters>  
</RunSettings>

// test fixture
public TestContext TestContext { get; set; }

[TestInitialize]
public void Setup()
{
    foreach (var item in TestContext.Properties)
    {
        Environment.SetEnvironmentVariable(item.Key.Replace(EnvVarPrefix, string.Empty), item.Value.ToString());
    }
}

References

• https://github.com/Microsoft/vstest-docs/blob/e19d62610a78b568183edb0f121f33747d6d2d8c/docs/RunSettingsArguments.md
• https://blogs.msdn.microsoft.com/devops/2015/09/04/supplying-run-time-parameters-to-tests/ (task definition examples are out-of-date)

vsts_steps.md

This is for configuring VSTS hosted build agents and Azure Storage emulators (also CosmosDB).

1. After build task, add Command Line task (I used Version 2.*)
   1. Name it "Initialize Azure Storage Emulator" or something
   2. Add following to the Script field: "C:\Program Files (x86)\Microsoft SDKs\Azure\Storage Emulator\AzureStorageEmulator.exe" init /server "(localdb)\MSSQLLocalDb"
2. After "Initialize Azure Storage Emulator" task, add another Command Line task
   1. Name it "Start Azure Storage Emulator" or something
   2. Add following to the Script field: "C:\Program Files (x86)\Microsoft SDKs\Azure\Storage Emulator\AzureStorageEmulator.exe" start

Note: The Azure Piplines Docker image readme does not mention that the Azure Storage Emulator is installed but these tasks worked for me so it must be. :tongue_stuck_out: Moreover, it does mention that the CosmosDB Emulator is installed.

References

• https://github.com/Microsoft/azure-pipelines-image-generation/issues/165
• https://stackoverflow.com/a/51726177/34315
• https://stackoverflow.com/questions/46345563/failed-to-start-azurestorageemulator-on-vsts-hosted-vs2017-agent#comment85210199_46357767

I know this is not a forum, but anyway I would like to mention my findings about this strategy, focusing on the DevOps part.
As far as I could see, the --no-restore --no-build parameters are not needed.
The --output parameter is only there to make sure all steps work in the same directory. For some reason that I could not resolve, in my solution some project dependencies are not found when I use the explicit output directory. Therefore my solution was to identify the default working directory of my test step and to create the appsettings file right there. Not beautiful, but working.