In the past, I've run into trouble running integration tests against docker containers. Let me give you a recent example, I've been working with AWS DynamoDB, when it comes to testing, I don't want to run my integration tests against either a real or a mocked DynamoDB table. It just so happens that AWS provides us with a DynamoDB docker image, this allows us to spin up a local instance of DynamoDB inside a Docker container.

The problem we face is how do we pull down and run a docker container using the DynamoDB docker image from inside our .NET Core application? Thankfully there is a handy library that we can use, that was created by Microsoft called Docker.DotNet.

Let's look at how we can implement this library into our .NET Core application.

A prerequisite is to ensure you have Docker installed on your computer. If you don't head along to the Install Docker page.


Install Docker.DotNet NuGet Package

Inside our application, we want to first install the Docker.DotNet NuGet package. Inside Visual Studio, head along to Package Manager Console and install Docker.DotNet

Install-Package Docker.DotNet -version 3.125.2

At the time of writing this blog post, I am using Docker.DotNet version 3.125.2


Setting up Docker.DotNet Client


Using the Docker.DotNet library we are going to set up our client, this will allow us to interact with the docker engine.  The docker engine is a part of Docker that creates and runs Docker containers.

Once our docker client is setup, we will pull down a docker image. Using the Docker client we will interact with the docker engine to create and run the docker container using the docker image that we have pulled down.

In order to set up our docker client, we need to pass in a URI. This URI allows us to connect to the Docker API, by default the Docker daemon listens on npipe://./pipe/docker_engine for Windows and unix:/var/run/docker.sock for Linux.

The first step in creating our client is to determine what operating system we are running our tests from. We do this so we can pass in the correct URI to our docker client, based on the operating system where the tests are being run from.

private string DockerApiUri()
{
    var isWindows = RuntimeInformation.IsOSPlatform(OSPlatform.Windows);

    if (isWindows)
    {
        return "npipe://./pipe/docker_engine";
    }

    var isLinux = RuntimeInformation.IsOSPlatform(OSPlatform.Linux);

    if (isLinux)
    {
        return "unix:/var/run/docker.sock";
    }

    throw new Exception("Was unable to determine what OS this is running on, does not appear to be Windows or Linux!?");
}

Now that we have a method that determines if we are running our tests from Linux or Windows and assign the appropriately named pipe we can create our docker client.

We add a field to the top of our class of type DockerClient and named _dockerClient. Then in our classes constructor, we new up the DockerClientConfiguration class, this class is from our Docker.DotNet library that we pulled down earlier. Inside the DockerClientConfiguration parameters, we will add a call to the DockerApiUri method that we created above. This will allow us to pass in the correct URI to talk to the docker engine based on what operating system our tests are running from.

 private readonly DockerClient _dockerClient;
 
 public DockerSetup()
 {
     _dockerClient = new DockerClientConfiguration(new Uri(DockerApiUri())).CreateClient();
 }

PullImage

We now have a DockerClient setup allowing us to interact with the Docker Engine or in other words the Docker API, we want to use this client to pull down a docker image. In this example, I'll be pulling down the Amazons DynamoDB Local docker image. Below is an example of what the PullImage method looks like

private async Task PullImage()
{
    await _dockerClient.Images
        .CreateImageAsync(new ImagesCreateParameters
            {
                FromImage = "amazon/dynamodb-local",
                Tag = "latest"
            },
            new AuthConfig(),
            new Progress<JSONMessage>());
}

Let's talk through what is going on inside this method. Using our docker client that we have setup we call off to the CreateImageAsync method. Inside this method, we new up an ImagesCreateParameters. We need to set our docker image name and set the image tag, in our case we set this to 'latest' to grab the latest amazon/dynamoDB docker image.

We are required to set an AuthConfig, this allows us to set a username and password if the docker image we are pulling down requires us to login. In our case, we can pull down the dynamodb-local docker image anonymously so we can leave this empty.

We also need to add the Progress parameter. We can use this to check the progress when pulling down our docker image. For the moment we will leave this empty.

IAsyncLifetime

I'm going to take a moment to speak about the IAsyncLifetime interface. The times when I have needed to spin up a docker container from inside my solution is when running integration tests.

I want to be able to run multiple methods when the class is instantiated, because the PullImage method is asynchronous and as it currently stands, calling asynchronous methods from a class's constructor is tough work to avoid deadlocks. A way to avoid having to write a lot of code that would be needed to correctly call an asynchronous method from within a constructor is by using Xunits IAsyncLifetime interface. This interface contains an InitializeAsync method and a DisposeAsync. The InitializeAsync method will be run straight after the class's constructor and because this method is asynchronous we can call off to other asynchronous methods.

public class DockerSetup : IAsyncLifetime
public async Task InitializeAsync()
{
    await PullImage();
}

Of course, if you are calling these methods from somewhere else you can skip adding the IAsyncLifetime interface completely, but it was worth pointing out how I get around calling asynchronous methods from a constructor inside my integration tests.

Start container

Now that we have our docker image pulled down, we can create our container ready to be run.

In the example below, we are able to use our docker client and use the .Containers.CreateContainer method. Inside CreateContainer we want to new up a CreateContainerParameters then set a few of its properties.

private async Task StartContainer()
{
    var response = await _dockerClient.Containers.CreateContainerAsync(new CreateContainerParameters
    {
        Image = "amazon/dynamodb-local",
        ExposedPorts = new Dictionary<string, EmptyStruct>
        {
            {
                "8000", default(EmptyStruct)
            }
        },
        HostConfig = new HostConfig
        {
            PortBindings = new Dictionary<string, IList<PortBinding>>
            {
                {"8000", new List<PortBinding> {new PortBinding {HostPort = "8000"}}}
            },
            PublishAllPorts = true
        }
    });
}

We are setting the Image property with the Docker image that we downloaded earlier, in our case we are using amazon/dynamodb-local.
We want to set and expose a host port that will allow us to interact with our docker container. We do this by setting the ExposedPorts property in our example we set this to 8000.

We also need to set our container port, we do this by setting the PortBindings property inside the HostConfig.

You will notice that I've set the PublishAllPorts to be true, this is an important field to ensure that all ports you have set are exposed.

Once we have created our container, we grab the container id that is given to us. I've added a field to the top of our class

private string _containerId;

We then add to the bottom of our StartContainer method

_containerId = response.ID;

Finally, using our Docker client we use containers.StartContainerAsync method to start our container, passing in the containerId we set above

await _dockerClient.Containers.StartContainerAsync(_containerId, null);

DisposeAsync

Once we have finished interacting with our docker container, we can use the IAsyncLifetime interface's method DisposeAsync to kill off the container.

In the example below, I'm first checking if a containerId exists if it does then use the Containers.KillContainerAsync method and pass in the containerId. Remember that we set this field at the top of our class.

public async Task DisposeAsync()
{
    if (_containerId != null)
    {
        await _dockerClient.Containers.KillContainerAsync(_containerId, new ContainerKillParameters());
    }
}

Summary

I use Docker containers a lot for my integration tests. Rather than interacting with real AWS services that require access and secret keys, it also requires us to spin up and down real AWS services. This can increase costs and time to run the integration tests. A better way I have found is by using docker images.

In this blog post, I've shown how I use Docker.DotNet to help me spin up these containers when running the tests. Before using Docker.DotNet I would spin the containers up manually before running my tests, but I like to automate as much as possible. This is so myself or someone else working on the solution is able to click run and that's it without any manual intervention.

While the example above shows using the amazon/dynamodb-local docker image, you can use any image you want.

You can find the full working example from my GitHub account