Dyego Maas - Blog

Generative AI Consultant and Software Architect

Faster integration tests with Docker and in-memory MongoDB

Faster integration tests with Docker and in-memory MongoDB

Learn how to run integration tests quickly and easily with Docker using the tmpfs file system.

3 min read

The easiest way to set up MongoDB to run in memory is to use a tmpfs file system.

tmpfs file systems don’t store data on permanent storage devices like hard drives, USB sticks, etc. Instead, they handle data directly in RAM, which means they’re volatile. That’s why many Unix distributions use them for the /tmp temporary directory or even for shared memory.

And we can use MongoDB with a tmpfs file system very simply using Docker:

docker run -p 27017:27017 --tmpfs=/data/db mongo:3.6.0

As we can see, all it takes is specifying the directory where the in-memory directory will be mounted, /data/db in this example, and MongoDB takes care of the rest.

To run the tests locally against this MongoDB instance, especially if you already have another instance running, you may need to map this in-memory database to an alternate port. That’s easy to do by changing the external port (the first of the two) with the -p 27018:27017 setting. Once that’s done, just configure the tests to use that port.

Below, an example with Docker Compose:

version: "3"
services:
mongo-in-memory:
  image: mongo:3.6.0
  tmpfs: /data/db
  ports:
    - "27017:27017"

Alternatives to an in-memory database

One alternative is to use some kind of library that emulates MongoDB, so the tests run fast without depending on external tools. One example is mongomock for Python.

This option has some downsides, though. In particular, some indexes may not behave the way they would on a real MongoDB instance. I’ve been in situations where tests managed to violate unique indexes without failing, so I don’t trust this solution much.

Another option is to skip such a library altogether and mock every MongoDB query. That leaves us with a suite full of unit tests.

The biggest downside of this approach is that, by mocking the database calls, our tests end up knowing the internal structure of the code they test. They become coupled to the implementation and turn brittle, also known as anemic tests.

Running tests in a CI pipeline

This way, it’s also very easy to run the integration tests in a continuous integration pipeline. Next, let’s look at an example in Azure DevOps.

One way to use MongoDB in tests with Azure DevOps is to add a DockerCompose task to the job that runs the tests.

steps:
- task: DockerCompose@0
  displayName: 'Turn-on MongoDB InMemory'
  inputs:
    containerregistrytype: 'Container Registry'
    dockerComposeFile: 'mongo-in-memory.yml'
    dockerComposeCommand: 'up --detach'
    workingDirectory: '$(WorkingDirectory)'

# script to run the tests
- script: |
    dotnet test $(ProjectName).sln --configuration Debug

The Docker Compose file mentioned in the example above, mongo-in-memory.yml, is quite simple too:

version: "3"
services:
mongo-in-memory:
  image: mongo:3.6.0
  tmpfs: /data/db
  ports:
    - "27017:27017"

That way, when the tests run, MongoDB will be available. And best of all: the tests will run really fast.