How test API with Snapshot Tests?

Introduction

Testing is a crucial aspect of any software project. Tests are necessary to improve and evolve the project. However, with great power comes great responsibility, and tests should be easy to maintain; otherwise, our development process will slow down.

In this article, we will focus on integration testing, particularly for a REST API endpoint that returns a paginated list.

Classic Assertions Problem

There are several ways to assert the results of an API test, such as hard-coding expected items and asserting one item or hard-coding all items and asserting all items. However, when the API is nested and the contract changes, you will need to rewrite your assertions, and you may need to modify several places.

How to make assertions easier to maintain❓

I would like to highlight the snapshot testing approach (also known as approval testing).

Approval testing is a different approach to asserting results. In snapshot testing, we execute the test for the first time and record the output as a snapshot. We then compare subsequent test results against this snapshot to ensure that the API is still behaving as expected. This approach makes it easier to maintain tests as the contract changes since we only need to update the snapshot instead of multiple assertions.

Let’s skip the theory and proceed step by step.

[UsesVerify]
public class ProductsControllerTests : IClassFixture<WebApplicationFactory<Program>>
{
    [Theory]
    [ClassData(typeof(GetProductTestCases))]
    public async Task GetProducts_ShouldReturnListOfProducts(IReadOnlyCollection<Product> products)
    {
        // Arrange
        await CreateProduct(products);

        // Act
        var response = await _client.GetAsync(ProductsApi);

        // Assert
        response.EnsureSuccessStatusCode();
        var result = await response.Content.ReadFromJsonAsync<List<Product>>();
        Verify(result);
    }
}

Snapshot Testing with [UseVerify] Attribute

In these integration tests, a notable feature is the use of the [UseVerify] attribute in the classes. Instead of employing traditional assertions, the test results are passed to a function called Verify().

To enhance the workflow with snapshot testing, the Rider Plugin (Verify Support) is utilized. As illustrated in the diagram, when a test is executed for the first time, the Verify function is invoked, opening a git diff window.

Reviewing and Approving Test Results

Upon execution of the Verify function, the test results are reviewed. Based on the outcomes, there is the option to either approve them as expected or reject them, initiating an attempt to fix the test. After making necessary corrections, the results can be reviewed and approved again.

Following approval, a text file containing the results is generated, known as a snapshot. For instance, if a change in functionality causes the method to return an empty list instead of a filled one, the test will fail.

Contract Changes and Diffs

When adding a property to the contract, a diff will be opened. Developers can review the changes and approve them if they appear correct. Notably, despite making changes to the functionality, the modification of tests is minimal.

Trade-off Analysis

Pros

• Snapshot testing makes it easier to maintain tests as the contract changes.
• It reduces the number of assertions required for testing.
• It reduces the complexity of tests and improves readability.
• Developers can see the difference between the current test output and the previous one.
• It is more flexible than traditional assertion testing, allowing testing of various scenarios with a single snapshot.

Cons

• Test results may be invalid if developers approve them without careful review.
• Snapshot testing may not be suitable for scenarios involving randomness or unpredictable outputs.
• It may not be as precise as traditional assertion testing, as it only checks if the output has changed, not how it has changed.
• Additional tooling or plugins may be required for effective use, adding complexity to the testing process.

As with most techniques, there are trade-offs. While snapshot testing offers advantages, it is essential to be aware of its limitations.