Application Programming Interfaces (APIs), or for the context of this article, Web APIs, are the backbone of most modern web applications. It’s within these APIs where most of the critical business logic lives. Be that complex algorithms, retrieving and storing of data or integrating with other systems, they are the brains of the application. As systems become more distributed, businesses are supporting multiple entry points into their systems such as web, mobile, API to API and even Model Context Protocol (MCP) servers, therefore having a functional stable API is crucial to business operations. Testing, and more specifically, Automated Testing, can play a significant part in achieving this stability. However, there are more approaches we can explore beyond just sending some data and asserting a response.
We’re going to explore four key types of automated API testing: Functional Testing, Contract Testing, Integration Testing, and Mock Testing. We'll look at their purposes, the risks they help mitigate, and how they each contribute to an holistic API testing strategy.
Functional Testing
Functional testing is by far the most common approach we see to automated API testing. It focuses on validating whether an API is functioning as expected, commonly by sending some data to the API and asserting against the response. These assertions usually always involve checking the status code, and ideally will check the body of response when present.
It’s important here to understand the system we are testing, and how it’s architected. It’s common to hear people explain an API as the middle layer between an user interface and the database. This is true, but for me I like to distinguish between the API and the internals of the API, something I refer to as the service layer. Most API implementations are just a library or a wrapper, such as Express.JS that wraps our application code, but importantly handles the routing, authentication and creation of an HTTP endpoint for us. We have to use specific markup and functions of the API library to ensure our APIs function properly, but those libraries are not responsible for our business logic or data storing, that happens in the internals, the service layer.
Most user interfaces show, collect and store data from the user, which they then arrange in the required data structure and send to an API. When we do functional API testing we are bypassing that UI layer, and arranging the data ourselves. This allows us to get closer to the risk we are trying to mitigate, typically, business logic. Depending on your application, we will still be calling the downstream parts of the system such as the database and other APIs, but we’ve removed the UI layer, which is not the focus of our test.
We must acknowledge that we are still testing a large part of our system, and while the term end-to-end is traditionally associated with UI tests, I often refer to functional APIs tests as end-to-end API tests, as they are still traversing a lot of our application. That said they are often significantly quicker to execute, and are more targeted due to skipping the UI layer, but still require the majority of the system to be stood up.
This is not a general rule though, depending on your system architecture, it’s possible to manipulate your APIs to use mocked database data, and even mocked downstream services, but I refer to these types of tests as unit testing. That’s why I like the distinction I mentioned about the API, versus the service layer. The service layer is just classes and objects, which are perfect for unit testing. When we bring in the API layer we are increasing the scope, and forcing us to have to interact with the API via HTTP.
Therefore it’s crucial to understand the risk you are trying to mitigate and target the most appropriate layer of the application. Your testability will play a crucial part here, while you know it should be a unit test, your context may not allow this, therefore moving to the API layer is justified.
Risks We Can Mitigate:
- Status code mapping. Ensuring that the right status codes are returned based on the behaviour we triggered.
- Response body data. Did the API return the values we were expecting to get?
- Headers. Did the API return all the headers we were expecting?
- Did API trigger the downstream code we were expecting it to? E.g. Did the data we sent get saved in the database?
- Data structure. Was the response from the API structured how we expected it to be, or did it have less or additional fields. More on this in the contract testing section.
- Error/Validation messages. Did the API return a sensible/expected error message?
- Basic security testing. Does the API reject our calls if we don’t provide the correct authentication?
How It's Performed
Functional API testing requires us to use a tool to simulate sending requests to our APIs just like a browser or server would. Typically we have to create the following parts: URL, verb, headers and body. The URL is the distinction or API, the endpoint. The verb is the method we want the server to perform, such as PUT, POST, GET. Then we have the headers which typically set the context of our requests. They are very contextual to the application under test, but typically include things like authentication, response type and tracking information. Finally we have the body, this is the data we want the endpoint to process.
When to Use Functional Testing?
If your team has Web APIs such tests should be created as early as possible. They should be used in conjunction with other types of testing such as UI and Unit Testing. They should specifically be used to target risks that cannot be mitigated lower down the system, or where testability doesn’t allow it. Functional APIs tests provide information that our API is alive, performing the business logic, and that our status codes and errors are mapped correctly.
Contract Testing
Contracting testing is a vastly underutilised approach to testing APIs. Contracting testing focuses on testing the contract between provider and consumer. The provider is typically your system, your API, and the consumer could be another of your APIs, a third party or even your user user interface. When we integrate two systems that send and receive data, we write our code to expect that data to be there, and in specific format, the contract. The contract codifies those expectations, and allows us to check if either party breaks the contract.
Contract testing is especially powerful in microservice architecture where all the APIs are interlinked and dependent on each other.
Risks We Can Mitigate:
- Data changes to the contract, such as properties being removed, added or renamed.
- Breaking changes that could impact the consumers.
- Misunderstanding and ambiguity in how the API behaves.
How It's Performed
You’ll need a tool specifically designed for contract testing such as Pact.
There are generally two approaches:
- Consumer-Driven Contract Testing: This is where the consumer defines the contract, and the provider ensures it’s being met. The contract will provide a description, name, required state (e.g. user exists with name Richard) and then the request it’s going to make. Following that it specifies what the response should look like. In short, the consumer is writing the automated test for you.
- Schema Validation: This requires there to be an API Specification, and tests are created to ensure the API is validating the specification. We are less focused on the specific data here, but more on that the data is the specified types, e.g. ID should be a number, name is a string.
When to Use Contract Testing?
Contract testing will provide a lot of value in several contexts. If different teams and building/maintaining the API to the team that consumes it. If you have third party customers consuming your API. They are typically quicker to create, maintain and execute than functional API testing.
Integration Testing
Integration testing in the context of automation API testing is very similar to functional testing. Ideally with functional testing we try to focus on a single endpoint, and limit its downstream communications, whereas with integration testing this is exactly the risks we are trying to mitigate. Our focus with integration testing is to ensure that our API is playing nicely with its integrations, be that other APIs, databases, messaging queues or other systems.
Deep technical system knowledge is crucial to perform integration testing well, as then you can design tests to better target integration risks.
What it Detects:
- Data flow between services. Is the correct data being passed and parsed from other services?
- Configuration. Are we correctly set up to integrate with the other services, including things like authentication.
- Performance issues. Any issues related to using the real services that cause issues such as timeouts.
How It's Performed
Integration tests typically require a full system deployment using real third parties. The same tools used for functional testing can be utilised here. The complexity comes in configuring those systems and targeting test/sandbox instances of any third parties. Data also becomes a big focus here as all the data needs to marry up and sync, unlike when we test a single API.
When to Use Integration Testing?
API integration tests are vital in the release process. If other API testing types exist you can view integration tests as smoke tests for the APIs, if they don’t, then these tests are your protection. If you rely on third parties APIs and don’t have contract testing in place, these will provide some insight into the health of your APIs.
Mock Testing
While mock testing is not specifically a way to test your APIs, it can prove a worthwhile investment to assist with the testing of systems that rely on your APIs. Or to assist you with testing your integrations with third parties, without actually having to connect to the third party.
A Mock API is a simulated instance of your API where we can simulate any scenario we require. For instance you could configure a mock API to always return a 404 status code, often referred to as a stub. Or you could configure it to return the same data every time. This increases our controllability, and allows us to make more deterministic tests. We can also make more sophisticated mock servers that have the ability to store data and respond as if it were the real API, tailoring their response to the data received instead of sending the same response each time. There’s a final type of mock APIs called spies, this type of mock API keeps track of how many times it was called.
A challenge with any mock server is keeping them in sync with the real API. If you make changes to your real API, you need to mirror that in your mocks. This can be time consuming and error prone.
Risks We Can Mitigate:
- A mock server increases our test controllability and allows for more deterministic tests on the consumer, such as testing error status codes that would be very difficult to simulate without a mock.
- They allow us to target our API code, and ensure its functionality without the impact of downstream APIs reducing the scope of our tests.
- How It's Performed:
The vast majority of API frameworks support mocking in some form, or standalone tools can be used. Typically an engineer will trigger the real API and capture its behaviour, such as the status codes, headers and data returned. We can then use that data to create a mock that will simulate that response each time. We can then save several behaviours and configure the mock as required for the test we are running.
When to Use Mock Testing?
Utilising mock servers can be very fruitful in contexts where you lack control over the APIs you are consuming, as it allows you to control the data to test your system based on the back of those APIs. It can also be very useful when another team is dependent on your API, but the implementation isn’t finished yet, you can provide a mock while the implementation is completed.
Conclusion
There are multiple ways to utilise automated testing in the context of APIs, and it’s important to pick the right technique for the risk you are mitigating. The more techniques you are exposed to, the more options you have to tackle risks. On top of that, deep technical knowledge of how your system is built and architecture will allow you to choose specific techniques and discard others.
As our systems grow in complexity and connectivity, we have to utilise all the approaches available to design and implement small targeted tests. By doing this we can ensure the reliability and functionality of our APIs and have tests that are fast to create, execute, maintain and debug. A layered approach to API testing is the best approach I’ve seen to this, combining functional, contract, unit testing and integration testing. Then providing mocks to allow consumers to test those additional scenarios that are hard to simulate.
About Automated Testing series
You're at the third part of our Automated Testing series - a collaboration with Richard Bradshaw. The series will explore many facets of Testing Automation and its implementations.
- In the first part, we explored the Manual and Automated Testing paradox. Check it out here.
- In the second part, we explored the different types of UI Automated Testing. Check it out here.
