Introduction
Testing a car engine is one thing, but testing a car is another. Since a car will typically include a lot more components, testing individual parts doesn’t necessarily guarantee that they can work well together. Even if the individual functionality of each component proved to be fine, we still need to verify that they integrate without any problems.
On the other hand, we can’t just start testing the car as a whole from the beginning. It will be quite difficult to isolate and track down the issues that appear, and more importantly, it will be costly to take a step back to fix an individual component after we moved to test the whole car.
So we need to have the best of both worlds, testing our individual components separately to gain more confidence in their behavior and find any issues early in the process, and also verifying that these different components can actually work together.
The same concept also holds true for software, with the difference that —obviously— our final product is an application and not a car. Software applications are usually built from multiple components, each playing a specific role or function in the overall application behavior. For example, an authentication service to verify the user identity, a shopping cart service to store items bought by a user, and a payment service to process a payment transaction.
To ensure that our application are working as expected, we need to test each individual service, as well as testing the flow when multiple services work together. In software terms, the individual component testing is called Unit Test, while the aggregated components testing is called Integration Test.
In this article we are going to compare these two testing techniques and explain how each works. Before we dive into the comparison, let’s start with an overview about software testing and why we need it.
What is Software Testing?
Software testing is a key phase in the software development lifecycle. It ensures that the software meets a set of business requirements and works as expected. These requirements and expectations are usually broad terms that differ from one software to another, and they should be validated according to a comprehensive testing strategy.
There are many aspects to validate in software. For example, we need to check that it doesn’t contain bugs or errors, delivers a certain level of performance under high load, and satisfies the overall required user functionality.
These diverse validations are addressed by using different testing approaches, which can be roughly classified as follows:
- Unit Testing: This is the most fundamental type of testing. It focuses on validating the behavior of individual components in isolation.
- Integration Testing: This type of testing combines multiple components and verifies they work well together. It tests the interfaces between these components and the flow of interactions between them.
- System Testing: System testing validates the overall system behavior by using the entire software with its fully integrated components. Unlike integration testing which focuses on combining individual components to work together, system testing focuses on the whole software functionality.
- Acceptance Testing: This type of testing is usually carried out after system testing. It validates that the software satisfies customer requirements and meets business expectations. Different stakeholders might be involved in this testing including the customer, product owner, developers, and testers.
- Performance Testing: As the name implies, this type of testing validates the performance of the software under certain conditions. This performance includes the responsiveness, stability, and scalability of the system.
- Security Testing: This type of testing aims at detecting security vulnerabilities in software. It reduces the risk of malicious attacks, unauthorized access, and data breaches.
There are other classifications of testing approaches that could be carried out for specific validations, but these are the most common and broad types.
What is Unit Testing?
Unit Testing is a type of software testing that validates individual parts of the code —called units— in software. A unit is the smallest piece of code that can be logically tested in isolation. What we mean by logically is that it provides a useful value for the developer when being tested. There’s no rigid definition for the term unit. It can vary from an individual function or class to a group of related functions, the idea is that it performs a specific task on its own that is considered a small building block in the whole software functionality.
Unit testing also is considered a type of automated test. The unit test itself is actually implemented as a piece of code that exercises the real software code. It provides a test input and validates that the output matches the expected result.
Unlike other types of tests, unit tests are not designed to validate business requirements or user expectations, but rather are targeted towards the developer’s work. In other words, unit tests aim to validate that a piece of code implemented by a developer is correctly doing what it is supposed to do.
This specific characteristic of unit tests requires that the person who writes the tests understands quite well the code that is being tested. This is why unit tests are typically written by the developers themselves.
Let us demonstrate this with a simple example, assume we have a basic function that compares two numbers:
public class numOperations {
public int compare(int num1, int num2) {
if (num1 > num2) return 1;
else if (num1 < num2) return -1;
return 0;
}
}To verify that it’s working fine, we create a separate code that tests the different scenarios for this function:
public class numOperationsTest {
@Test
public void scenario1() {
numOperations mytest = new numOperations();
int value = mytest.compare(3, 2);
Assertions.assertEquals(1, value);
}
@Test
public void scenario2() {
numOperations mytest = new numOperations();
int value = mytest.compare(2, 3);
Assertions.assertEquals(-1, value);
}
@Test
public void scenario3() {
numOperations mytest = new numOperations();
int value = mytest.compare(2, 2);
Assertions.assertEquals(0, value);
}
}
The test code here uses the real function code and provides different inputs for different scenarios. For example, the scenario1 function uses the compare function with the first input greater than the second, and it evaluates that the result will be equal to 1 as expected by the compare function. If the result didn’t match the expected output, the test will fail.
The above example can be considered a unit test, we can see that it focuses on the code behavior and validates a specific part of the software which is the compare function. It is written in Java but of course the same idea applies for any other language.
Isolating Unit Tests
The most differentiating characteristic of a unit test is that it tests only an individual part of the software. The code being tested (usually called system under test or SUT) should be validated in isolation from other components or dependencies. For example, a unit test shouldn’t talk to a database or a network service.
This isolation provides the following benefits:
- Speed: Communicating with external services like databases, filesystems, or the network, causes additional latency and slows down the tests. When unit tests are isolated correctly, the feedback and results of the tests are usually quick.
- Determinism: When a unit test doesn’t rely on any external component, we can be sure that we will get the same result each time we run the same test. So a failing/passing test should continue to fail/pass unless a change is applied to the code. However, if we rely on an external dependency, the result of the test might change due to a change in the dependency and not necessarily the code under test.
- Feedback Scope: When the test runs against a specific part of the code, any errors that happen should be only scoped to this part and no other dependency. This makes isolating and fixing the issues easier and provides a more precise feedback.
Mocking and Stubbing
The idea of isolation isn’t always trivial in unit testing. Some components might have strong dependency on other components and there must be an interaction between them in order for the part under test to work as expected. For example, to test the behavior of a module that writes to a database we’ll need an existing database to be running so that the module executes successfully.
So how can we apply the isolation on our module and also allow it to execute without its dependency ? The answer is using test doubles.
Test doubles are objects that mimic the real dependencies without actually providing their full functionality. They enable the component under test to run normally as if it were interacting with its dependency. This way, they help keep unit tests isolated as required (no real dependencies used) while also providing a smooth execution.
Test double is a generic term which has different types and ways of implementation. The two most common types of test doubles are mocks and stubs.
Both mocks and stubs simulate a component of the software to provide isolation for the system under test, but they differ in the way each of them works.
Mocks verify that the system under test sends the correct calls to the dependency it uses. When mocks are used, we validate the result of the test on our component and also the interactions it performed against the mocked object. As part of our tests, we set the expectations for the mock object on what calls it should receive, and verify that the system under test correctly issued these calls. This type of verification is called behavior verification.
Stubs, on the other hand, are pre-programmed with specific responses to calls. They focus on the state of the system under test regardless of its interactions with other components.The responses provided by stubs are usually hardcoded and predefined, which makes stubs predictable. For example, we can configure a stub to respond to a request with success or failure, and then verify that the system under test handles the response correctly. This type of verification is called state verification.
The example in the image above shows a simplified payment transaction scenario. In a normal flow, whenever a transaction takes place, the corresponding account object is invoked to deduct the amount of the transaction from the total amount of money in that account. This creates a dependency between the transaction class and the account class.
To verify the behavior of the transaction class with a unit test, we need to isolate it from its dependency. So we need to use a test double to replace the account class and simulate its behavior.
In the first case we use a mock object for the account. As part of our test code, we set the expectation for our mock object to wait for a deduct amount call from the transaction object. Our test will then verify that this call was sent to the mock, and that our transaction was successful. This way, we’ve validated our component under test, and also its interaction with its dependencies.
A simple implementation for this part could go like this:
//expectations
accountMock.expects(once()).method("deductAmount")
.with(eq(amount_of_money))
.will(returnValue(true));
//exercise
transaction.execute((Account) accountMock.proxy());
//verify
accountMock.verify();
assertTrue(transaction.isSuccessful());In the second case we use a stub object for the account. We configure the stub with a predefined response of success whenever it receives a deduct amount call. In our validation, we only check the state of the component under test, that is, our transaction object is successful.
A simple implementation of this stub will not contain an expectation part as we don’t need to validate the interaction with the stub. But the stub object will be set to respond with a predefined value:
//stub class
class StubAccount implements Account {
public boolean deductAmount() {
//deductAmount returns true whenever it’s called
return true;
}
}
//exercise
transaction.execute((Account)
//verify
assertTrue(transaction.isSuccessful());Unit Testing in CI/CD
CI/CD (continuous integration/continuous development) is a process that aims to deliver software reliably, securely, and more frequently. It automates the steps required for getting code changes from development to production including the build, test, and deployment. By automating these steps, we get faster feedback loops, improve the software quality, and deliver changes more frequently.
Since unit testing is a crucial step in the software lifecycle, it is a very common practice to include it as part of the CI/CD process. This provides a fast and consistent way for running our tests. Unit tests are also perfect candidates for CI/CD because they’re already automated as code, which makes executing them in the CI/CD process easier.
By including unit tests in the CI/CD pipeline, we can detect bugs and issues in the code early in the delivery process, which saves time and money. Unit tests in CI/CD also help provide faster feedback for developers, a typical CI/CD pipeline can be configured to run automatically with every change introduced to the code, which ensures the tests are executed as frequently and quickly as possible. Integrating unit tests in the CI/CD helps produce a high-quality and more reliable software with minimal risk of failure or unexpected issues.
Automating the software delivery process is typically implemented using a CI/CD tool. These tools provide different components, plugins, runtime environments that help you create and customize the appropriate CI/CD workflow tailored specifically for your needs. These tools usually provide a simple syntax for users to define the different steps of the workflow.
Travis CI is one tool that offers these functionality and more. It provides a cloud-based CI/CD solution that you can use as a service without the need to setup and configure everything on your own. There’s also an on-premises enterprise version if you want to install and use it on your infrastructure. It supports a wide range of programming languages and allows integrating different tools into your CI/CD pipeline for things like code quality checks, secrets management, and infrastructure deployment. It provides a simple YAML syntax to configure the steps and stages in the workflow.
What is Integration Testing?
Integration testing is a type of software testing that combines multiple components or modules of the software and validates that they work together with no issues. It tests the interactions between the different components and ensures that the expected flow between them is executed.
Testing the interaction between components is usually carried out after verifying that each component works well individually. So integration testing is typically run after unit testing. Issues that can appear between components include filesystem or database access problems, inconsistent data exchange format, or different execution logic.
A simple example for integration testing would be a website that has a login page and a user home page. Each of them has been tested individually, the login page correctly receives the credentials, and the home page displays the user preference. The integration test would validate that the login and home pages work together, that is when a user performs a login, he gets directed to his personal home page.
Integration tests may require setting up a dedicated environment where the tests take place. Because it uses real components, these components need to be deployed together and prepared correctly according to the test scenario.
Integration Testing Strategies
There are different approaches for implementing integration tests. The two most common are Big Bang integration testing and incremental integration testing.
In the Big Bang approach, all modules of the software are combined together and tested at once. It requires that all the components are developed and ready, also successfully unit tested. This approach fits more in small and simple software systems with fewer component dependencies. It is easier to set up and requires little planning as all components are integrated at the same time. The disadvantage of this approach is that it’s harder to debug and isolate the issues, and we have to wait until all modules are developed so we can run the tests. It can also be difficult to use the Big Bang approach in large and complex systems with lots of integrations.
The incremental approach involves grouping only a subset of the modules which are closely related in functionality and testing them, then gradually including more modules and groups until all modules are integrated and tested. The subset can contain two or more modules depending on the test plan. This approach allows for easier fault isolation since it starts with a smaller number of integrations and increases gradually. It also enables earlier detection of errors because it doesn’t require all modules to be ready at once, so a couple of ready modules can be immediately tested as soon as they’re developed. The disadvantage of this approach is that it requires detailed planning and usually includes a large number of tests.
The incremental approach can be further divided into two types, a bottom-up approach and a top-down approach.
The bottom-up approach starts by testing lower-level modules first, then gradually move to the higher-level modules, while the Top-down approach starts by testing the higher-level modules first, then moves to the lower-level ones.
The lower-level modules refer to the more simple and basic components of the software that perform fundamental tasks of the system. They usually reside in the bottom of the system hierarchy. For example, a database connection module or an input verification module. Higher-level modules are the more complex and large modules that perform multiple functions and can rely on one or more of the lower-level modules.
Integration Testing in CI/CD
Including integration tests in the CI/CD pipeline is a common practice that allows identifying integration problems and bugs before releasing to production. This process may require an automated deployment to some type of a testing/staging environment where components under test are deployed.
In some cases we can also automate the provisioning and configuration of the testing environment as part of the CI/CD pipeline. For example, we can create a specific stage in the pipeline to spin up some infrastructure with the appropriate configuration like network connections, environment variables, web servers, or database instances. Then we create another stage that deploys the required components of the software to this environment according to the test plan. Finally, we execute the set of automated integration tests and get the results.
By automating as much as we can of the integration testing process and including it in our CI/CD pipelines, we enable a faster and more frequent delivery of software changes without sacrificing the quality or reliability of the system. Since the integration tests are frequently executed with each pipeline trigger, we allow for early detection of issues and quick feedback loop.
Unit Testing vs Integration Testing: Which One Should You Choose?
Instead of considering unit and integration testing as alternatives, it’s better to perceive them as complementary. We’ve seen throughout this article how they differ in the way they work and the role they play in the software testing. They shouldn’t be used as replacements for each other since they address different validation criteria in the software.
A more comprehensive testing plan should include both types of tests while correctly differentiating the function of each and precisely scoping them to their suitable goal. We should know where to put each type of test in the software release cycle and what outcome we should expect from each.
Unit testing should be used to validate the behavior of individual components according to what the developer intended. They should be designed to execute as fast as possible and provide precise feedback about individual parts of the code. Unit tests are also cheaper to execute in terms of cost and time, as they don’t need any specific environment configuration or setup. We should always prioritize executing unit tests as early as possible in the software.
Integration testing, on the other hand, tends to be slower and more costly. As it involves interacting with different components that could slow down the tests and also require setting up dedicated environments with specific configurations. This doesn’t mean that integration tests are bad, they’re still needed to provide a different type of validation which a unit test doesn’t perform. However, it’s important to separate them from faster unit tests that can immediately provide feedback for each code change. Imagine having to configure a specific environment or wait for deploying a set of components together to test a single line of code that you introduced to the codebase, not the best option right?
To summarize, unit tests are best suited for providing fast feedback about individual parts of the code and should be executed as early as possible in the process. While integration tests are suitable for more complex testing scenarios that involve the interaction between multiple components of the software. They’re usually slower and more costly, so it’s always a good idea to include them at a later stage of the software release after the unit tests.
Conclusion
Software testing is a key step in the software development lifecycle. It helps releasing a high quality software that meets business and customer requirements. Among the different types of testing, unit testing and integration testing are two of the most common. Unit testing focuses on validating individual components of the software and ensuring that a particular piece of code is working as intended by the developer. Integration testing focuses on combining multiple components and validating the interactions between them to ensure they work together as expected. Unit and Integration testing are not alternatives for each other, but rather they complement each other and should be used together for a better outcome.
