Common errors when doing continuous integration
Some examples of things we have to avoid if we are looking for efficient continuous integration
Have you already asked yourself why continuous integration is not providing value to your project or organization, even if you have all the tools and infrastructure needed for it?
Maybe the problem is in other stages of the software development life cycle, or in the process and the way the team works.
Because of this, we need to understand what’s the primary goal of this software engineering practice so that we can comprehend what to expect from it.
The purpose of continuous integration is to reduce the time between the development and the release of a working software update in production, providing quick feedback in case the new package may cause any issue to the application.
In this post, some techniques, processes and common errors that need to be avoided will be explained to help in achieving better results when doing continuous integration.
Unreliable feedback
One of the advantages of continuous integration is to have different pipelines that give us quick feedback, and that allows us to save time, besides taking the appropriate actions as soon as possible when things go wrong.
There is no benefit in having a complete continuous integration infrastructure if we can’t trust in its feedback.
For instance, let’s say a pipeline fails in the automated tests stage, but the team ignores it because it is not enough reliable to identify the reason for the failure by looking to its result. The cause may be some instability in the test environment, the data being used, or even an error in the application code.
Another example could be that there is an error in the build stage, but the team responsible for it doesn’t have the culture of fixing the errors immediately, leaving the fix for later together with lots of other issues.
The point here is that it doesn’t make any sense having quick feedback if you can’t trust in it, or if no action is immediately taken.
Slow feedback
By just using a good continuous integration tool you can not ensure that you will have quick and reliable feedback. It is also necessary to apply good practices.
Imagine working without an organized structure, without pipelines to run automated tests, without parallelism, and receiving feedback of an error only by the end of a slow regression tests suite. Is it vital to wait until the end of this process to have some feedback? The answer is no.
To make it even worse, what about if the test suite had more graphical user interface (GUI) tests than unit tests? Besides this not being recommended, it could drastically increase the test execution time and the feedback loop.
Techniques such as the test pyramid (having more unit tests than integration tests, and more integration tests than GUI tests), having a smoke test suite (to test the main functionalities first), and the use of parallelism are good examples of things that can provide us quick feedback. The point is that as soon as we find issues, as quick we can fix them, saving valuable time to any project.
Accumulation of local code pending integration
A culture that doesn’t fit with continuous integration is of a team that doesn’t integrate its code with the remote repository frequently (at least once a day). Because by integrating big chunks of code or old branches we increase the chances of having conflicts with the code in the main branch, which increases the time, complexity and cost to fix such conflicts.
One of the advantages of using continuous integration is to allow developers to refactor and integrate their code fast since every integration is analyzed by an automated build that checks if the new package won’t generate any conflict or break the code. Such automated validations can be static code analysis, unit tests, integration tests, end-to-end tests and even visual regression tests. And if the updated software is small and recent, better chances are that fixing an issue will happen quickly, in case it occurs.
With that it is encouraged that all developers in a team integrate their code at least once a day, avoiding working on long life branches.
Specific Environments
Running automated tests in shared environments where manual testing is also executed is very risky, because changes in one environment for other purposes may cause false negative results in the automated tests’ execution (e.g., a test fails because a test script was expecting some data to be available in the system, but someone deleted such data when running manual tests). This leads to the team not trusting in the feedback provided by automated tests. Another example is when tests pass with false positive results, which mean that they are passing when they should be failing. In this case, the tests are not being efficient in finding real bugs.
Is continuous integration always ideal?
Despite the many benefits of such software engineering practice, and based on what was written here so far, some minimum requirements are needed to implement continuous integration:
- Automated tests in all layers of the application
A continuous integration tool can’t find issues on its own without automated tests that will verify if the new code has broken something. - Usage of a version control system
To be able to frequently integrate code a version control system is needed. This may also be the trigger for a continuous integration pipeline to start. - The need to find and fix issues, bugs, failures as soon as possible
There is no point in implementing continuous integration if the team doesn’t have an agile mindset to analyze and fix the problems as soon as they happen. - Running continuous delivery and/or continuous deployment
Continuous delivery or deployment are a consequence of efficient continuous integration, making such process automatic. - Automatic creation of environments
The creation of environments can also be done automatically, and in this way, it is possible to ensure that the automated tests will be executed in an exclusive environment, and no unknown factor will influence its result, making it more reliable.
Going beyond continuous integration
After correctly applying continuous integration, the next steps are to implement continuous delivery (an automated procedure manually triggered to implement a new software version in a specific environment), or even better, continuous deployment (similar to continuous delivery, but where the process is completely automated and triggered after continuous integration pipelines succeed).
These are some tips and common errors that need to be avoided to improve the efficiency of continuous integration, because besides the tools, the team needs to know and work in its concepts and practices.
This is a translation of a post created by one of my pupils (Ricardo Kohatsu). He created this post (in Portuguese) as part of a mentorship program in continuous integration between him and myself. The original post can be found through the following URL:
https://medium.com/inmetrics/erros-comuns-na-integração-cont%C3%ADnua-769556bd78d
Happy continuous integration!
