Some people smile skeptically when they hear the “testing in production” expression. Indeed, there is a persistent myth that this is an extremely unprofessional and risky approach. However, if you dig deeper into the topic, everything will turn out to be the opposite. In this article, we figure out what testing in production is and why it has long been a mandatory part of any QA strategy aimed at releasing a quality product.
A step of the continuous testing process
People doubt the efficiency of testing in production because of the misunderstanding of the term. They think that this is testing after deploying untested code. Of course, in this case, it is very unlikely that we will get a good result.
Testing in production means a stage in the process of continuous testing, when the application, after being deployed, is already in the production environment. Thus, testing in production does not mean unsystematic testing. This approach requires a carefully designed strategy, otherwise, it will not be useful. It is only important for you to understand what gaps in traditional testing this method will solve. In addition, a high-quality automation testing platform makes it possible to use all the advantages of testing in production.
The difference between traditional testing and testing in production
Traditionally, a QA team focuses on software testing before an application is released. It is assumed that testing should be carried out in a separate environment, usually a QA environment or Staging. The goal is to check if the application is ready for release and to prevent a buggy application from entering the market. Nonetheless, some problems cannot be identified and solved this way.
The testing in production technique involves verification of an application in a Production environment. The goal is to find problems in real use.
It should be noted that the Staging environment is very similar to the Production environment. Some even consider them the same, with the difference that the Staging environment is not available to end-users. Nevertheless, when you are working on a complex application, synchronizing and maintaining the Staging environment becomes cumbersome.
In addition, when integrating with third-party services, the settings for the test and real environments are different. For this reason, it cannot be guaranteed that if certain tests pass in one environment, all services and components will work correctly in another. Testing in production solves this issue.
As we see, testing in production is not a replacement for pre-production testing, both of them are components of one workflow.
Benefits
Testing in production makes it possible to see and fix those bugs that cannot be seen before the release. However, the list of advantages of testing in production is not limited to this.
Test the app under the real load
Some software errors can only be detected under continuous load in real conditions. Such failures include memory leaks, issues with the stability of the application, and speed. In practice, it may turn out that two resource-intensive tasks are executed in parallel, which negatively affects the performance of the application. For this reason, full load testing is recommended to be done in a production environment.
Get real-world scenarios
It is often difficult to replicate all real-world scenarios in a test environment. The production makes it possible to see exactly what users do in the application and verify these scenarios. Everyone knows that it is better when a tester notices a critical error than a user.
Fix all the remaining bugs
A well-thought-out QA strategy, a team of experienced test engineers, and the use of the best tools cannot 100% guarantee the detection of all bugs. In this case, testing in production is a working way to find all the missed bugs.
Top-5 testing in production best practices
Check delivered tasks. It allows you to see how they perform in the new environment. Particular attention should be paid to integration since many settings depend on the environment (API, URL, various components). In addition, bugs may appear on the side of integrated services.
Monitor the app according to certain metrics. When problems arise, report this to the responsible person for this area.
Use the practice of сanary deployment. The miners took canaries with them to understand in time that there was gas in the mine. As soon as the bird fell silent, people had to urgently go upstairs to stay alive. In testing, using the canary deployment approach, you are sending some real production traffic to the new version. This is how you sacrifice small things for a big goal. Canary deployment allows you to roll back a version if something goes wrong.
Apply A/B testing. This practice makes it easy to understand how well the modified software version is done. Divide the user base into two groups. Group A is using the latest version of your application. Group B uses a modified version. Next, you compare how users in both groups behave.
Adapt user acceptance testing. User acceptance testing (UAT) helps to find out if the application can perform all the necessary functions in real-world scenarios. Many organizations hide the new feature behind a flag and only allow authorized personnel to access it. After QA engineers make sure that everything works as it should, the feature will become available to all users.
You may be interested in: Main Beneficial Aspects of QA Automation for a Business
