“Test Automation Circles” is a graphic representation of the key points we look for when building and continuing E2E test automation. We believe that by moving from the center of the concentric circles outward, and by examining and implementing each item of the concentric circles simultaneously, we can achieve "less failure-prone and more continuous" test automation.
Core : Why?
Why do you test?
Items of decision
- Objective
- Expected Results
When implementing test automation, why test? is the most important question. There must always be a core objective. It is important to decide firmly. If test automation is introduced without a defined purpose, the objective may become to run automated tests in the middle of the project, or it may become a factor that prevents the direction of the project when increasing the number of tests. It is always advisable to make a decision at the start of test automation and have an agreement with stakeholders. We have seen cases where the decision is made at the start of test automation, such as, "We purchased a tool, so let's automate it," or "We want to automate it, so we can do it with the 00Tool. However, automation will not continue if the company moves forward without deciding on the objective. In some companies, the tool may have been decided first, but even in that case, the purpose should be clearly defined. By deciding on the purpose, how will the tool be used? This will help you to determine how to use the tool.
Concept : How?
How do you test?
Items of decision
- Test strategy
- Test scope
- Test design
- Data design
- Execution plan(Pipeline design)
Next, determine the concept of "what kind of testing" is to be done in accordance with the objectives you have set. Since test strategy, test scope, and design affect each other, it is recommended that they be considered together. If the concept here is not set in stone, rework will be significant. As with the objectives of testing, you need to be aware of them with your stakeholders as well. Also, when designing, it is necessary to not only determine the flow of operations such as functional and scenario testing, but also data design. Since automated tests are often executed repeatedly, consider power equality (the result should be the same no matter how many times the same operation is performed).
Architecture : What?
What do you use for automated testing?
Items of decision
- Tools
- Framework
- Environment
- CI/CD
Once the core and concept are determined, the architecture is determined and the actual test automation system is built. Decide on tools and frameworks (e.g., languages and test frameworks) for creating test scripts. How will you execute automated tests? If there is an existing CI/CD system, we will make sure that automated tests can be executed at the appropriate time. Consider the environment in which tests will be executed and the environment of the system to be tested, and organize the environments so that tests do not depend on each other.
Monitoring and control : Real.
Realize automated testing
Items of decision
- Test autonation solution
- Test scripts
- Test Data
- Execution
- Reporting
- Analysis of results
The next step is monitoring and control. In test automation operations, test scripts are created/modified, executed, results analyzed, and reporting is repeated. Note that automated testing does not end when it is created. The items to be reported will depend on the expected results and objectives determined at the beginning. Is it the daily test results? Is it the trend of test success or failure? Trends in test run times? Can you quickly retrieve information to analyze the results? etc. Monitoring often requires a dashboard. As for test scripts, they need to be refactored periodically. As long as the test objects are updated, the test scripts must also continue to be updated. In addition to changing the basic elements, we also review the test cases themselves. In doing so, we need to ask ourselves: Is it necessary as a test? Or, can they be merged with other test cases? etc. Refactoring will keep the test cases at an appropriate size so that they do not grow to the point where they become unmaintainable.
Base : Base.
Base for building automated testing
Items of decision
- Resources
- Teams
- Skill Set
- Culture
Finally, let's talk about the base. We have described the processes and priorities required to implement test automation. However, in order to achieve these goals, a foundation is also necessary. To begin with, it is difficult to continue without a certain understanding and culture of automated testing. There must be a clear reason why it is necessary as a foundation, and the team must work together for the expected results. Do you put in the necessary resources and create automated tests that more than adequately utilize your existing skill set, or do you bring in new skills as a team and move forward with automation? This is often a critical juncture. If engineers are going to maintain test scripts, it is preferable that the scripting language be the same as the development language. Also, if there is no culture or if the team is negative, the automated tests that have been created will not be well utilized. Also, if the resources and skill sets are not available, the test automation system created will not be maintained and will not be sustainable, or it will become a factor that makes the system more personable. Therefore, these foundational elements are very important in implementing test automation.