“Test Automation Circles” is a graphical representation of the points we emphasize when implementing E2E(End-To-End) test automation. We believe that by i) working from the center of the concentric circles outward and ii)considering and implementing each item in the concentric circles at the same time, we can achieve "less failure-prone and more continuous" test automation.
Core : Why?
Why do you test?
Items of decision
- Objective
- Expected Results
"Why do this test?" is the most important question when implementing test automation.
There must be a core objective that was firmly defined. If test automation was implemented without a core objective,
to run automated tests itself might become the objective, or it may become a factor that makes project direction undecidable when increasing the tests number.
Make sure to define the core objective that was agreed by stakeholders before beginning the test automation.
There are cases where automation proceeds without a core objective, for example, "Let do automation because we purchased the tools" or "We want to automate it and the ~~tools can do it" - in this case the automation will not continue. In some companies, the tools may be determined first, but even in that case, be sure to clearly define the core objective.
By deciding on the core objective, it will help you to determine how to use the tool : "How to use the tools? "
Concept : How?
How do you test?
Items of decision
- Test strategy
- Test scope
- Test design
- Data design
- Execution plan(Pipeline design)
Next, determine the concept of "what kind of test to be conducted'' according to the objective you have set.
Since "test strategy", "test scope", and "design" influence each other, it is recommended to consider them together.
If the concept is not set properly here, it may lead to a lot of rework. Same as the objectives of testing, the concept also must be agreed by stakeholders.
Also, when designing, it is necessary to not only determine the flow of operations such as functional and scenario testing, but also to design the data. Since automated tests are often executed repeatedly, consider idempotency (the result is the same no matter how many times the same operation is performed).
Architecture : What?
What do you use for automated testing?
Items of decision
- Tools
- Framework
- Environment
- CI/CD
Once the core and concept are determined, let's decide on the architecture and actually build the test automation system. Decide on tools and frameworks (e.g., languages and test frameworks) for creating test scripts.
Next question will be "How will you execute automated tests?" - if there is an existing CI/CD system, you can enable automated test execution at the appropriate time. Consider the environment in which tests will be executed and the environment of the system to be tested, and organize the environments so that tests do not depend on each other.
Monitoring and control : Real.
Realize automated testing
Items of decision
- Test autonation solution
- Execution
- Result analysis
- Reporting
The next step is monitoring and control. Test automation operations involve creating/modifying test scripts,
repeating execution, result analysis, and reporting. Please note that automated testing does not end when it is created.
The items to be reported will vary depending on the expected results and objectives determined at the beginning.
"Is it the daily test results?", "Is it the trend of test success or failure?", "Is it a change in test execution time?",
"Can you quickly retrieve information to analyze the results?", etc.
Dashboards are often prepared for monitoring.
As for test scripts, regular refactoring is required. If the test objects are updated, the test scripts must also be updated. In addition to changing the basic elements, we also review the test cases itselves.
In doing so, we need to ask ourselves: "Is it necessary as a test?" or "Can they be merged with other test cases?", etc. Keep the size appropriate so that refactoring does not result in an increase in test cases and lack of maintenance.
Base : Base.
Base for building automated testing
Items of decision
- Resources
- Teams
- Skill Set
- Culture
Finally, let's talk about the base. We have described the processes and priorities required to implement test automation. However, in order to achieve these goals, a base is also necessary.
To begin with, it is difficult to continue without a certain understanding and culture of automated testing.
As a base there must be a clear reason why it is necessary, and the team must work together for the expected results.
Do you invest the necessary resources to create automated tests that fully utilize your existing skill set, or do you bring in new skills as a team to move forward with the automation? - This is often a critical turning point.
If engineers are going to maintain test scripts, it is preferable that the scripting language be the same as the development language.
Also, if there is no culture or if the team is negative, the automated tests that have been created will not be well utilized.
In addition, if there are no resources or skill sets, the test automation system created will not be maintained and cannot be continued, or it will become dependent on the individual.
Therefore, these base elements are very important in implementing test automation.