If you are interested in solving “real” business problems, questioning is one skill that you would want to master as a tech professional. Until and unless you refine the actual problem to what the business actually wants, you might always risk solving the wrong problem.
Real Problem solving involves asking questions at scale and the same is applicable to the field of automation. I had read an article on Context-Free Questions for Testing by Michael Bolton some months back, and I became a great fan of his question bank. His question bank had all possible questions to basically dissect and solve a large unknown unclear testing problem.
In this article, I will try to list such context-free questions that can be used to reveal the actual context of the automation problem that you might be trying to solve. Of course, you might already know the answers to some questions, whenever that is the case, just skip that question. So now, let’s start the list:
- Is it okay to ask some questions?
- What is the development methodology?
- What is the sprint duration?
- By when would we ideally get the test build?
- What is the dev technology of the app to be tested?
- Is there any existing automation solution?
- Do we have any budget for procuring tools? (some automation tools might not be free).
- What is the current team’s tech expertise?
- What is the current team’s available bandwidth?
- What is the team composition?
- Do we have a separate automation team?
- How testable is the app?
- Can the developer help us in making the app more automation-friendly?
- Can we discuss testability if we have not already discussed them yet?
- How frequent are the releases?
- Is the app stable right now?
- Are we expecting major refactoring in the near future?
- Which layer of the app is relatively stable?
- Do we even need automation right now?
- What is the current stage of our product?
- What are the important quality criteria?
- Are we looking for a long-term solution or a short-term one?
- Do we have any team expansion plans? Automation might need maintainers in the future.
- What are our customer pain points?
- What are the known risks?
- What is the release plan(s)?
- Do we have any existing procured tool licenses?
- Have we considered the cost of migration? Is it even worth migrating to a new tech stack?
- Do you even have enough resources? Cloud infra? Machines? VMs? Hardware devices?
- Can this project be abandoned in the middle?
- How would we execute the automation suite? CI? Manually?
- What would be the frequency of automation execution?
- Will the developers also contribute to the automation suite?
- Do we have any coding guidelines in place?
- Will we have a separate repository for the automation suite or will we have it in the product repository only?
- How many test suites do we need?
- Where will we use which test suite?
- Do we deprecate some automation scripts too? If yes, When & Why?
- Do we link the automation scripts with the test case ids? Do we need to trace them to some requirements?
- What kind of scripts should we cover as a part of automation? UI? Functional? Accessibility? Other?
- Do we define the Definition of Done for Automation too?
- Do we need to create a testing framework of our own? or we will use one created by someone else? In the case of someone else, do they help us with support queries or framework updates?
- What format of report do you expect?
- Where should we store the reports? Should we auto-archive test reports after a time period?
- What platforms will this automation suite also run on?
- Should we use cross-platform libraries/tools?
- Is there something else that I should be asking but I have not asked?
I know this list is not complete but this is good to get anyone started in the process of problem-solving. The information you uncover from such questions will help you plan for your automation in a better way.
If you feel something is missing, I request you to help me extend this list. Please share your thoughts/additions in the comments.
UPDATE: If you are tired of the above questions, there is another interesting perspective shared by Brijesh Deb that uses three questions to understand the actual problem statement. Here it goes:
1. What problems are you trying to solve for your customer?
2. What problems will automation solution for you?
3. What challenges keep you awake at night?