I have been learning AI in Testing through the 30 Days of AI in Testing Challenge by the Ministry of Testing. As part of this challenge, I get to engage with fellow participants. This checklist is the result of one such collaborative effort of me and one of my fellow participants and awesome tester, Joyz Ng. We collaborated on our learnings from the study on prompt engineering, prompting techniques as well as practical experience with AI LLM bots and came up with this checklist to help our fellow testers understand the art of crafting better prompts to help them with testing.
Check this and do share your feedback to us in the comments section or via LinkedIn messages. Happy Prompting!
For good examples of testing prompts that uses this checklist, you can refer to the AI Prompt Repository for Testers – Rahul’s Testing Titbits
General
- Use simple and complete sentences
- Define the task well
- Be specific (precision is important) on what you want to recieve
- Minimize ambiguity (avoid vague terms)
- Avoid grammar mistakes
-
Mention tone / target audience
- Examples:
- Professional
- Personal
- Funny
- Creative
-
Mention formatting guidelines
- Examples:
- Bullet Pointers
- “Double Quotes”
- CSV
- Tabular
- Avoid sharing sensitive / intellectual information
Context
-
Set a persona role
- Examples:
- Tester
- Test Lead
- Regular Customer
- Developer
- Automation Engineer, etc.
- Specify background information. Ex: Sites or people to refer to (e.g., works of James Bach)
System Under Test (SUT)
- Explain Via Images
- Explain Via System Description
- Explain Via Product Model
- Explain Via Technical Description
Expectations from the Language Model (LLM)
-
Specify What’s Expected?
- Examples:
- Positive Cases Only or Diverse Cases?
- Functional Coverage or Data Coverage?
- Test Data or Program to Generate Test Data?
Input & Output
-
Give Examples of Desired Format and Style Expected
- Examples:
- Code Snippet
- Email Template
- Report Structure
-
Define Data Requirements
- Examples:
- Username – String (20 characters)
- Age – Integer (>18 characters)
- Reference Existing Content or Link for Inspiration
- Examples:
- Coding Guideline
- Writing Sample
- Ask for “Step by Step” Explanation for Reasoning Tasks
- Ask for “Examples” or Real world usecases if you want to use AI for Research / Learning.
Provide Feedback to LLM and Refine Prompts
- Provide feedback to LLM & Refine prompts
- Don’t stop at first response. Ask follow up questions.
- Compare Results with Other AI Models’ Responses
-
- Examples:
- Gemini
- Copilot
- ChatGPT 3.5
- Claude
Quality Measurement
-
Define How You’ll Measure Response Quality
- Examples:
- Creativity
- Accuracy
- Fluency
Update Log:
- First Draft Created by Rahul Parwal & Joyz Ng on 13th March 2024
- Published on 14th March 2024
- v1.1 Updated on 19th March 2024. Updated the checklist based on first level feedback from Maneesh.
Rahul Parwal
Joyz Ng