AI in Testing – Resource Repository

Curated Field Guide · Updated May 2026

AI in Testing

A hand-curated field guide to the resources that actually matter u2014 the foundations worth understanding, the prompts and tools worth using, and the critical voices worth trusting. Built for testers who want signal over slop. This is AI in Testing & not Testing of AI.

73Hand-picked resources
8Curated tracks
Rahul ParwalCurated & annotated by
The Signature Collection

Start Here

7
Course★ Editor’s pick

Gen AI for Testers — Free Certification Course

A free, structured on-ramp — the place to begin if GenAI for testing is new to you.

shiftsync.tricentis.com ↗
Framework★ Editor’s pick

AI Prompt Repository for Testers

A living library of battle-tested prompts built for real testing tasks.

testingtitbits.com ↗
Guide

Prompting Checklist for Testers

The pre-flight checklist that turns a vague prompt into a reliable one.

testingtitbits.com ↗
Framework★ Editor’s pick

AI Usage Quadrants for Testers

A decision model for when AI genuinely helps — and when it quietly hurts.

testingtitbits.com ↗
Framework

AI Model Selection for Testers

Choose the right model by weighing task complexity against knowledge currency.

testingtitbits.com ↗
Read

The Fine Print of Using AI Tools in Testing

The risks, limits and disclaimers every tester should read before trusting output.

testingtitbits.com ↗
Watch

AI in Testing — Video Series

An ongoing video series on applying AI to everyday testing work.

youtube.com ↗
First Principles

Foundations

10
Read★ Editor’s pick

What Is ChatGPT Doing… and Why Does It Work?

Stephen Wolfram’s definitive deep-dive on how LLMs actually generate text.

writings.stephenwolfram.com ↗
Tool

Transformer Explainer

An interactive, visual walkthrough of a live GPT-2 — watch attention in motion.

poloclub.github.io ↗
Read

A Conceptual Guide to Transformers

A philosopher’s clear, math-light explanation of the transformer.

benlevinstein.substack.com ↗
Tool★ Editor’s pick

LLM Visualization

Walk through a working LLM in 3D, layer by layer, weight by weight.

bbycroft.net ↗
Tool

Tiktokenizer

See exactly how a prompt is tokenized — and why token count matters.

tiktokenizer.vercel.app ↗
Read

FineWeb: Decanting the Web for the Finest Text Data

How a state-of-the-art training dataset is filtered out of the raw web.

huggingface.co ↗
Dataset

FineWeb-Edu Dataset

The educational-quality dataset behind many strong open models.

huggingface.co ↗
Tool★ Editor’s pick

Chatbot Arena (LMArena)

Crowd-sourced blind rankings of frontier models, head to head.

lmarena.ai ↗
Directory

All Large Language Models Directory

A searchable directory of nearly every model on the market.

llmmodels.org ↗
Read

AI News by Buttondown

A dense daily digest of what actually shipped in AI.

buttondown.com ↗
The Craft of Prompting

Prompting

14
Guide★ Editor’s pick

Prompt Engineering Guide

The most comprehensive open reference on prompting techniques.

promptingguide.ai ↗
Framework

2,500+ ChatGPT Prompt Templates

A vast starter pack to adapt and remix for testing workflows.

ignacio-velasquez.notion.site ↗
Framework

Awesome ChatGPT Prompts

The classic community-curated prompt collection on GitHub.

github.com ↗
Guide

Google Cloud Prompt Gallery

Production-grade prompt samples straight from Google’s playbook.

cloud.google.com ↗
Directory

Snack Prompt

Browse and remix trending prompts by category.

snackprompt.com ↗
Playground

Google AI Studio

A free playground to prototype prompts against Gemini.

aistudio.google.com ↗
Playground

Vertex AI Studio

Enterprise-grade prompt optimization and refinement.

console.cloud.google.com ↗
Tool

AICamp

Compare multiple LLMs side by side in one workspace.

app.aicamp.so ↗
Tool★ Editor’s pick

LM Studio

Discover, download and run open LLMs entirely on your own machine.

lmstudio.ai ↗
Framework★ Editor’s pick

System Prompts of AI Tools

A collected archive of the system prompts behind popular AI tools — a masterclass in prompt design.

github.com ↗
Tool

PromptGPT

Turns a rough task into a clean, structured Markdown prompt.

chatgpt.com ↗
Tool

PromptPro

An open-source app for refining and enhancing prompt outputs.

github.com ↗
Guide

AI-Powered Prompt Writing (Google Cloud)

Google’s guide to using AI to write better prompts.

cloud.google.com ↗
Tool

ChatHub

Query several chatbots at once and compare their answers.

chathub.gg ↗
From Theory to Practice

In Practice

8
Course★ Editor’s pick

30 Days of AI in Testing

Ministry of Testing’s structured, month-long hands-on challenge.

ministryoftesting.com ↗
Guide

AI in Testing Roadmap

A staged learning pathway from the basics to applied practice.

muuktest.com ↗
Read

Incorporating AI in Testing at Work

Practical patterns for bringing AI into a real team.

shiftsync.tricentis.com ↗
Read

Prompt Engineering for Testers

Tester-specific prompting techniques, with worked examples.

testsigma.com ↗
Book★ Editor’s pick

Software Testing with Generative AI

Mark Winteringham’s hands-on book on applying GenAI to testing.

manning.com ↗
Read

What Skills an AI-Assisted Testing Team Needs

The capability mix an AI-era testing team actually requires.

muuktest.com ↗
Watch

SLOP to KINO — ShiftSync Webinar

A live session on telling AI slop apart from genuine quality signal.

shiftsync.tricentis.com ↗
Course★ Editor’s pick

MCP Servers & Automation for Testers

A course on wiring up MCP servers to automate testing tasks.

shiftsync.tricentis.com ↗
The Skeptic’s Shelf

Critical Thinking

10
Read★ Editor’s pick

Is There Something in Your u201cIu201d?

James Bach on what intelligence really means for testers.

satisfice.com ↗
Read

ChatGPT Sucks at Being a Testing Expert

A pointed appendix documenting where the model falls down.

satisfice.com ↗
Read★ Editor’s pick

A Super-Quick Guide to Evaluating AI Claims

Michael Bolton’s sharp filter for cutting through AI hype.

developsense.com ↗
Read

A Reply to a Crowd-Sourced LLM Testing Experiment

A careful critique of a widely shared LLM-for-testing study.

developsense.com ↗
Read★ Editor’s pick

The LLMentalist Effect

Why LLMs feel intelligent — the cold-reading illusion, explained.

softwarecrisis.dev ↗
Read

The 70% Problem

Addy Osmani on why AI gets you 70% of the way and then stalls.

addyo.substack.com ↗
Paper

Reversal Curse

Research showing models trained on u2018A is B’ often fail to learn u2018B is A’.

arxiv.org ↗
Paper

ChatGPT vs Stack Overflow — Who Answers Better?

An empirical comparison of ChatGPT and Stack Overflow answers.

arxiv.org ↗
Read

Shining a Light on Shadow Prompting

On the hidden prompts quietly shaping AI behaviour.

techpolicy.press ↗
Read

Pivot to AI

Running commentary on AI overreach and broken promises.

pivot-to-ai.com ↗
The Next Frontier

Agentic AI

4
Read★ Editor’s pick

AI Agents in Testing: The 5W1H Framework — Part 1

A clear framework for reasoning about AI agents in testing.

shiftsync.tricentis.com ↗
Read

AI Agents in Testing: The 5W1H Framework — Part 2

Part two — applying the 5W1H framework in practice.

shiftsync.tricentis.com ↗
Watch

AI Agents for Testers — Masterclass

A practical masterclass recording built on the 5W1H framework.

shiftsync.tricentis.com ↗
Framework

AI Engineering Resources (aie-book)

Chip Huyen’s open resource set for serious AI engineers.

github.com ↗
Voices Worth Following

People

10
Community★ Editor’s pick

Andrej Karpathy

Foundational, accessible teaching from a leading AI researcher.

karpathy.ai ↗
Watch

Andrej Karpathy — YouTube

His legendary build-an-LLM-from-scratch lectures.

youtube.com ↗
Community

Michael Bolton

Rapid Software Testing co-author and a leading critical voice.

linkedin.com ↗
Community

Rahul Verma

A deep thinker on testing craft and automation design.

linkedin.com ↗
Community

Dragan Spiridonov

An active voice at the AI-and-testing intersection.

linkedin.com ↗
Community

Rahul Parwal

Follow along for ongoing AI-in-testing work and experiments.

linkedin.com ↗
Community★ Editor’s pick

AI Chapter — Ministry of Testing

Ministry of Testing’s community hub for AI in testing.

ministryoftesting.com ↗
Community

ShiftSync Community

Tricentis’ community for modern testing practice.

shiftsync.tricentis.com ↗
Watch

AI Engineer — YouTube

Talks from the leading AI engineering conference.

youtube.com ↗
Watch

AI & Beyond AI — YouTube

AI literacy explained for a general audience.

youtube.com ↗
The Testing Stack

Tools

10
Directory

AI Testing Tools (Compilation)

A broad, regularly-updated compilation of AI-powered testing tools.

aitesting.tools ↗
Directory

TestingTools.ai

Another curated directory of AI-powered testing tools.

testingtools.ai ↗
Tool

TestCraft

AI-assisted, codeless test automation.

home.testcraft.app ↗
Tool

TestHatch

Generates test cases from your requirements.

testhatch.com ↗
Tool

CodiumAI

Generates meaningful tests for busy developers.

codium.ai ↗
Tool

Joe Bot AI (TestGuild)

TestGuild’s AI assistant for automation questions.

testguild.com ↗
Tool

Amikoo by MuukTest

AI agents paired with human experts for test automation.

muuktest.com ↗
Tool★ Editor’s pick

Gretel.ai

Synthetic test-data generation at scale.

gretel.ai ↗
Directory★ Editor’s pick

Future Tools

A constantly-updated finder for the exact AI tool you need.

futuretools.io ↗
Directory

There’s An AI For That

Search for an AI tool for almost any task.

theresanaiforthat.com ↗

No resources match that search. Try a broader term.

Changelog

  • May 2026 u2014 Rebuilt as a curated, searchable field guide with editorial annotations on every resource.
  • Jan 2026 u2014 Added the Agentic AI track.
  • Mar 2025 u2014 Refactored into Foundations, Prompting and more.
  • Jul 2024 u2014 First compilation released.

This list is curated, not crowdsourced u2014 but good suggestions are always welcome. Found something that belongs here?

Suggest a resource

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top