AI in Testing
A hand-curated field guide to the resources that actually matter u2014 the foundations worth understanding, the prompts and tools worth using, and the critical voices worth trusting. Built for testers who want signal over slop. This is AI in Testing & not Testing of AI.
Start Here
7Gen AI for Testers — Free Certification Course
A free, structured on-ramp — the place to begin if GenAI for testing is new to you.
shiftsync.tricentis.com ↗AI Prompt Repository for Testers
A living library of battle-tested prompts built for real testing tasks.
testingtitbits.com ↗Prompting Checklist for Testers
The pre-flight checklist that turns a vague prompt into a reliable one.
testingtitbits.com ↗AI Usage Quadrants for Testers
A decision model for when AI genuinely helps — and when it quietly hurts.
testingtitbits.com ↗AI Model Selection for Testers
Choose the right model by weighing task complexity against knowledge currency.
testingtitbits.com ↗The Fine Print of Using AI Tools in Testing
The risks, limits and disclaimers every tester should read before trusting output.
testingtitbits.com ↗AI in Testing — Video Series
An ongoing video series on applying AI to everyday testing work.
youtube.com ↗Foundations
10What Is ChatGPT Doing… and Why Does It Work?
Stephen Wolfram’s definitive deep-dive on how LLMs actually generate text.
writings.stephenwolfram.com ↗Transformer Explainer
An interactive, visual walkthrough of a live GPT-2 — watch attention in motion.
poloclub.github.io ↗A Conceptual Guide to Transformers
A philosopher’s clear, math-light explanation of the transformer.
benlevinstein.substack.com ↗LLM Visualization
Walk through a working LLM in 3D, layer by layer, weight by weight.
bbycroft.net ↗Tiktokenizer
See exactly how a prompt is tokenized — and why token count matters.
tiktokenizer.vercel.app ↗FineWeb: Decanting the Web for the Finest Text Data
How a state-of-the-art training dataset is filtered out of the raw web.
huggingface.co ↗FineWeb-Edu Dataset
The educational-quality dataset behind many strong open models.
huggingface.co ↗Chatbot Arena (LMArena)
Crowd-sourced blind rankings of frontier models, head to head.
lmarena.ai ↗All Large Language Models Directory
A searchable directory of nearly every model on the market.
llmmodels.org ↗AI News by Buttondown
A dense daily digest of what actually shipped in AI.
buttondown.com ↗Prompting
14Prompt Engineering Guide
The most comprehensive open reference on prompting techniques.
promptingguide.ai ↗2,500+ ChatGPT Prompt Templates
A vast starter pack to adapt and remix for testing workflows.
ignacio-velasquez.notion.site ↗Awesome ChatGPT Prompts
The classic community-curated prompt collection on GitHub.
github.com ↗Google Cloud Prompt Gallery
Production-grade prompt samples straight from Google’s playbook.
cloud.google.com ↗Snack Prompt
Browse and remix trending prompts by category.
snackprompt.com ↗Google AI Studio
A free playground to prototype prompts against Gemini.
aistudio.google.com ↗Vertex AI Studio
Enterprise-grade prompt optimization and refinement.
console.cloud.google.com ↗AICamp
Compare multiple LLMs side by side in one workspace.
app.aicamp.so ↗LM Studio
Discover, download and run open LLMs entirely on your own machine.
lmstudio.ai ↗System Prompts of AI Tools
A collected archive of the system prompts behind popular AI tools — a masterclass in prompt design.
github.com ↗PromptGPT
Turns a rough task into a clean, structured Markdown prompt.
chatgpt.com ↗PromptPro
An open-source app for refining and enhancing prompt outputs.
github.com ↗AI-Powered Prompt Writing (Google Cloud)
Google’s guide to using AI to write better prompts.
cloud.google.com ↗ChatHub
Query several chatbots at once and compare their answers.
chathub.gg ↗In Practice
830 Days of AI in Testing
Ministry of Testing’s structured, month-long hands-on challenge.
ministryoftesting.com ↗AI in Testing Roadmap
A staged learning pathway from the basics to applied practice.
muuktest.com ↗Incorporating AI in Testing at Work
Practical patterns for bringing AI into a real team.
shiftsync.tricentis.com ↗Prompt Engineering for Testers
Tester-specific prompting techniques, with worked examples.
testsigma.com ↗Software Testing with Generative AI
Mark Winteringham’s hands-on book on applying GenAI to testing.
manning.com ↗What Skills an AI-Assisted Testing Team Needs
The capability mix an AI-era testing team actually requires.
muuktest.com ↗SLOP to KINO — ShiftSync Webinar
A live session on telling AI slop apart from genuine quality signal.
shiftsync.tricentis.com ↗MCP Servers & Automation for Testers
A course on wiring up MCP servers to automate testing tasks.
shiftsync.tricentis.com ↗Critical Thinking
10Is There Something in Your u201cIu201d?
James Bach on what intelligence really means for testers.
satisfice.com ↗ChatGPT Sucks at Being a Testing Expert
A pointed appendix documenting where the model falls down.
satisfice.com ↗A Super-Quick Guide to Evaluating AI Claims
Michael Bolton’s sharp filter for cutting through AI hype.
developsense.com ↗A Reply to a Crowd-Sourced LLM Testing Experiment
A careful critique of a widely shared LLM-for-testing study.
developsense.com ↗The LLMentalist Effect
Why LLMs feel intelligent — the cold-reading illusion, explained.
softwarecrisis.dev ↗The 70% Problem
Addy Osmani on why AI gets you 70% of the way and then stalls.
addyo.substack.com ↗Reversal Curse
Research showing models trained on u2018A is B’ often fail to learn u2018B is A’.
arxiv.org ↗ChatGPT vs Stack Overflow — Who Answers Better?
An empirical comparison of ChatGPT and Stack Overflow answers.
arxiv.org ↗Shining a Light on Shadow Prompting
On the hidden prompts quietly shaping AI behaviour.
techpolicy.press ↗Pivot to AI
Running commentary on AI overreach and broken promises.
pivot-to-ai.com ↗Agentic AI
4AI Agents in Testing: The 5W1H Framework — Part 1
A clear framework for reasoning about AI agents in testing.
shiftsync.tricentis.com ↗AI Agents in Testing: The 5W1H Framework — Part 2
Part two — applying the 5W1H framework in practice.
shiftsync.tricentis.com ↗AI Agents for Testers — Masterclass
A practical masterclass recording built on the 5W1H framework.
shiftsync.tricentis.com ↗AI Engineering Resources (aie-book)
Chip Huyen’s open resource set for serious AI engineers.
github.com ↗People
10Andrej Karpathy
Foundational, accessible teaching from a leading AI researcher.
karpathy.ai ↗Andrej Karpathy — YouTube
His legendary build-an-LLM-from-scratch lectures.
youtube.com ↗Michael Bolton
Rapid Software Testing co-author and a leading critical voice.
linkedin.com ↗Rahul Verma
A deep thinker on testing craft and automation design.
linkedin.com ↗Dragan Spiridonov
An active voice at the AI-and-testing intersection.
linkedin.com ↗Rahul Parwal
Follow along for ongoing AI-in-testing work and experiments.
linkedin.com ↗AI Chapter — Ministry of Testing
Ministry of Testing’s community hub for AI in testing.
ministryoftesting.com ↗ShiftSync Community
Tricentis’ community for modern testing practice.
shiftsync.tricentis.com ↗AI Engineer — YouTube
Talks from the leading AI engineering conference.
youtube.com ↗AI & Beyond AI — YouTube
AI literacy explained for a general audience.
youtube.com ↗Tools
10AI Testing Tools (Compilation)
A broad, regularly-updated compilation of AI-powered testing tools.
aitesting.tools ↗TestingTools.ai
Another curated directory of AI-powered testing tools.
testingtools.ai ↗TestCraft
AI-assisted, codeless test automation.
home.testcraft.app ↗TestHatch
Generates test cases from your requirements.
testhatch.com ↗CodiumAI
Generates meaningful tests for busy developers.
codium.ai ↗Joe Bot AI (TestGuild)
TestGuild’s AI assistant for automation questions.
testguild.com ↗Amikoo by MuukTest
AI agents paired with human experts for test automation.
muuktest.com ↗Gretel.ai
Synthetic test-data generation at scale.
gretel.ai ↗Future Tools
A constantly-updated finder for the exact AI tool you need.
futuretools.io ↗There’s An AI For That
Search for an AI tool for almost any task.
theresanaiforthat.com ↗No resources match that search. Try a broader term.