AI testing services
12 testing services. Each count is the number of the 88 published entries whose AI testing page names that service.
| Service | Scope and references | Entries |
|---|---|---|
| LLM output evaluation | Hallucination and factuality checks, answer quality rubrics, LLM-as-judge with human review | 36 |
| RAG pipeline validation | Retrieval quality, grounding, citation accuracy, chunking regressions | 14 |
| Prompt and model regression testing | Golden sets, non-deterministic assertions, repeated runs, version diffs | 17 |
| Red teaming and adversarial testing | Jailbreaks, harmful output elicitation, OWASP Top 10 for LLM Applications, MITRE ATLAS | 56 |
| AI security penetration testing | Prompt injection, data exfiltration, agent and tool abuse, supply chain; OWASP LLM Top 10 | 42 |
| Bias and fairness audit | Disparate impact across groups; NYC Local Law 144; EU AI Act Article 10 | 44 |
| Compliance and conformity assessment support | EU AI Act Articles 9, 10, 43; NIST AI RMF; ISO/IEC 42001 | 16 |
| ML model validation | Accuracy, precision, recall, F1, calibration, confusion analysis, metamorphic tests | 31 |
| Training data and labeling QA | Dataset audits, label accuracy sampling, leakage checks | 25 |
| Performance and latency testing | Inference load, token throughput, time to first token, cost per request under load | 23 |
| Conversational flow testing | Intent coverage, multi-turn context, tone, handoff to humans, channel parity | 11 |
| Production monitoring and drift testing | Post-release evaluation, drift detection, incident regression suites | 31 |
Questions
What types of AI testing services do companies offer?
This directory groups AI testing into 12 services, from LLM output evaluation and RAG pipeline validation to red teaming, bias audits and drift monitoring. The service stated by the most published entries is red teaming and adversarial testing, stated by 56 of 88.