ISTQB® Certified Tester AI Testing (CT-AI)

Artificial Intelligence (AI) and Machine Learning (ML) are increasingly being applied in business-critical systems across many industries. While these technologies create significant opportunities, they also introduce new quality risks and testing challenges that cannot be addressed using traditional testing approaches alone. Data quality, bias, explainability, robustness, model drift, non-deterministic behaviour and Generative AI all require specialised testing knowledge and techniques.

The accredited ISTQB® Certified Tester AI Testing (CT-AI) training provides testing professionals with a solid foundation in testing AI-based systems. You will learn the fundamental concepts of Artificial Intelligence and Machine Learning, understand the quality characteristics specific to AI-based systems and apply testing techniques that address the unique challenges of machine learning models and Generative AI solutions.

Throughout the course you will gain insight into the complete AI system lifecycle. You will learn how training data is prepared, how machine learning models are developed and evaluated, which quality risks arise during development and deployment, and how these risks can be mitigated through effective testing. The course also covers Large Language Models (LLMs), prompt-based systems, red teaming, bias detection, model drift, adversarial testing and deployment-related testing.

The training follows the official ISTQB® Certified Tester AI Testing (CT-AI) v2.0 syllabus and combines theory with practical exercises. Participants will gain hands-on experience with machine learning workflows, data preparation, performance evaluation and AI-specific testing techniques. As a result, you will be well prepared for the certification exam and able to apply the acquired knowledge directly in real-world AI projects.

Special attention is given to the challenges of testing modern AI systems, including probabilistic behaviour, statistical testing approaches, the absence of traditional test oracles, and the growing importance of regulatory frameworks such as the EU AI Act. The course provides practical guidance on how testers can contribute to the quality, reliability and trustworthiness of AI-based systems throughout their lifecycle.

Target Audience

This course is intended for professionals involved in the development, implementation, validation or testing of AI-based systems.

The training is particularly suitable for:

• Software Testers
• Test Analysts
• Test Engineers
• Test Automation Engineers
• Test Consultants
• Test Managers
• Quality Assurance Professionals
• Software Developers
• Data Analysts
• Data Scientists
• Machine Learning Engineers
• Product Owners
• Business Analysts involved in AI initiatives
• Project Managers responsible for AI-based projects

The course is also valuable for professionals who want to understand the quality risks, testing challenges and validation approaches associated with Artificial Intelligence, Machine Learning and Generative AI systems.

Programme

1. Introduction to Artificial Intelligence

• What is Artificial Intelligence?
• AI-based systems versus conventional systems
• Narrow AI, General AI and Super AI
• Different AI technologies and architectures
• Generative AI fundamentals
• Large Language Models (LLMs)
• Development and hosting of AI models
• Regulations, standards and the EU AI Act

2. Quality Characteristics for AI-Based Systems

• AI-specific quality characteristics
• Functional correctness
• Robustness and reliability
• Transparency and explainability
• Controllability and adaptability
• AI safety considerations
• Acceptance criteria for AI-based systems

3. Machine Learning Fundamentals

• Machine Learning workflows and lifecycle
• Supervised, Unsupervised and Reinforcement Learning
• Data preparation and dataset management
• Training, validation and testing datasets
• Pretrained models and fine-tuning
• Retrieval Augmented Generation (RAG)
• Neural networks and model architectures
• ML functional performance metrics
• Accuracy, Precision, Recall and F1-score
• Confusion matrices and model evaluation

4. Testing AI-Based Systems

• Challenges of testing AI systems
• Locked versus adaptive AI systems
• Risk-based testing approaches
• Statistical testing techniques
• Test oracles for AI systems
• Test design techniques for AI-based systems
• Testing Generative AI systems
• Testing Large Language Models
• Exploratory testing and red teaming

5. Input Data Testing

• Data quality risks
• Bias detection and mitigation
• Representativeness of datasets
• Testing data pipelines
• Label correctness and validation
• Input constraints and data integrity
• Data preparation verification

6. Model Testing

• Machine learning model testing strategies
• Overfitting and underfitting
• Adversarial testing
• Metamorphic testing
• Back-to-back testing
• A/B testing
• Model drift detection
• Performance validation and monitoring

7. Machine Learning Development and Deployment Testing

• Testing throughout the ML development lifecycle
• Verification and validation activities
• Deployment testing
• Production monitoring
• Maintaining AI model performance
• Managing AI-related risks in production
• Continuous quality assurance for AI systems

• Duration: 3 days
• Training hours: 09:15 – 17:00
• Includes digital course material, lunch, coffee and tea
• Three months of free e-coaching after the course
• Examination fee not included
• The ISTQB® Certified Tester Foundation Level (CTFL) certificate is a prerequisite for the CT-AI examination
• Delivered by an experienced AI Testing trainer
• Also available as an in-company course
• Accredited training based on the official ISTQB® CT-AI v2.0 syllabus

Trainer: Johan van Berkel