CT-AI ISTQB Certified Tester AI Testing Exam Questions and Answers

AI test environments have several unique characteristics that differentiate them from traditional test environments. Let’s evaluate each option:

A. Challenges resulting from low accuracy of the models.

Low accuracy is a common challenge in AI systems, especially during initial development and training phases. Ensuring the model performs accurately in varied and unpredictable scenarios is a critical aspect of AI testing.

B. The challenge of mimicking undefined scenarios generated due to self-learning.

AI systems, particularly those that involve machine learning, can generate undefined or unexpected scenarios due to their self-learning capabilities. Mimicking and testing these scenarios is a unique challenge in AI environments.

C. The challenge of providing explainability to the decisions made by the system.

Explainability, or the ability to understand and articulate how an AI system arrives at its decisions, is a significant and unique challenge in AI testing. This is crucial for trust and transparency in AI systems.

D. Challenges in the creation of scenarios of human handover for autonomous systems.

While important, the creation of scenarios for human handover in autonomous systems is not a characteristic unique to AI test environments. It is more related to the operational and deployment challenges of autonomous systems rather than the intrinsic technology-related characteristics of AI.

Given the above points, optionDis the correct answer because it describes a challenge related to operational deployment rather than a technology-related characteristic unique to AI test environments.

The syllabus states:

“A/B testing can be used to test updates to an AI-based system where there are agreed acceptance criteria, such as ML functional performance metrics, as described in Chapter 5. A/B testing is used to compare the updated variant with the previous variant.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.4, page 68 of 99)

The ML workflow typically involves iterative steps, beginning with data preparation once the model and framework are selected. The syllabus explains:

"The steps shown in Figure 1 (the ML workflow) do not include the integration of the ML model with the non-ML parts of the overall system. Typically, ML models cannot be deployed in isolation and need to be integrated with the non-ML parts... The next step would be data preparation as part of the ML workflow to provide input data to support training by an ML algorithm or prediction by an ML model."

(Reference: ISTQB CT-AI Syllabus v1.0, Sections 3.2 & 4.1)

A. 64-bit CPUs.

While 64-bit CPUs are essential for handling large amounts of memory and performing complex computations, they are not specifically optimized for the types of operations commonly used in machine learning.

B. Hardware supporting fast matrix multiplication.

Matrix multiplication is a fundamental operation in many machine learning algorithms, especially in neural networks and deep learning. Hardware optimized for fast matrix multiplication, such as GPUs (Graphics Processing Units), is most suitable for implementing AI and ML because it can handle the parallel processing required for these operations efficiently.

C. High powered CPUs.

High powered CPUs are beneficial for general-purpose computing tasks and some aspects of ML, but they are not as efficient as specialized hardware like GPUs for matrix multiplication and other ML-specific tasks.

D. Hardware supporting high precision floating point operations.

High precision floating point operations are important for scientific computing and some specific AI tasks, but for many ML applications, fast matrix multiplication is more critical than high precision alone.

Therefore, the correct answer isBbecause hardware supporting fast matrix multiplication, such as GPUs, is most suitable for the parallel processing requirements of machine learning​​.

Section2.2 – Data Preparationand4.1 – Challenges in Testing AI-Based Systemsdescribe difficulties in obtaining and managing large, representative datasets. AI-based systems requirerealistic, diverse, and representativedata reflecting real-world variations. The syllabus emphasizes that assembling such datasets is time-consuming, resource-intensive, and often constrained by availability, privacy, or domain complexity. Option B directly corresponds to these documented challenges.

Option A is incorrect: using the same implementation risksdefect masking, not preventing it; the syllabus warns against this practice. Option C is incorrect because real-world data naturally evolves, and the syllabus notes thatdriftis normal; expecting stable input data contradicts operational reality. Option D is incorrect: although data privacy is important, the syllabus does not claim that artificially generated data always requires legal approval, nor that sanitization/encryption is mandatory for synthetic data.

Thus,Option Baccurately reflects syllabus-defined difficulties in producing representative test data.

The ISTQB CT-AI syllabus states in Section2.10 – Explainability, Transparency, and Interpretabilitythat interpretability isuser-dependent, meaning different users understand explanations differently. This is because interpretability depends not only on the ML algorithm but also on the user’s domain knowledge, experience, and expectations. OptionBdirectly reflects this syllabus principle: interpretability testing must consideruser background, and explanations must be comprehensible to the intended user group.

Option A is incorrect because explainability testing requires substantial information about the model, data, and expected behavior—not just exploratory effort. Option C is incorrect because explainability isnotgenerally quantifiable through dynamic testing alone, and the syllabus does not assert model-type specificity in this way. Option D exaggerates LIME’s capabilities. LIME offers approximate local explanations, but cannotpreciselystate root causes; the syllabus emphasizes itslimitationsand that explanations areapproximations, not exact reasons.

Therefore,Option Bis the most syllabus-aligned and correct statement.

The ISTQB CT-AI syllabus (Section3.2 – Model Evaluation) specifies the correct usage oftraining,validation, andtestdatasets. It emphasizes that thetest dataset must be representative of the real operational dataand must beequivalent in distribution to the training and validation sets, ensuring a fair and unbiased evaluation. Option D precisely matches this requirement.

Option A contradicts the syllabus because validation and test sets servedifferent purposes: validation is for tuning, test is for final evaluation. Combining them undermines the reliability of results. Option B is incorrect because even with limited data, the syllabus recommends maintaining a test set or using techniques such ascross-validationrather than eliminating testing. Option C is wrong because equal distribution (33/33/33) isnot recommended; typically, the training set is much larger (e.g., 70–80%).

Thus, OptionDis the only statement aligned with the syllabus' guidance.

The syllabus states:

"Concept drift occurs when the operational environment changes without the trained model changing correspondingly. The outputs of the model become less accurate and less useful. Therefore, the operational model should be regularly evaluated against its acceptance criteria."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 7.6, Page 54 of 99)

Section1.7 – Neural Networksof the ISTQB CT-AI syllabus explains that neural networks consist of neurons connected by weighted links. During training,learning occurs by adjusting the weights on these connections. This is the essence of gradient descent and backpropagation. Option B correctly states this behavior: only theweightsare modified, not the activation functions, neuron counts, or architectural structure.

Option A is incorrect because a neuron’sbiasis not determined by previous activations; it is an independent trainable parameter added to the weighted input sum. Option C is incorrect because the syllabus states that asingle-layer perceptron is a valid type of neural network, although limited to linearly separable problems. Option D is incorrect because no rule requires the number of input neurons to exceed or equal the number of output neurons. Instead, input neurons correspond to thenumber of features, while output neurons correspond totasks or classes.

Therefore,Option Bprecisely reflects the syllabus definition of what changes during neural network training.

The ISTQB CT-AI syllabus (Section5.3 – AI Support for Defect Analysis) explains that AI can categorize defect reports using natural language processing or classification models. Categorization helps route defects efficiently and determine which areas of the system are affected. Thus, OptionCis correct: AI canidentify defect categories, supporting assignment and triage.

Option A is incorrect because ML cannot infer severity or consequence without domain-specific risk modeling. Option B inaccurately ties duplicate detection to criticality; the syllabus separatesduplicate detectionfromseverity analysis. Option D is incorrect because AI may assist in routing defects using categories, but the syllabus does not claim AI identifies specific developers.

Thus,Option Cis the syllabus-accurate statement.

The ISTQB CT-AI syllabus indicates inSection 5.2 – AI for Testingthat AI-generated test cases may not come withpredefined expected results. This is because test-case generation methods—such as evolutionary algorithms, reinforcement learning, or clustering-based sampling—produceinputs, but the tester must still determine the correct outputs. Therefore, OptionCis correct: expected results may not be available, especially when AI produces novel or previously unseen input combinations.

Option A is incorrect: AI-based test generation can improve coverage by exploring large input spaces. Option B is incorrect because the need for oracles does not depend on whether the test case is AI-generated. Option D is incorrect because the syllabus allows using a model as an oracle inback-to-back testing, provided limitations are understood.

Therefore,Option Cis the correct syllabus-aligned statement.

Setting model and algorithm hyperparameters is an essential step in the machine learning workflow, primarily occurring during the tuning phase.

Evaluating the model (A): This stage involves assessing the model's performance using metrics and does not typically include the setting of hyperparameters.

Deploying the model (B): Deployment is the stage where the model is put into production and used in real-world applications. Hyperparameters should already be set before this stage.

Tuning the model (C): This is the correct stage where hyperparameters are set. Tuning involves adjusting the hyperparameters to optimize the model's performance.

Data testing (D): Data testing involves ensuring the quality and integrity of the data used for training and testing the model. It does not include setting hyperparameters.

Hence, the most appropriate stage of the ML workflow to set model and algorithm hyperparameters isC. Tuning the model.

The syllabus describes metamorphic testing as:

“Testing involves defining metamorphic relations and then applying those relations to check that the transformations result in expected outcomes, even when the expected output of the system is unknown or not well-defined.”

In this scenario, applying the metamorphic relation (speed differences) and checking the transformed outcome (arrival times) fits the definition of metamorphic testing.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.5, page 69 of 99)

The syllabus states:

"Transparency: This is considered to be the ease with which the algorithm and training data used to generate the model can be determined."

Access to the training data is an example of transparency.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.7, page 24 of 99)

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification: This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options:

Reinforcement Learning: This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression: This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering: This involves grouping similar data points together without predefined classes, which is not the case here.

The syllabus explains that supervised learning requires correctly labeled data so the algorithm can learn the relationship between input features and output labels:

"In supervised learning, the algorithm creates the ML model from labeled data during the training phase. The labeled data is used to infer the relationship between the input data and output labels."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.1)

A. Black box attacks based on adversarial examples create an exact duplicate model of the original.

Black box attacks do not create an exact duplicate model. Instead, they exploit the model by querying it and using the outputs to craft adversarial examples without knowledge of the internal workings.

B. These attack examples cause a model to predict the correct class with slightly less accuracy even though they look like the original image.

Adversarial examples typically cause the model to predict the incorrect class rather than just reducing accuracy. These examples are designed to be visually indistinguishable from the original image but lead to incorrect classifications.

C. These attacks can't be prevented by retraining the model with these examples augmented to the training data.

This statement is incorrect because retraining the model with adversarial examples included in the training data can help the model learn to resist such attacks, a technique known as adversarial training.

D. These examples are model specific and are not likely to cause another model trained on the same task to fail.

Adversarial examples are often model-specific, meaning that they exploit the specific weaknesses of a particular model. While some adversarial examples might transfer between models, many are tailored to the specific model they were generated for and may not affect other models trained on the same task.

Therefore, the correct answer isDbecause adversarial examples are typically model-specific and may not cause another model trained on the same task to fail​​.

 Labelling Approaches:Among the options provided, pre-labeled datasets require the least time and effort because the data has already been labeled, eliminating the need for further manual or automated labeling efforts.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 4.5 Data Labelling for Supervised Learning, which discusses various approaches to data labeling, including pre-labeled datasets, and their associated time and effort requirements​​.

The ISTQB CT-AI syllabus describes theML data preparation workflowin Section2.2 – Data Preparation. Data preparation consists ofdata gathering,cleaning,transformation, andsampling. The syllabus emphasizes that one significant challenge duringdata gatheringis combining data frommultiple heterogeneous sources, which often differ in structure, quality, and format. Ensuring the resulting dataset is accurate, complete, and representative can be complex, making this a critical challenge in the ML workflow. This aligns directly with OptionC.

Option A is incorrect because erroneous data correction is part ofcleaning, not transformation. Option B contradicts the syllabus: while automation can help,not all steps should be automateddue to the need for expert oversight, especially in detecting subtle data quality issues. Option D is incorrect because sampling continues to involve risk—particularly around representativeness—and the syllabus emphasizes caution, not complacency.

Thus, OptionCis the only statement that accurately reflects the syllabus.

The ISTQB CT-AI syllabus distinguishes between several types of bias relevant in AI testing, includingsample bias,algorithmic bias, andinappropriate bias. In Section3.3 – Bias in AI-Based Systems, the syllabus stresses the importance of identifying biases that originate fromtraining data,model development, anddecision logic. Sample bias occurs when the training data does not adequately represent the population; algorithmic bias arises when the model produces systematically skewed results due to learned patterns; inappropriate bias involves ethically or socially problematic distortions in the outcomes. All three of these bias types directly affect theoutputs of the AI modeland are therefore highly relevant when testing an industrial inspection system intended to reliably detect defects. These biases can lead to defective items being missed or false alarms being raised, which impacts quality assurance significantly .

Automation bias, however, is fundamentally different. It refers to ahuman cognitive bias, where users (e.g., inspectors) overly trust or rely on the AI system’s output. While important in user-interaction testing, it isnota biaswithin the ML model itself. Since the question asks which bias isleast important when testing the model, automation bias can be legitimately deprioritized duringmodel-level testing. Therefore, OptionBis correct.

 Testing Autonomy:Testing for human handover when it should not be relinquishing control is the least appropriate because it contradicts the very definition of autonomous systems. The other tests are relevant to ensuring smooth operation and transitions between human and AI control.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Testing Autonomous AI-Based Systems and Testing for Human-AI Interaction​​​​. ​

The ISTQB CT-AI syllabus describes thatself-learning systems continuously adjust their behaviorduring operation as new data arrives. Section4.1 – Challenges of Testing AI-Based Systemshighlights that such systems are vulnerable todata attacks, particularly through adversarial inputs, poisoning, or malicious drift. The risk arises because unexpected changes in the input distribution may alter the learned model in harmful ways. OptionD – Unexpected changescorresponds directly to this syllabus-defined risk.

Option A refers to system specification issues but does not relate to data attacks. Option B discusses environment complexity, which makes testing difficult but is not tied to adversarial threats. Option C (insufficient testing time) affects quality but does not specifically increase vulnerability to malicious data manipulation.

Unexpected changes—including data drift, poisoned samples, or maliciously constructed training data—pose the greatest risk. When a self-learning system adapts to altered data patterns, it may unknowingly learn incorrect associations, causing model degradation or manipulation. Therefore,Option Dcorrectly identifies the challenge that increases exposure to data attacks.

Automation bias is defined in Section4.4 – Human Factors in AI Testingof the ISTQB CT-AI syllabus. It refers to the human tendency to overly trust, rely on, or defer to automated system outputs. The syllabus explains that this bias arises especially indecision-support systems, where humans may accept AI judgments without adequate verification. This aligns directly with Option B.

Option A is incorrect: automation biasdoesinfluence testing, especially when testers rely excessively on AI outputs. The syllabus cautions about testers adopting the same cognitive biases as end users. Option C is incorrect because autonomous systems are not the primary context; rather,systems supporting human decisionsare most impacted. Option D is incorrect because the quality of human inputmatters significantly, and poorly designed user studies can mask or distort automation bias.

Thus,Option Bis the syllabus-accurate description of automation bias.

The ISTQB CT-AI syllabus explains that reusing pre-trained models is strongly related tosimilarity between the original task and the new task. Section1.8 – Pre-trained Models and Transfer Learningstates that reuse is effective when the new task is similar to the original one, such as adapting a cat-classifier to classify dog breeds. The syllabus warns about risks related toinput differences,data preparation inconsistencies,inherited shortcomings, andexplainability issues. These are legitimate objections (matching options A, B, and C) because large differences in image inputs or patterns can undermine transfer learning; misclassification risk can increase; and explainability often decreases when reusing pre-trained models .

However,output differences are NOT a valid concernhere. Both the leaf-based and flower-based ML models classifythe same plant species, meaning theiroutputs are identical. The syllabus does not identify output mismatch as a transfer-learning risk. Real risks concerninputs,bias inheritance,model transparency, andtraining differences—not output labels. Therefore, OptionDdescribes an objection that can be safelyneglected, because output classes are the same and do not hinder reuse.

The ISTQB CT-AI syllabus defines these two concepts clearly inSection 2.1 – Flexibility and Adaptability. Flexibility is described as the ability of a system to operate in situationsnot explicitly covered in its original requirements, while adaptability refers to how easily the system can bemodifiedto handle new environments or conditions. The syllabus stresses that both flexibility and adaptability are crucial, particularly inself-learning AI systemsthat may need to respond to changes in their environment and adjust their behavior accordingly. It states that systems must be capable of determiningwhenandhowto adjust behavior in evolving situations, especially when the operational environment is not fully known at deployment time . This directly aligns with OptionA.

Option B reverses definitions—the syllabus states flexibility (not adaptability) relates to unspecified situations. Option C is incorrect: self-learning systems requirebothflexibility and adaptability; they are not categorized as one or the other. Option D incorrectly defines flexibility; the syllabus defines adaptability—not flexibility—as ease of modification.

Thus,Option Acorrectly reflects the syllabus.

The syllabus states that while pairwise testing is effective at finding defects by reducing the number of test cases needed, the resulting test suite can still be extensive and require automation:

"Even the use of pairwise testing can result in extensive test suites… automation and virtual test environments often become necessary to allow the required tests to be run."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.2, Page 67 of 99)

 AI-Based Autonomous Functions:An AI-based autonomous system is one that can respond to its environment without human intervention. The other options either involve human decisions or do not use AI at all.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Autonomy and Testing Autonomous AI-Based Systems​​​​.

The syllabus mentions that experience-based testing and EDA are effective for detecting biases:

“Experience-based testing can be used to verify that the training dataset is operationally relevant and identify potential sources of bias. EDA is also useful for exploring the data and understanding any relationships that might lead to bias in the model.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.3, page 58 of 99)

From the confusion matrix:

True Positives (TP) = 15

False Positives (FP) = 5

False Negatives (FN) = 15

True Negatives (TN) = 65

Accuracy= (TP + TN) / Total

= (15 + 65) / 100

=80%

Precision= TP / (TP + FP)

= 15 / (15 + 5)

= 15 / 20

=75%

Section3.2 – Functional Performance Criteriain the syllabus explains accuracy and precision exactly these ways when evaluating ML classification performance.

Option B is therefore the only correct pair of values.

The ISTQB CT-AI syllabus emphasizes that testing AI-based systems requirescross-functional collaborationandexperience-based testingwhen parts of the team lack domain knowledge. In this scenario, the ML expert understands ML and dataset preparation but lacks knowledge ofcamera system behavior, the device’s operational data pipeline, and end-user workflows. The remainder of the team understands the domain and system testing but not ML. Section4.4 – Human Factors and AI Testingand4.3 – System Testing of AI Componentshighlight that when domain understanding is unevenly distributed,experience-based testing conducted by the full team(testers, developers, domain experts) is the most effective approach. This ensures that AI outputs align with actual user expectations and system behavior. OptionCaligns exactly with this principle.

Option A is too limited and does not address the need to validate ML integration. Option B is incorrect because reusing old test cases overlooks AI-specific risks in the operating data pipeline. Option D is useful but focuses only on data representativeness, not system-level user validation. Therefore,Option Cis the best, syllabus-aligned test approach.

The syllabus explains the fundamental distinction betweenconventional systemsandAI-based systems using supervised machine learningin Section1.3 – AI-Based and Conventional Systems. A conventional system relies on human-programmed logic—such as branches, conditions, and explicit rules—to interpret input data. The system behaves exactly as specified by its developers.

In contrast,AI systems using supervised learning automatically learn patternsfrom labeled data. The syllabus states that“patterns in data are used by the system to determine how it should react in the future… The AI determines on its own what patterns or features in the data can be used”. This aligns directly with Option C: an AI system identifies relevant diagnostic patterns in X-ray images during training, whereas a conventional system requires human experts to explicitly program those patterns.

Option A is incorrect because AI outputs are typicallylessexplainable, not more. Option B is incorrect because both systems can use thesame X-ray images; ML does not require structurally different images. Option D is oversimplified and not fully accurate; while training data is central to ML, AI systems also include architecture, algorithms, and preprocessing—not just data.

Thus,Option Cis the correct and syllabus-aligned answer.

The ISTQB CT-AI syllabus introducesAI-specific quality characteristics, includingevolution,functional safety,compatibility, andbias-related data quality. Section5.1 – AI-Specific Test Objectivesexplains thatevolutionrefers to an AI system’s capability to continue improving or at least maintain performance as it undergoes additional training. GPT_Legal is explicitly described as aself-learning systemexpected to:

continuously reduce false positives,

achieve weekly accuracy improvements of 10%,

reach and maintain 90% accuracy,

adapt to new environments (patent law firm → corporate legal department).

This aligns perfectly with the syllabus definition ofevidence of evolution: ensuring the model doesnot degradeas additional training data is introduced. OptionBtherefore directly supports the described acceptance criteria for this evolving, self-learning application.

Option A (functional safety) is irrelevant because patent searching and drafting do not constitute safety-critical domains. Option C (compatibility) is necessary but not the primary AI-specific objective. Option D addresses bias, which is important but not central to the described performance and continuous-learning expectations.

Thus,Option Bis the most appropriate AI-specific test objective.

The ISTQB CT-AI syllabus (Section5.2 – AI for Testing) explains that various AI approaches can support testing activities. Probabilistic methods—one of the three major AI technique groups—are used topredict system failures, especially when dealing with uncertainty, likelihood estimation, and reliability analysis. This aligns precisely with OptionB.

Option A is incorrect because regression test optimization is typically performed usingsearch-based optimization, not classification. Option C is incorrect because fuzzy logic is more suited to reasoning under vagueness, not generating test cases. Option D is incorrect: defect prediction relies on statistical learning or classification models, not computational optimization.

Thus,Option Bis the most syllabus-consistent mapping of AI techniques to testing tasks.