TMMi-P_Syll2.1 TMMi Test Maturity Model Integration Professional (TMMi-P) Questions and Answers

In an Agile context, the emphasis is typically placed on lightweight, flexible documentation. Test documentation should be "just enough" to support the test process, and excessive documentation may be seen as unnecessary overhead. Therefore, calling for more rigorous and thorough documentation is not in line with Agile principles. Instead, Agile favours working software and collaboration over comprehensive documentation, supported by mentoring and just-in-time guidance rather than formalised documentation​.

A test policy is a high-level document that outlines the objectives, goals, and strategic views regarding testing. It serves to align testing activities with the organisation's business objectives and quality goals. One of the critical elements in a test policy is defining the quality levels to be achieved, which ensures that testing aligns with the expectations for product quality, both functional and non-functional. Other aspects, such as risk categories, test types, and regression testing approaches, are typically defined in more detailed test strategies or plans, not in the high-level policy.

The statement that Specific Goal 1 of the Test Measurement (4.1) process area covers the communication of test measurement results after analysis is FALSE. Specific Goal 1 focuses on aligning test measurement and analysis activities, which involves setting measurement objectives, specifying the measures to be taken, and establishing procedures for data collection, storage, and analysis. However, the communication of results comes under Specific Goal 2, which deals with the analysis and reporting of test measurement results.

TMMi References:

Test Measurement at TMMi Level 4 includes defining measures, data collection, and storage as part of aligning test measurement and analysis activities, while communicating results is part of a different goal​​.

The specific goal during the implementation of the "Test Planning" process area at TMMi Level 2 is to establish well-founded plans for managing and performing the testing activities. One of the crucial objectives is to "Establish a Master Test Plan" at TMMi Level 3, particularly under the "Test Lifecycle and Integration" process area. This is designed to ensure that a coherent, high-level plan integrates the activities across all levels of testing.

The process includes defining a test approach based on identified risks and involves the creation of the test estimates, schedules, and plans. Test plans should align with organisational strategies and cover test activities for all relevant phases of the development cycle​​.

The specific practice "Define product risk categories and parameters" is part of the Test Policy and Strategy process area in TMMi. This practice involves identifying and categorizing the product risks to guide testing efforts and establish a clear understanding of potential product issues. Product risk categories help in determining the focus areas for testing, ensuring that testing is prioritized based on business and technical risks.

TMMi References:

The practice of defining product risk categories is directly linked to the Test Policy and Strategy process area at TMMi Level 2, where risk-based testing approaches are established​​.

The statement that all defects should be analyzed to prevent as many defects as possible is FALSE. In the Defect Prevention process area at TMMi Level 5, only a subset of defects—those that are recurring or considered to be of significant impact—are analyzed. Analyzing all defects would be inefficient and not always necessary. Instead, a targeted approach is used to identify common causes of defects and implement actions to prevent their recurrence.

TMMi References:

The Defect Prevention process area is based on the analysis of a subset of defects to find and eliminate common causes​​.

This statement is false because informal assessments do not provide an official result on test process maturity. Informal assessments are designed to give indicative information only and focus on identifying key areas for improvement without leading to a formal maturity rating or certification. In contrast, formal assessments are more comprehensive and provide an official maturity rating when conducted under an accredited assessment method.

An indirect benefit of a test improvement programme is improved staff motivation. While direct benefits, such as shortened lead time or higher defect detection rates, are focused on measurable improvements, indirect benefits include enhanced morale and engagement of the staff involved. A well-structured test improvement programme can lead to better recognition of the testing function within the organisation, leading to increased satisfaction and motivation among testing professionals.

The process area Advanced Reviews belongs to TMMi Level 4 (Measured). At this maturity level, the focus is on quantitatively measuring and managing product quality, and advanced review techniques are employed to integrate peer reviews (static testing) with dynamic testing. These advanced reviews play a key role in ensuring that product quality is assessed early in the lifecycle and that testing is driven by measurable goals.

In an Agile context, the SCRUM team model is a typical format for a test organisation. Testing is integrated into the cross-functional SCRUM team, with testers working alongside developers and product owners throughout the sprint. This ensures that testing is a continuous activity, rather than a separate phase. The test organisation in Agile environments focuses on collaboration and team-based activities, rather than a separate or independent test group​​.

CMMI and TMMi complement each other in many ways. While CMMI focuses on improving broader software development processes, TMMi specifically addresses testing. Implementing CMMI provides value to test organizations by enabling a structured and continuous improvement approach that can incorporate TMMi as part of the overall process.

CMMI’s continuous representation allows organizations to implement specific process areas related to testing (like verification and validation) and integrate TMMi practices for continuous improvement in their testing processes. Organizations following TMMi may also refer to CMMI practices for complementary process improvements in areas such as configuration management, planning, and performance measurement.

TMMi References:

TMMi outlines how CMMI’s process areas, such as Verification and Validation, provide critical value for testing organizations​​.

CMMI’s structure supports continuous process improvement that aligns well with TMMi’s structured approach to test process maturity​.

The absence of a master test plan spanning multiple test levels points to a weakness in the Test Lifecycle and Integration process area, which is a TMMi Level 3 process area. This process area focuses on ensuring that testing is integrated throughout the development lifecycle and includes the development of comprehensive test plans (including master test plans) that cover multiple test levels and align with the overall testing strategy.

TMMi References:

The Test Lifecycle and Integration process area at TMMi Level 3 addresses the need for comprehensive and integrated test planning across all test levels​​.

This statement is FALSE because TMMi does not focus only on higher test levels like system and acceptance testing. In fact, TMMi covers all test levels, including lower levels like unit and integration testing, as well as static testing (e.g., reviews and inspections).

TMMi is designed to address testing at all levels of the software lifecycle and goes beyond dynamic testing (e.g., system and acceptance) to include static testing techniques as well. The model is comprehensive and is intended to support testing in both systems and software engineering. Furthermore, TMMi addresses the four cornerstones of structured testing: lifecycle, techniques, infrastructure, and organization. These are essential to ensure a thorough and structured approach to improving test processes and aligning them with business goals.

TMMi References:

TMMi documentation clearly indicates that it covers all test levels including static testing (such as peer reviews) and dynamic testing across different stages of development​​.

It is lifecycle-independent and can support different models, such as Agile, DevOps, and traditional V-models​.

Peer Reviews is NOT a TMMi Level 2 process area. Peer Reviews is a process area at TMMi Level 3. At Level 3, organizations formalize their testing processes, including peer reviews, which involve static testing activities like inspections and walkthroughs to identify defects early.

Test Policy and Strategy, Test Monitoring and Control, and Test Environment are all process areas at TMMi Level 2.

TMMi References:

Peer Reviews are introduced at TMMi Level 3, whereas Test Policy and Strategy, Test Monitoring and Control, and Test Environment are process areas at Level 2​​.

The three TMMi Level 5 process areas are as follows:

Defect Prevention (A): The key specific goal here is to determine common causes of defects. This process involves analysing defect patterns and establishing actions to prevent similar defects in the future.

Quality Control (B): This area focuses on evaluating new testing technologies to determine their impact on the testing process. Statistical methods help predict product quality, and these evaluations drive improvements in the test process.

Test Process Optimization (C): The goal here is to establish a statistically controlled test process, where baselines are set, and improvements are implemented in an orderly manner​​​.

A typical business reason for starting a test improvement program is to achieve higher productivity. Test process improvements can lead to better efficiency in test execution, reduction of defects, and optimised resource usage, all of which contribute to the overall productivity of the organisation. By improving testing processes, organisations can reduce time-to-market and minimise costs associated with fixing defects post-production, which in turn boosts productivity.

For safety-critical systems, reliability testing is a key concern. The identified shortcoming relates to the lack of formal techniques and approaches for reliability testing. The Non-Functional Testing process area, at TMMi Level 3, addresses testing non-functional aspects like reliability, performance, and usability. This process area would need specific attention as it provides a structured approach to assess non-functional risks, define testing techniques, and execute tests to verify the reliability of the product, which is critical for medical systems.

Regular code reviews and architectural reviews are part of the Advanced Reviews process area, which belongs to TMMi Level 4. This process area emphasizes conducting static reviews (e.g., code and design reviews) early in the lifecycle to measure product quality and adjust the testing strategy based on the insights gained from these reviews.

TMMi References:

The Advanced Reviews process area in TMMi Level 4 aims to ensure that static testing activities such as code and architectural reviews are used to measure product quality early in the lifecycle​​.

In a staged model like TMMi, an organisation follows a predefined improvement path that includes a set of process areas organised by maturity levels. Each level builds upon the previous one, and the organisation progresses in a step-by-step manner through these maturity levels to improve its processes. The staged model is well-structured and prescriptive, offering a clear improvement roadmap for the organisation.

One of the key differences between informal and formal assessments is that informal assessments are not as rigorous. Informal assessments are more flexible, quicker, and less costly; however, they do not provide the same level of detail or accuracy as formal assessments. The statement that "informal assessments provide a result just as accurate as with a formal assessment" is therefore false. Formal assessments involve multiple types of evidence and a more thorough evaluation, while informal assessments typically rely on fewer types of evidence (often only interviews)​​.

The description provided—where testing is multi-level, with distinct test levels (component, integration, system, and acceptance), and where the processes of testing and debugging are differentiated—applies to TMMi Level 2 (Managed). At this maturity level, testing is separated from debugging, and there are formalized processes for different testing levels. A company-wide or program-wide test strategy is established, and specific testing objectives are defined for each test level.

TMMi References:

The TMMi Level 2 framework defines structured testing with multiple levels, including component, integration, system, and acceptance testing, as well as the separation of testing and debugging​​.

In the scenario described, where an organisation suffers from poor test estimations despite many practices being in place, the process area that requires attention is Test Planning. Specifically, SG 3 (Establish Test Estimates) focuses on developing well-founded estimates for testing effort and cost. This includes creating a top-level work breakdown structure, defining the test lifecycle, and determining estimates for the test effort and costs associated with different test activities. By addressing these practices, the organisation can improve its ability to generate accurate test estimations, leading to better project planning and resource allocation.

Planning poker is an Agile technique commonly used to estimate effort, and it can be adapted for use in the Test Planning process area to help estimate test efforts (SG3: Establish Test Estimates). This collaborative technique brings stakeholders together to estimate the effort required for tasks by assigning numerical values in a consensus-driven manner. It is particularly effective in Agile environments where estimation is iterative and frequently updated based on evolving project needs​.

"Provide Necessary Test Training" is part of the Test Training Program process area, not the Test Organization process area. Test Organization focuses on establishing a structured group responsible for testing, defining roles, career paths, and implementing process improvements. Therefore, providing test training is not a specific goal of Test Organization but rather falls under the Test Training Program process area.

In the TMMi framework, specific goals can indeed have different specific practices across maturity levels. The practices associated with a specific goal evolve as an organization matures, reflecting the need for more sophisticated approaches at higher levels of maturity. For example, test design practices at Level 2 are more basic compared to the more comprehensive practices at Level 3, which include integration with other lifecycle processes.

The other statements are incorrect. For instance, sub-practices are informative and not mandatory, and while generic goals have the same set of practices across process areas, specific goals vary by maturity level.

TMMi References:

TMMi process areas include specific goals that may have different practices depending on the maturity level, as the process areas evolve with higher maturity​​.

To improve the management and control of test environments, one of the specific practices under SG 3 (Manage and Control Test Environments) of the Test Environment process area is to coordinate the availability and usage of test environments. This involves managing reservations, scheduling, and ensuring the efficient use of test environments across different projects. This practice is essential for ensuring that the test environments are available when needed, reducing delays, and improving the overall efficiency of the testing process.

To improve the quality of test cases and reduce defects in production, addressing the practice of "Identify and Prioritize Test Conditions" under Specific Goal 1 (SG 1) of the Test Design and Execution process area is essential. This specific practice involves identifying conditions based on an analysis of the test items and prioritising them based on risk and other factors. By improving the identification and prioritisation of test conditions, organisations can ensure that they focus on the most critical aspects of the system, thereby increasing the effectiveness of their testing efforts and reducing defects in production.

The TMMi model is designed to be versatile and applicable across all test levels (including static and dynamic testing) and all types of development methodologies, such as waterfall, Agile, and DevOps. It provides comprehensive coverage of the testing process, ensuring that all critical aspects of testing—such as lifecycle processes, techniques, infrastructure, and organization—are addressed.

TMMi is not limited to specific development methods or organizational sizes; it is lifecycle-independent and can be used effectively with various development approaches, making it widely applicable in both small and large organizations.

TMMi References:

TMMi covers all test levels, including both static testing (such as reviews) and dynamic testing at multiple stages, and is applicable across different software development lifecycles​​.

At TMMi Level 3, Test Lifecycle and Integration builds on the practices of Test Planning introduced at TMMi Level 2. This process area focuses on establishing a standardised test lifecycle, integrating the test process with the development lifecycle, and coordinating the testing effort across multiple levels. It also includes creating a master test plan that builds upon the risk assessments, test estimates, and approaches defined at Level 2 during Test Planning.

By further refining these practices and introducing lifecycle integration, Test Lifecycle and Integration ensures that the testing processes are consistent, well-defined, and fully embedded into the overall project lifecycle​​.

Valid reasons to perform a TMMi assessment include identifying areas that need improvement (2), determining which accomplishments have been made (3), and using the assessment as a basis to become formally TMMi certified (4). Assessing individual tester performance (1) is not a typical objective of TMMi assessments, which focus more on process maturity and improvement rather than individual performance evaluation.