300-640 Implementing Cisco Data Center AI Infrastructure (DCAI) Questions and Answers

NVIDIA Collective Communications Library (NCCL) is the appropriate framework for validating communication performance between NVIDIA GPU nodes. NCCL provides highly optimized implementations of collective operations such as AllReduce, AllGather, Reduce, Broadcast, and ReduceScatter. These operations closely represent the communication patterns generated by distributed AI training and therefore provide much more meaningful validation of GPU-cluster performance than conventional network-throughput tests.

NCCL-based benchmarks can reveal whether GPU-to-GPU communication is achieving expected bandwidth and latency across NVLink, PCIe, and the external backend network. They are particularly useful when validating RoCEv2 fabrics because distributed training performance depends on collective communication efficiency rather than simple host-to-host TCP throughput.

iPerf measures conventional TCP or UDP network performance between hosts and is useful for basic network validation, but it does not exercise NVIDIA GPU collective communications. Spirent and Ixia provide sophisticated packet-generation and network-testing platforms, yet they test the network primarily from a packet-forwarding perspective rather than measuring actual GPU collective-operation behavior.

For a newly deployed NVIDIA AI cluster, NCCL benchmarking therefore provides workload-representative validation of the end-to-end GPU communication architecture and can expose problems involving topology, NIC configuration, congestion control, bandwidth imbalance, or fabric design.

Study Guide Reference: AI Infrastructure Operations and Troubleshooting — performance benchmarking, NVIDIA GPU communications, backend-fabric validation, and distributed AI workload testing.

===============

Congestion Notification Packets are control-feedback messages used by RoCEv2 congestion-management mechanisms. When a receiving endpoint observes an ECN-marked packet, it generates a CNP toward the sender, instructing the transmitting endpoint to reduce its transmission rate. Losing these packets would delay congestion response and allow queues to continue building.

Cisco's validated RoCEv2-over-VXLAN design explicitly states that CNP packets use DSCP 48 and are classified into a high-priority queue . In Cisco's example, ordinary RoCEv2 traffic is mapped to QoS group 3, while CNP is mapped to QoS group 7. Cisco further states that the high-priority classification ensures that CNP packets are not dropped in the network.

Traffic shaping in option A would regulate CNP transmission rather than guarantee timely feedback. Option C places CNP in the same potentially congested queue as the data traffic that generated the congestion, defeating the purpose of reliable feedback. Option D is also incorrect because PFC is intended to protect a lossless data priority; CNP itself should be forwarded through a high-priority queue and should not depend on pause behavior.

Study Guide Reference: AI Infrastructure Deployment and Data Management — RoCEv2, ECN, CNP, QoS classification, and congestion management.

===============

The UCS X210c M8 combined with the UCS X580p PCIe Node satisfies the complete requirement set. The X210c M8 uses Intel Xeon 6 processors, while the X580p provides modular GPU expansion and supports up to four PCIe GPUs. Cisco specifically positions this combination for generative AI, inference, and other GPU-accelerated workloads. Management is policy-driven through Cisco Intersight, providing the required cloud-based operational model.

The X580p also supports NVIDIA H200 NVL GPUs and NVLink Bridge capabilities. This is significant where low-latency communication is required between specific GPU pairs because NVLink provides substantially higher-bandwidth GPU-to-GPU communication than standard PCIe-only communication. Cisco's UCS ordering guidance specifies that H200 NVL supports NVL Bridge configurations with a minimum of two GPUs.

Options C and D are high-density AI systems, but they use AMD EPYC-based architectures rather than satisfying the stated Intel CPU requirement. Option A likewise does not match the specified X-Series composability, cloud-managed GPU resource model, and Intel-based requirement as directly as the X210c M8 configuration.

Study Guide Reference: 2.0 AI Infrastructure Components and Architecture — 2.2 Compute deployment; 2.4 power and efficiency considerations.

===============

In an Intersight Managed Mode UCS domain, DNS and NTP configuration is policy-controlled through the UCS Domain Profile deployed to the Fabric Interconnects. Changes entered locally through the Device Console can restore temporary reachability, but the policy configuration maintained by Intersight remains authoritative. If the Domain Profile still contains the previous DNS or NTP configuration, a subsequent deployment can overwrite the locally modified values.

Cisco documents that the Domain Profile references both the Network Connectivity Policy , which contains DNS settings, and the NTP Policy , which defines the NTP servers. The appropriate persistent remediation is therefore to edit the configuration associated with the deployed Domain Profile and redeploy it.

The IMC Access Policy applies primarily to server-management addressing and access rather than Fabric Interconnect DNS/NTP persistence. A Device Connector policy does not supply these domain DNS settings. Option C incorrectly applies a Network Connectivity Policy to the blade chassis; for the Fabric Interconnect domain, the relevant network-connectivity configuration is referenced through the UCS Domain Profile.

Cisco also advises that DNS and NTP should be configured on the management interface rather than being treated as Device Connector settings.

Study Guide Reference: AI Infrastructure Deployment and Data Management — Intersight Managed Mode, UCS Domain Profiles, DNS/NTP policies, and management connectivity.

Rapid provisioning supports scalability by reducing the time required to make compute, GPU, networking, storage, and orchestration resources available for workloads. The most appropriate description is therefore the efficient launch of computational platforms . When new AI capacity can be provisioned consistently and quickly, organizations can expand training environments, increase inference capacity, respond to workload bursts, and introduce additional accelerator resources without lengthy manual configuration.

Cisco's DCAI objectives explicitly cover orchestration as a component of AI environments and require candidates to evaluate infrastructure scalability. Cisco also emphasizes that AI infrastructure must scale to accommodate changing workloads and that cloud-oriented infrastructure offers rapid access to computational resources when demand changes.

Option A introduces fragmented management, which generally increases operational complexity. Option B restricts connectivity and therefore conflicts with scalable distributed AI execution. Option C explicitly delays resource availability and is the opposite of rapid provisioning.

The architectural objective is to make infrastructure consumption repeatable and automated. Efficient provisioning allows the underlying platform to expand at the pace demanded by GPU-intensive training and production inference rather than becoming an operational bottleneck.

Study Guide Reference: 1.0 AI Fundamentals and Applications — 1.5.d Orchestration; AI infrastructure scalability concepts.

===============

A rail-optimized architecture is intentionally designed so that communication among corresponding GPU ranks within a single scalable unit can remain on the rail or leaf switches. This provides single-hop forwarding and avoids unnecessary traversal through spine switches, reducing latency for collective communication.

Spines become necessary when the deployment expands beyond one scalable unit. Cisco explicitly documents that multiple scalable units can be interconnected through leaf-and-spine switching. Traffic among GPUs inside one scalable unit benefits from rail-optimized single-hop communication, whereas traffic between GPUs located in different scalable units traverses the spine layer.

Therefore, option C precisely describes when spine switches provide architectural value: scaling the AI fabric beyond the boundaries of one scalable unit .

Option A is inaccurate because the rail design already provides the required local GPU bandwidth within the scalable unit; adding spines is not primarily intended to increase local rail bandwidth. Option B mischaracterizes the principal design purpose. Although additional topology can contribute to resiliency, spines are introduced fundamentally for interconnecting scalable units and enabling scale-out. Option D is also incorrect because increased buffering is not the architectural reason for introducing a spine tier.

Study Guide Reference: AI Infrastructure Components and Architecture — rail-optimized topology, scalable units, nonblocking network design, latency, and horizontal scaling.

===============

Two settings determine the required behavior. First, Power Restore must be configured as Always On . Cisco defines this setting as instructing the server to power on after power is restored following a loss event. Immediate is not one of the documented Power Restore states; supported states include Last State, Always On, and Always Off.

Second, Power Priority should be High . Cisco explains that Power Priority determines initial power allocation among servers. A server assigned a higher priority receives preference when power is being allocated, which directly supports the requirement that these systems restart as soon as possible. Cisco validated designs explicitly show Power Profiling enabled with Power Priority set to High for performance-oriented deployments.

Power Profiling permits the platform to determine server power requirements. The Processor Package Power Limit should remain at Default for the X-Series scenario. Cisco documentation also notes that the selectable PPL setting is specifically applicable to particular C225/C245 M8 systems rather than being the mechanism for prioritizing X-Series restart behavior.

Option C provides Always On but fails to assign High power priority. Options A and D contain the invalid Immediate restore setting.

Study Guide Reference: AI Infrastructure Operations and Troubleshooting — UCS power management, recovery after power loss, power prioritization, and infrastructure availability.

The organization already owns third-party AI servers and NVMe storage, so it requires a network-centric AI fabric rather than a Cisco full-stack solution that bundles UCS compute. Cisco Nexus Hyperfabric supports bring-your-own infrastructure scenarios while providing cloud-based design, deployment, management, telemetry, and lifecycle operations.

The licensing distinction is critical. Cisco's current Nexus Hyperfabric documentation defines Essentials primarily for general-purpose fabrics, whereas Premier is the entitlement tier designed for AI fabrics. Premier includes AI-specific functionality such as alignment with NVIDIA reference architectures, adaptive routing support, AI-validated blueprints, automated lossless backend networking, and monitoring between switches and servers.

Option A uses the correct bring-your-own model but the wrong subscription tier for an AI fabric. Option D represents the full-stack Hyperfabric AI model, which includes Cisco UCS servers and therefore conflicts with the requirement to retain existing third-party compute. Option C does not match the requested Cisco 6000 Series cloud-managed Hyperfabric architecture.

Accordingly, BYO AI Hyperfabric using Cisco 6000 Series switches with the Premier entitlement satisfies both reuse of existing infrastructure and AI-specific cloud-managed networking.

Study Guide Reference: 3.3.c Hyperfabric — deploy AI-ready fabrics using Cisco orchestration tools.

===============

Cisco UCS X-Series Direct is specifically designed for edge, retail, remote-office, and branch-office environments that require the operational characteristics of UCS without deploying separate top-of-rack Fabric Interconnect appliances. It combines the UCS X9508 chassis with integrated UCS 9108 100G Fabric Interconnect modules and supports centralized management through Cisco Intersight.

The decisive requirement is 200 Gbps of unified fabric connectivity per compute node . Cisco documents that two UCS 9108 100G Fabric Interconnect modules provide a total of 200 Gbps per compute node. The solution can support up to eight compute nodes per chassis and is explicitly positioned for remote and branch-office deployments.

The S3260 is primarily a storage-intensive UCS platform and does not represent the required branch-oriented modular compute architecture. Unified Edge is purpose-built for edge locations, but the specific 200-Gbps-per-compute-node requirement aligns directly with X-Series Direct. The C885A M8 is a high-density rack server for GPU-intensive AI workloads rather than a chassis solution optimized for these branch deployment requirements.

Study Guide Reference: 2.0 AI Infrastructure Components and Architecture — 2.2 compute scalability and redundancy; 2.5 hybrid AI deployment.

===============

Configuring an email address as a notification endpoint is not sufficient by itself. Cisco Nexus Hyperfabric requires the recipient endpoint to undergo email verification . After an email endpoint is created, Hyperfabric sends a verification code; the administrator must enter that code and confirm that the endpoint status is verified and active. An unverified address therefore causes the condition described in option B.

The endpoint must also be associated with a notification policy . Cisco documents that assertion notifications are configured by selecting one or more fabrics and one or more notification endpoints in the notification policy. If the relevant fabric and the newly created email endpoint are not paired in the policy, assertion messages are not sent to that address. This makes option C correct.

Options A and E incorrectly assume that Hyperfabric directly uses the organization's own email server. Nexus Hyperfabric provides the notification service through its controller architecture. Option D similarly does not represent the required notification configuration described by Cisco.

The correct operational workflow is therefore: create the email endpoint, complete verification, then associate the desired fabric and endpoint through the notification policy.

Study Guide Reference: AI Infrastructure Operations and Troubleshooting — Hyperfabric monitoring, assertions, notification endpoints, and notification policies.

===============

A UCS Domain Profile provides the configuration model for a pair of Cisco UCS Fabric Interconnects operating in Intersight Managed Mode. Cisco explicitly states that the Domain Profile allows configuration of FI ports and port channels and defines the VLANs and VSANs used by the domain. This makes option C an exact match for the functionality described.

The Domain Profile references reusable policies rather than embedding every setting directly. VLAN and VSAN policies are associated during the VLAN and VSAN configuration stage, Port Policies are assigned during port configuration, and additional management or compute-related domain policies are attached subsequently. When the Domain Profile is deployed, Intersight applies the assembled configuration to the assigned Fabric Interconnect domain.

A Chassis Profile configures chassis-specific properties and cannot replace domain-level network configuration. A Server Profile configures an individual server's identity, boot configuration, vNICs, vHBAs, firmware, BIOS, storage, and similar server-specific characteristics. Unified Edge uses its own distributed edge management model and is not the FI configuration object in question.

This distinction is fundamental in Intersight Managed Mode: Domain Profiles configure the Fabric Interconnect domain, while Server Profiles configure individual compute resources attached to that domain.

Study Guide Reference: AI Infrastructure Deployment and Data Management — Cisco Intersight, UCS Domain Profiles, VLAN/VSAN policies, and port configuration.

===============

RoCEv2 traffic must be mapped to a no-drop QoS class when lossless Ethernet operation is required. Cisco's current Intersight VIC configuration guidance provides an explicit configuration for this requirement: create a System QoS policy, select Platinum as the priority, and clear the Allow Packet Drops checkbox. The policy is then associated with the UCS Domain Profile.

Cisco's storage best-practice documentation gives the same configuration for NVMe over RoCEv2: Platinum enabled, Allow Packet Drops unchecked, and a jumbo MTU—commonly 9216 at the system-class level. This establishes the UCS-side no-drop class required for RDMA traffic.

Disabling Platinum, as proposed in option A, would remove the intended lossless traffic class. Enabling every system class in option B does not automatically make RoCEv2 lossless because packet-drop behavior remains a property of the selected class. Disabling Gold in option C is unrelated to ensuring that the RoCE traffic class receives no-drop treatment.

A complete RoCE design must maintain consistent QoS classification and no-drop handling end to end. However, for the System QoS policy requested here, option D is the precise Cisco configuration.

Study Guide Reference: AI Infrastructure Deployment and Data Management — RoCEv2, PFC, lossless Ethernet, and UCS System QoS.

===============

Priority Flow Control is a foundational mechanism used to provide lossless Ethernet behavior for RoCEv2. PFC operates on individual Ethernet priorities and allows a congested receiver or switch to pause transmission for the affected priority while permitting traffic in other classes to continue. Cisco's AI networking guidance explicitly states that PFC is enabled for RoCEv2 traffic to prevent packet drops caused by congestion.

This characteristic is critical for RDMA workloads because packet loss can trigger retransmission behavior and significantly degrade GPU collective communication performance. In an AI backend fabric, RoCEv2 is normally placed in a no-drop QoS class with PFC configured for that traffic class. ECN is commonly used alongside PFC to provide proactive congestion signaling, but PFC is the mechanism that directly prevents loss when buffers approach critical utilization.

Traffic shaping in option A may moderate burst rates but does not establish lossless transport. A high DSCP marking in option C provides classification and forwarding preference, not loss prevention. Weighted fair queuing in option D allocates scheduling bandwidth but also does not guarantee that packets will not be discarded during congestion.

Study Guide Reference: 3.1.a Congestion control mechanisms — PFC, ECN, ETS; 3.1.b RoCE/RoCEv2; 3.1.c QoS.

===============

The UCS X9516 X-Fabric Module provides PCIe Gen5 connectivity between supported M8 compute nodes and X580p PCIe nodes. Before enabling that connectivity, the affected compute nodes must be powered down and equipped with the appropriate UCS PCI Gen5 mezzanine cards .

Cisco identifies the UCSX-V5-PCIME rear mezzanine card as a required hardware component for X580p connectivity. The card provides PCIe Gen5 electrical lanes between each CPU and the X9516 X-Fabric modules. Cisco also specifies that companion compute-node host power must be off for insertion or removal operations involving X580p PCIe connectivity, and that X9516 modules must be installed before the X580p PCIe nodes are installed.

Therefore, powering down the affected blades and installing their required PCI mezzanine cards are the appropriate preparatory actions.

The IFMs provide normal UCS network connectivity and do not need to be powered off or disconnected from Fabric Interconnects merely to install the X-Fabric modules. Similarly, disconnecting the Device Connector is unrelated to physical PCIe fabric installation and would unnecessarily interrupt Intersight connectivity.

Study Guide Reference: AI Infrastructure Deployment and Data Management — Cisco UCS X-Series GPU expansion, X-Fabric, PCIe connectivity, and hardware deployment.

===============

The server contains six 3000-W supplies and must preserve N+2 redundancy . A 4,500-W workload requires two actively supplying PSUs because a single 3000-W supply cannot support the current load. Maintaining N+2 redundancy requires two additional PSUs to remain available beyond those required for the workload. The minimum operational complement is therefore four supplies: two supporting the power demand plus two retained for redundancy.

With six PSUs physically installed, the remaining two PSUs can be placed into standby , making option C correct.

The calculation is:

Required active capacity = ceiling(4500 W ÷ 3000 W) = 2 PSUs

Required redundancy = 2 PSUs

Minimum available PSUs = 2 + 2 = 4

Installed PSUs = 6

Eligible for standby = 6 − 4 = 2 PSUs

Power Save Mode is intended to reduce conversion losses when installed PSU capacity exceeds current demand. Cisco uses the same operating principle in UCS power management: additional supplies that are not needed to satisfy both the requested power budget and configured redundancy can be placed into power-saving operation.

Placing three supplies into standby would leave only three available supplies, violating the N+2 requirement for a workload requiring two PSUs.

Study Guide Reference: AI Infrastructure Operations and Troubleshooting — power efficiency, PSU redundancy, Power Save Mode, and UCS AI server power management.

===============