CCDAK Confluent Certified Developer for Apache Kafka Certification Examination Questions and Answers

According to the Apache Kafka Consumer API documentation, the assign() method allows a consumer to manually assign specific partitions to itself, bypassing Kafka’s consumer group coordination and rebalancing mechanism. When a consumer uses assign() with all partitions of a topic, that consumer will read from every partition, regardless of how many other consumers exist.

This behavior is required when every consumer instance must independently read the full dataset, such as in auditing, monitoring, or replication use cases. In contrast, subscribe() is designed for consumer group usage, where partitions are distributed among consumers, ensuring that each partition is consumed by only one consumer in the group.

Option B is invalid because assign() requires a collection of TopicPartition objects, not just a topic name. Option C still uses group coordination and does not guarantee all partitions per consumer. Option D is invalid because an empty group ID is not supported by Kafka.

Thus, explicitly calling assign() with all topic partitions is the only correct and documented way to ensure each consumer reads from all partitions.

The Apache Kafka documentation explains that Kafka’s parallelism model is based on topic partitions and consumer groups. Within a single consumer group, each partition can be assigned to at most one consumer at a time. As a result, the maximum level of parallelism for a consumer group is equal to the number of partitions in the topic.

In this scenario, the topic has five partitions, which means that up to five consumers in the same consumer group can actively consume data in parallel. If more than five consumers are added, the extra consumers will remain idle because there are no additional partitions to assign.

Option B is incorrect because Kafka does not enforce equal message consumption across partitions; message distribution depends on producers and partitioning strategy. Option C is false because partitions may be hosted on different brokers, and consumers fetch data from multiple brokers. Option D is incorrect because a single partition cannot be consumed by multiple consumers within the same group simultaneously.

Therefore, the only valid assumption is that a maximum of five consumers can actively consume messages in the same consumer group.

=========================

For real-time applications, the key metric to monitor is consumer lag — the difference between the latest offset on the broker and the last committed offset by the consumer.

From Kafka Monitoring Guide:

“Consumer lag is the most important metric for real-time applications. It tells you how far behind the consumer is from the latest data in Kafka.”

B relates to replication, not client responsiveness.

C tracks group membership stability, not data delay.

D tracks input throughput, not processing latency.

If a key is not specified, the default Kafka partitioner uses round-robin logic to assign partitions. This leads to even distribution of messages across all partitions.

From Kafka Producer Design:

“If the key is null, the producer will assign the partition in a round-robin fashion to achieve load balancing.”

There is no error, and partition 0 is not always chosen in this case.

The Kafka Streams DSL provides three primary abstractions:

KStream: Represents an unbounded stream of records (append-only).

KTable: Represents a changelog stream where each key has the latest value (like a materialized view).

GlobalKTable: A fully replicated version of KTable across all stream instances, enabling global joins.

From Kafka Streams Developer Guide:

“Kafka Streams provides three core abstractions: KStream, KTable, and GlobalKTable, each serving different purposes in stream processing and joins.”

GlobalKStream and StreamTable do not exist in Kafka’s API.

In Kafka Streams, the number of stream tasks = number of input partitions × num.stream.threads × number of instances, but only as many as the number of partitions can actively consume at once.

However, in this case, Kafka Streams assigns one task per partition, and because there are 5 partitions and 15 threads (3 instances × 5 threads), 15 tasks are created, and all 15 can be active depending on processing topology.

From Kafka Streams Developer Guide:

"Kafka Streams creates one task per input partition. If you increase the number of stream threads, it runs multiple tasks in parallel within a single instance."

So, 15 stream tasks will be created and 15 will be actively consuming.

fetch-rate: Measures how frequently consumers fetch data — critical for monitoring throughput.

incoming-byte-rate: Shows how much data is being written to the broker per second — a key producer-side metric.

From Kafka JMX Metrics Guide:

“fetch-rate, incoming-byte-rate, request-latency are key performance metrics for producers and consumers.”

B, C, E, and F are application-layer business metrics, not Kafka performance metrics.

Using assign() gives manual control over partition assignment. In this mode:

B is true: The application chooses partitions directly without involving the group coordinator.

C is true: Since no group coordination occurs, no rebalancing happens if a consumer dies.

From Kafka Consumer API Docs:

“assign() manually assigns a list of partitions to this consumer. No coordination takes place and no rebalancing is triggered.”

A is false: subscribe() and assign() are mutually exclusive.

D is incorrect: Partitions don't need to be equal across topics.

The offset in the RecordMetadata object returned by the producer represents the position of the record in the partition — i.e., the sequential ID assigned by Kafka once the message is committed.

From Kafka Producer API Documentation:

“The offset is the position of the record in the partition. This is a unique, sequential number assigned by the broker.”

D refers to metadata.partition(), not offset().

B and C are unrelated to how Kafka handles committed offsets.

This scenario describes vertical scaling, as defined in the Apache Kafka Streams documentation and general distributed systems terminology. Vertical scaling refers to adding more resources (CPU, memory, or disk) to a single machine to increase processing capacity.

In this example, the Kafka Streams application is moved from a server with four CPU threads to one with ten CPU threads, increasing the available compute resources without increasing the number of application instances. Kafka Streams can take advantage of additional CPU threads through its num.stream.threads configuration, allowing more partitions to be processed in parallel within the same application instance.

Horizontal scaling (also known as scaling out) would involve adding more application instances across multiple servers, not upgrading a single server. Option D is therefore incorrect, and Option C is not a recognized scaling term.

Thus, this example clearly illustrates vertical scaling, where performance is improved by increasing the capacity of a single node.

The StickyAssignor tries to minimize partition movement by preserving existing assignments as much as possible while still achieving a balanced assignment. This improves consumer stability and reduces rebalances.

From the Kafka Consumer Assignor Documentation:

“The StickyAssignor attempts to preserve as many existing assignments as possible, which helps minimize partition movement between rebalances.”

RoundRobinAssignor focuses on even distribution, not stability.

RangeAssignor groups partitions by topic and assigns them consecutively, but can lead to imbalances.

PartitionAssignor is an abstract base class, not an assignor used directly.

For SASL/PLAIN authentication over SSL, the correct combination is:

security.protocol=SASL_SSL

sasl.jaas.config=...PlainLoginModule required username=... password=...

From Kafka Security Documentation:

“For SASL/PLAIN over SSL, configure security.protocol=SASL_SSL and provide login module configuration using sasl.jaas.config.”

Options B and D are incorrect because they do not pair the correct protocol with the authentication method.

1st: Place the connector's JAR file in the directory specified by plugin.path

2nd: Restart the Kafka Connect cluster

3rd: Verify using REST API (/connector-plugins)

4th: Configure the connector

5th: (Repeat of 4th, duplicate step)

1st → Place the connector’s JAR file in the directory specified by the plugin.path configuration.

2nd → Restart the Kafka Connect cluster.

3rd → Verify that the connector is installed by listing all available connectors using the Kafka Connect REST API (/connector-plugins).

4th → Configure the connector using a JSON or properties file with the necessary settings.

5th → Configure the connector using a JSON or properties file with the necessary settings.

Kafka Connect requires that custom connectors be placed in the directory defined by plugin.path. After restarting the cluster, you can use the REST API to confirm availability and then deploy the connector configuration.

From Kafka Connect Documentation:

“After placing the JAR in the plugin.path, you must restart the Connect cluster to pick it up. Use the /connector-plugins REST endpoint to verify.”

The duplication of configuration is an error in the question options and should occur only once.

According to the official Apache Kafka Connect REST API documentation, restarting a connector can be done at different granularities: connector-level, task-level, or both. By default, calling POST /connectors/{name}/restart only restarts the connector instance, not its tasks.

To restart the connector and all of its tasks, the request must explicitly include the query parameter includeTasks=true. This instructs Kafka Connect to restart the connector and all associated tasks, regardless of whether they are currently running or failed.

Option B (onlyFailed=true) limits the restart to only failed tasks, which does not meet the requirement to restart all tasks. Option C restarts only the connector instance, leaving tasks untouched. Option D restarts a single specific task and does not restart the connector itself.

Therefore, the correct and officially documented REST request to restart the connector instance and all tasks is:

POST /connectors/s3-connector/restart?includeTasks=true

This ensures a full restart of the connector and its execution context across the Kafka Connect cluster.

Kafka message headers are the right place to include metadata such as:

Payload format (e.g., schema version)

Timestamps (custom)

Trace or correlation IDs

From Kafka Producer API Docs:

“Headers are a map of key-value pairs that can be used to include metadata alongside the message, without modifying the message body.”

Keys determine partitioning and ordering

Values are the message payload

Brokers do not store arbitrary metadata fields

This syntax is a clear match to Protocol Buffers (Protobuf). It defines a schema with fields, types, and tags, which is a format supported by Kafka when using Protobuf-based schema registry serialization.

From Confluent Schema Registry Docs:

“Kafka supports Protobuf serialization, where schemas are written in .proto files and include fields with tags.”

int32, double, and string are standard Protobuf types.

Avro uses JSON-style schema.

JSON Schema uses JSON object structure, not .proto.

YAML is unrelated.

The bootstrap.servers setting is used by clients (producers and consumers) to initially connect to the Kafka cluster. It does not need to list all brokers—just a few that are up and reachable. The client will then use the cluster metadata to discover the full broker list.

From Kafka Java Client Configuration Docs:

“This is a list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping.”

Hence, listing multiple brokers provides redundancy and fault tolerance in the event one is down during client startup.

According to the Apache Kafka consumer and scalability documentation, parallelism in message processing is primarily determined by the number of partitions and the number of consumers within a consumer group. Each partition can be assigned to only one consumer at a time. In this scenario, the topic already has four partitions, but only two consumers are processing them, meaning only two partitions are actively consumed in parallel.

Since monitoring indicates that message processing is CPU-bound, the bottleneck is not Kafka I/O but application-side processing. Adding more consumers (up to the number of partitions) allows Kafka to rebalance the partitions and distribute them across more consumer instances, increasing CPU parallelism and reducing lag.

Option A (adding more partitions) can increase parallelism but is more intrusive and may require repartitioning considerations for keys and downstream systems. Option B increases batch size, which can worsen CPU pressure. Option D may reduce per-poll workload but does not increase parallel processing capacity.

Therefore, the correct and recommended action per Kafka documentation is to add more consumers to the consumer group.

=========================

A 409 Conflict response in Kafka Connect REST API usually means:

The connector name provided already exists (i.e., a connector with that name is already deployed).

The Kafka Connect cluster is temporarily unavailable to accept new tasks due to a rebalance in progress.

From the Confluent Developer documentation and Kafka Connect REST API spec:

“409 – Conflict. The connector name is already in use or the worker group is in the process of rebalancing and cannot complete the request at this time.”

Option A is incorrect because invalid configurations return 400 Bad Request, not 409.

Option B is invalid because Kafka Connect does not have a strict concept of "capacity" that prevents connector creation with a 409.

The official Apache Kafka producer documentation states that setting enable.idempotence=true guarantees that messages are written to a partition exactly once, even in the presence of retries caused by network failures or broker timeouts. This feature prevents duplicate records by assigning producer sequence numbers and validating them on the broker side.

For idempotent producers to work correctly, retries must be enabled, which is why Kafka recommends a very large value such as retries=2147483647. Additionally, limiting max.in.flight.requests.per.connection to 1 ensures strict ordering during retries, preventing message reordering in older Kafka versions and providing the safest configuration.

Option A is irrelevant to producers. Option B explicitly disables idempotence, which causes duplicates. Option D disables retries, which increases the risk of message loss.

Therefore, the correct and fully documented solution to eliminate duplicate messages is enabling idempotence with retries and a safe in-flight request limit, as shown in Option C.

Frequent rebalances in a consumer group occur when:

Consumers are too slow and miss heartbeats (A), or

Instances crash or restart, triggering group membership changes (D)

From Kafka Consumer Group Coordination Docs:

“If a consumer fails to send a heartbeat in time (due to slowness or crash), it is removed from the group, causing a rebalance.”

Option B (partition mismatch) affects load balancing, not rebalance frequency.

Option C (broker storage) doesn't trigger consumer rebalances.

C is true: When a new broker is added, no partitions are assigned to it unless you create new topics or reassign existing ones using kafka-reassign-partitions.sh.

D is true: Kafka brokers are hot-pluggable; no need to stop the cluster when scaling.

From Kafka Operations Guide:

“A newly added broker won’t be assigned partitions until reassignments or new topic creation.”

“Kafka allows dynamic scaling by adding brokers without downtime.”

A is false: Broker IDs must be manually set unless using dynamic broker registration in KRaft mode.

B is false unless the cluster uses KRaft mode and the broker is specifically assigned a controller role.

Session windows are ideal for tracking periods of activity separated by inactivity, such as user sessions.

From Kafka Streams Documentation > Windowing:

“A session window captures streams of events that are intermittent and separated by a gap of inactivity.”

Tumbling/Hopping/Sliding windows are fixed in size

Session windows are dynamic and close after inactivity timeout

This makes them perfect for identifying gaps in user interaction.

According to the official Apache Kafka consumer semantics documentation, at-most-once delivery means that records may be lost but must never be processed more than once. To achieve this guarantee, a consumer must commit offsets before processing records. If the application crashes after committing but before processing, messages are skipped rather than reprocessed, which satisfies at-most-once semantics.

Option B correctly implements this pattern. It disables automatic offset commits (enable.auto.commit=false) and explicitly calls commitAsync() immediately after polling and before processing records. This ensures that offsets are advanced regardless of whether processing succeeds, eliminating the possibility of duplicate processing.

Option C represents at-least-once semantics, because offsets are committed after processing, which can lead to duplicates on failure. Options A and D rely on automatic commits or incorrect ordering and do not provide strict at-most-once guarantees.

The Kafka documentation explicitly states that manual offset management with commits before processing is required for at-most-once delivery.

Therefore, Option B is the best and officially documented configuration and code skeleton to avoid duplicate message processing.