Databricks-Certified-Data-Analyst-Associate Databricks Certified Data Analyst Associate Exam Questions and Answers

Option D is correct because the table has zero real null values, but the result contains null values representing subtotal and grand-total rows. That behavior is produced by WITH CUBE, which creates aggregations for combinations of grouping columns, including (group_1, group_2), (group_1), (group_2), and the grand total (). The Databricks SQL documentation states that GROUP BY supports advanced aggregations through CUBE, and that CUBE is shorthand for grouping sets. Option A only returns detailed groups. Options B and C use invalid syntax in Databricks SQL. Reference: Databricks GROUP BY clause documentation.

Databricks SQL is not meant to replace or substitute other BI tools, but rather to complement them by providing a fast and easy way to query, explore, and visualize data on the lakehouse using the built-in SQL editor, visualizations, and dashboards. Databricks SQL also integrates seamlessly with popular BI tools like Tableau, Power BI, and Looker, allowing analysts to use their preferred tools to access data through Databricks clusters and SQL warehouses. Databricks SQL offers low-code and no-code experiences, as well as optimized connectors and serverless compute, to enhance the productivity and performance of BI workloads on the lakehouse. References: Databricks SQL, Connecting Applications and BI Tools to Databricks SQL, Databricks integrations overview, Databricks SQL: Delivering a Production SQL Development Experience on the Lakehouse

The gold layer of the medallion architecture contains data that is highly refined and aggregated, and powers analytics, machine learning, and production applications. Data analysts typically use the gold layer to access data that has been transformed into knowledge, rather than just information. The gold layer represents the final stage of data quality and optimization in the lakehouse. References: What is the medallion lakehouse architecture?

The correct answer is C because Delta Lake is the open-source storage layer that provides reliability and structure for data lakes. Delta Lake brings ACID transactions, scalable metadata handling, and batch/streaming support to data lake storage, which are core capabilities behind the lakehouse architecture. MLflow is for machine learning lifecycle management, Apache Spark is a distributed processing engine, and Databricks SQL is a Databricks product for SQL analytics, not the open-source storage project described.

Official documentation extract used: Databricks states that Delta Lake is “open source software” and provides “ACID transactions and scalable metadata handling.”

Based on the images you sent, the two statements are SQL queries for different types of joins between the customers and orders tables. A join is a way of combining the rows from two table references based on some criteria. The join type determines how the rows are matched and what kind of result set is returned. The first statement is a query for a LEFT SEMI JOIN, which returns only the rows from the left table reference (customers) that have a match with the right table reference (orders) on the join condition (customer_id). The second statement is a query for a LEFT ANTI JOIN, which returns only the rows from the left table reference (customers) that have no match with the right table reference (orders) on the join condition (customer_id). Therefore, the result sets for the two statements will differ in the following way:

The first statement will return a subset of the customers table that contains only the customers who have placed at least one order. The number of rows returned will be less than or equal to the number of rows in the customers table, depending on how many customers have orders. The number of columns returned will be the same as the number of columns in the customers table, as the LEFT SEMI JOIN does not include any columns from the orders table.

The second statement will return a subset of the customers table that contains only the customers who have not placed any order. The number of rows returned will be less than or equal to the number of rows in the customers table, depending on how many customers have no orders. The number of columns returned will be the same as the number of columns in the customers table, as the LEFT ANTI JOIN does not include any columns from the orders table.

The other options are not correct because:

A. The first statement will not return all data from the customers table, as it will exclude the customers who have no orders. The second statement will not return all data from the orders table, as it will exclude the orders that have a matching customer. Neither statement will fill in any missing data with NULL, as they do not return any columns from the other table.

C. There is a difference between the result sets for both statements, as explained above. The LEFT SEMI JOIN and the LEFT ANTI JOIN are not equivalent operations and will produce different outputs.

D. Both statements will not fail, as Databricks SQL does support those join types. Databricks SQL supports various join types, including INNER, LEFT OUTER, RIGHT OUTER, FULL OUTER, LEFT SEMI, LEFT ANTI, and CROSS. You can also use NATURAL, USING, or LATERAL keywords to specify different join criteria.

E. The first statement will not return only the customer_id from the orders table, as it will return all columns from the customers table. The second statement is correct, but it is not the only difference between the result sets.

The SQL Editor page is where the analyst can write and execute SQL queries in Databricks SQL. The SQL Editor page has a query pane where the analyst can type or paste SQL statements, and a results pane where the analyst can view the query results in a table or a chart. The analyst can also browse data objects, edit multiple queries, execute a single query or multiple queries, terminate a query, save a query, download a query result, and more from the SQL Editor page. References: Create a query in SQL editor

Data Explorer is a user interface that allows you to discover and manage data, schemas, tables, models, and permissions in Databricks SQL. You can use Data Explorer to view schema details, preview sample data, and see table and model details and properties. Administrators can view and change owners, and admins and data object owners can grant and revoke permissions1. References: Discover and manage data using Data Explorer

The correct answer is C because axis settings are configured in the visualization editor, specifically under the Y-axis settings. The Query Editor is for writing SQL, not configuring chart axes. User Settings do not control individual chart scales. Dashboard editing may contain visualizations, but the specific control is in the visualization editor’s Y Axis configuration.

Official documentation extract used: Databricks chart options state that to configure formatting for the X axis or Y axis, click the corresponding axis tab, and the Y-axis supports scale and range controls.

In Databricks, a view is a saved SQL query definition that references existing tables or other views. Once created, a view remains persisted in the metastore (such as Unity Catalog or Hive Metastore) until it is explicitly dropped.

Key points:

Views do not store data themselves but reference data from underlying tables.

Logging out or being inactive does not delete or alter views.

Unless a user or admin explicitly drops the view or the underlying data/table is deleted, the view continues to function as expected.

Therefore, after logging back in—even days later—a user can still run SELECT * FROM stakeholders.eur_customers, and it will return the same data (provided the underlying table hasn’t changed).

Option D is correct. Databricks SQL alerts run a query on a schedule, evaluate a condition, and notify destinations when the condition is met. Since the requirement is notification through a messaging webhook, the alert should use a webhook alert destination. Official Databricks extract: alerts “run queries on a schedule and notify you when a condition that you define is met.” Databricks notification documentation also states that admins can configure alternate notification destinations using webhooks.

In this scenario, the data engineering team has configured a Structured Streaming pipeline that updates the gold-level tables every 10 minutes. To ensure that the dashboard reflects the most recent data, it is essential to set the dashboard ' s refresh schedule to an interval of 10 minutes or less. This synchronization ensures that stakeholders view the latest information shortly after it becomes available in the gold-level tables. Options B, C, and D do not directly address the requirement of aligning the dashboard refresh frequency with the data update interval.

This statement is incorrect. In Databricks SQL, any user with sufficient permissions on the query or dashboard can configure a refresh schedule—workspace administrator privileges are not required.

Here is the breakdown of the correct information:

A. True – Queries can be scheduled to refresh at intervals ranging from 1 minute to 2 weeks.

B. True – You can configure refresh schedules in the Query Editor.

C. False statement – A query being refreshed does use a SQL Warehouse. However, the option in question says it does not use a warehouse, which would be incorrect in a different context. Since this is a trickier one, we know that scheduled queries do require a SQL Warehouse to run.

D. True – Refresh schedules are different from alerts; alerts are triggered based on specific conditions being met in query results.

E. False (and thus the correct answer to this question) – You do not need to be a workspace admin to set a refresh schedule. You only need the correct permissions on the object.

 Option A uses the SELECT DISTINCT statement to remove duplicate rows from the table_bronze and create a new table table_silver with the deduplicated data. This is the correct way to deduplicate data using Spark SQL12. Option B simply inserts all the rows from table_bronze into table_silver, without removing any duplicates. Option C is not a valid syntax for Spark SQL, as there is no MERGE DEDUPLICATE statement. Option D appends all the rows from table_bronze into table_silver, without removing any duplicates. Option E overwrites the existing data in table_silver with the data from table_bronze, without removing any duplicates. References: Delete Duplicate using SPARK SQL, Spark SQL - How to Remove Duplicate Rows

Option D is correct because the desired result needs one row per customer, so the query must aggregate orders by customer_name. It also needs the output column name number_of_orders, so the aggregate expression must be aliased. Databricks SQL documentation states that count “returns the number” of rows in a group, and the SELECT clause supports a column alias for an expression result. Therefore, count(order_id) AS number_of_orders with GROUP BY customer_name is the correct statement. Option B counts correctly but does not alias the output column as required. Option A does not aggregate. Option C uses invalid SQL syntax because USE customer_name is not a grouping clause. References: Databricks count aggregate function and SELECT clause documentation.

The result set provided shows a combination of grouping by two columns (group_1 and group_2) with subtotals for each level of grouping and a grand total. This pattern is typical of a GROUP BY ... WITH ROLLUP operation in SQL, which provides subtotal rows and a grand total row in the result set.

Considering the query options:

A) Option A: GROUP BY group_1, group_2 INCLUDING NULL - This is not a standard SQL clause and would not result in subtotals and a grand total.

B) Option B: GROUP BY group_1, group_2 WITH ROLLUP - This would create subtotals for each unique group_1, each combination of group_1 and group_2, and a grand total, which matches the result set provided.

C) Option C: GROUP BY group_1, group 2 - This is a simple GROUP BY and would not include subtotals or a grand total.

D) Option D: GROUP BY group_1, group_2, (group_1, group_2) - This syntax is not standard and would likely result in an error or be interpreted as a simple GROUP BY, not providing the subtotals and grand total.

E) Option E: GROUP BY group_1, group_2 WITH CUBE - The WITH CUBE operation produces subtotals for all combinations of the selected columns and a grand total, which is more than what is shown in the result set.

The correct answer is Option B, which uses WITH ROLLUP to generate the subtotals for each level of grouping as well as a grand total. This matches the result set where we have subtotals for each group_1, each combination of group_1 and group_2, and the grand total where both group_1 and group_2 are NULL.

To get the number of orders per customer, you need to join the customers and orders tables on the customer_id, count the order_id, and group the results by customer_name. The correct SQL syntax, as outlined in Databricks SQL documentation, is to use GROUP BY on the selected customer field and use COUNT for aggregation. Only option A does this correctly, while the other options contain syntax errors or incorrect field names.

Option B is correct. PIVOT rotates unique values from rows into separate columns, which is exactly the process of converting long-format data into wide-format data. UNPIVOT does the reverse. SUM is an aggregate function, WHERE filters rows, and TRANSFORM is not the SQL keyword used for this reshaping task. Official Databricks extract: the PIVOT clause “rotat[es] unique values from a column into separate columns.”

Option A is correct. The Data Intelligence Engine is the platform-level intelligence layer that understands the semantics and uniqueness of an organization’s data and enables AI-assisted experiences across Databricks. Unity Catalog provides governance and metadata management, Genie Spaces provide a natural-language interface for curated business data, and Databricks Assistant is a user-facing assistant for code/query help. However, the question asks which component is primarily responsible for enabling these capabilities across the platform; that is the Data Intelligence Engine. Databricks describes the platform as powered by a Data Intelligence Engine that understands the uniqueness and semantics of data and helps optimize performance. References: Databricks Data Intelligence Platform documentation and Data Analyst Associate Exam Guide.

Option E is correct. A major benefit of open source technologies and open data formats is avoiding vendor lock-in. Databricks supports open formats and interfaces so data can be used across tools and systems rather than being locked into a proprietary platform. Cloud integrations, governance, and workload scalability are Databricks benefits, but the specific benefit of embracing open source is avoiding vendor lock-in. Official Databricks extract: “Using open data formats and interfaces helps to avoid” vendor lock-in, and Databricks states that no proprietary data formats are used because Delta Lake and Iceberg are open source.

External tables are tables that are defined in the Databricks metastore using the information stored in a cloud object storage location. External tables do not manage the data, but provide a schema and a table name to query the data. To create an external table, you can use the CREATE EXTERNAL TABLE statement and specify the object storage path to the LOCATION clause. For example, to create an external table named ext_table on a Parquet file stored in S3, you can use the following statement:

SQL

CREATE EXTERNAL TABLE ext_table (

col1 INT,

col2 STRING

)

STORED AS PARQUET

LOCATION ' s3://bucket/path/file.parquet '

AI-generated code. Review and use carefully. More info on FAQ.

 External tables

Option B is correct after correcting the uploaded typo withcolumn to withColumn. A 10% tax is calculated by multiplying salary by 0.1, not by 10. The age column should be removed using .drop( " age " ). Options A and D multiply salary by 10, which calculates 1000%, not 10%. Options A and C also use dropField, which is not the correct DataFrame method for removing a top-level column. Official Databricks PySpark documentation states that withColumn adds or replaces a column and that drop(*cols) returns a new DataFrame without the specified columns.

The correct answer is C because the task requires creating or replacing a managed table with an explicitly defined schema. In Databricks SQL, column definitions are placed directly after the table name inside parentheses. The USING clause is for specifying a data source format, not for defining columns. Option C correctly uses CREATE OR REPLACE TABLE database_name.table_name (...), which recreates the table if it already exists and creates an empty table with the specified schema.

Official documentation extract used: Databricks CREATE TABLE syntax supports [CREATE OR] REPLACE followed by the table name and a table_specification containing column identifiers and column types.

A continuous variable is a type of quantitative variable that can assume an infinite number of values within a given range. This means that between any two possible values, there can be an infinite number of other values. For example, variables such as height, weight, and temperature are continuous because they can be measured to any level of precision, and there are no gaps between possible values. This is in contrast to discrete variables, which can only take on specific, distinct values (e.g., the number of children in a family). Understanding the nature of continuous variables is crucial for data analysts, especially when selecting appropriate statistical methods and visualizations, such as histograms, to accurately represent and analyze the data.

Option B is correct. The function to_date converts a timestamp or date-like expression to a date value, which matches the requirement to extract the date from the timestamp field. unix_timestamp converts to a Unix timestamp value, date_format formats a date or timestamp as a string, and from_unixtime converts Unix time into a timestamp/string representation. Official Databricks documentation states that to_date(expr [, fmt]) returns the expression cast to a date.

Partner Connect is a feature that allows you to easily connect your Databricks workspace to Fivetran and other ingestion partners using an automated workflow. You can select a SQL warehouse or a cluster as the destination for your data replication, and the connection details are sent to Fivetran. You can then choose from over 200 data sources that Fivetran supports and start ingesting data into Delta Lake. References: Connect to Fivetran using Partner Connect, Use Databricks with Fivetran

A Structured Streaming pipeline that processes data in micro-batches and populates gold-level tables every minute requires a high level of compute resources to handle the frequent data ingestion, processing, and writing. This could result in a significant cost for the organization, especially if the data volume and velocity are large. Therefore, the data analyst should share this caution with the project stakeholders before setting up the dashboard and evaluate the trade-offs between the desired refresh rate and the available budget. The other options are not valid cautions because:

B. The gold-level tables are assumed to be appropriately clean for business reporting, as they are the final output of the data engineering pipeline. If the data quality is not satisfactory, the issue should be addressed at the source or silver level, not at the gold level.

C. The streaming data is an appropriate data source for a dashboard, as it can provide near real-time insights and analytics for the business users. Structured Streaming supports various sources and sinks for streaming data, including Delta Lake, which can enable both batch and streaming queries on the same data.

D. The streaming cluster is fault tolerant, as Structured Streaming provides end-to-end exactly-once fault-tolerance guarantees through checkpointing and write-ahead logs. If a query fails, it can be restarted from the last checkpoint and resume processing.

E. The dashboard can be refreshed within one minute or less of new data becoming available in the gold-level tables, as Structured Streaming can trigger micro-batches as fast as possible (every few seconds) and update the results incrementally. However, this may not be necessary or optimal for the business use case, as it could cause frequent changes in the dashboard and consume more resources. References: Streaming on Databricks, Monitoring Structured Streaming queries on Databricks, A look at the new Structured Streaming UI in Apache Spark 3.0, Run your first Structured Streaming workload

The correct answer is C because Databricks SQL supports bracket notation for arrays, and array indexing starts at 0. Therefore, the first element of the products array is accessed with products[0]. Options A and B use dot notation, which is not the correct syntax for array indexing. Option D returns the second element, not the first.

Official documentation extract used: Databricks states that the bracket operator returns an array element by index and that “the first element of an ARRAY is at index 0.”

A Sankey diagram is a type of visualization that shows the flow of data between different nodes or categories. It is often used to represent the movement of users through a website, as it can show the paths they take, the sources they come from, the pages they visit, and the outcomes they achieve. A Sankey diagram consists of links and nodes, where the links represent the volume or weight of the flow, and the nodes represent the stages or steps of the flow. The width of the links is proportional to the amount of flow, and the color of the links can indicate different attributes or segments of the flow. A Sankey diagram can help identify the most common or popular user journeys, the bottlenecks or drop-offs in the flow, and the opportunities for improvement or optimization. References: The answer can be verified from Databricks documentation which provides examples and instructions on how to create Sankey diagrams using Databricks SQL Analytics and Databricks Visualizations. Reference links: Databricks SQL Analytics - Sankey Diagram, Databricks Visualizations - Sankey Diagram

Option A is correct. The model being tuned is a Spark ML model, which is already distributed. SparkTrials is intended to distribute independent single-machine trials across Spark workers. For distributed ML algorithms such as Spark MLlib/Spark ML, Hyperopt should run trials from the driver so each trial can access the full cluster resources. Therefore, SparkTrials() should be changed to Trials(). Official Databricks documentation explains that this setup works for distributed machine learning algorithms including Apache Spark MLlib, and the Databricks notebook guidance states that SparkTrials is incompatible for that distributed-training pattern because each trial must be evaluated on the driver node.

Option D is correct. A temporary view is session-scoped and does not store physical data. It is suitable when the relational object is only needed in the current session or query context and should not be available to others in other sessions. A regular view also avoids copying physical data, but it is persistent and can be shared beyond the current session. Official Databricks extract: a view is a “virtual table that has no physical data,” and temporary views are scoped to the notebook/script or query level and cannot be referenced outside that scope.

The SQL query provided is designed to filter out records from “my_table” where the age is 75 or above and the country is Canada. Since I can’t view the content of the links provided directly, I need to rely on the image attached to this question for context. Based on that, Option E (the image attached) represents a table with columns “age” and “country”, showing records where age is 75 or above and country is Canada. References: The answer can be inferred from understanding SQL queries and their outputs as per Databricks documentation: Databricks SQL

In the Databricks Lakehouse Platform, business analysts primarily utilize Databricks SQL as their main service. Databricks SQL provides an environment tailored for executing SQL queries, creating visualizations, and developing dashboards, which aligns with the typical responsibilities of business analysts who focus on interpreting data to inform business decisions. While data scientists and data engineers also interact with the Databricks platform, their primary tools and services differ; data scientists often engage with machine learning frameworks and notebooks, whereas data engineers focus on data pipelines and ETL processes. Platform architects are involved in designing and overseeing the infrastructure and architecture of the platform. Therefore, among the roles listed, business analysts are the primary users of Databricks SQL.​