A data engineer, while designing a Pandas UDF to process financial time-series data with complex calculations that require maintaining state across rows within each stock symbol group, must ensure the function is efficient and scalable. Which approach will solve the problem with minimum overhead while preserving data integrity?

• A.Use a SCALAR_ITER Pandas UDF with iterator-based processing, implementing state management through persistent storage (Delta tables) that gets updated after each batch to maintain continuity across iterator chunks.
• B.Use a SCALAR Pandas UDF that processes the entire dataset at once, implementing custom partitioning logic within the UDF to group by stock symbol and maintain state using global variables shared across all executor processes.
• C.Use applyInPandas() on a Spark DataFrame that receives all rows for each stock symbol as a Pandas DataFrame, allowing processing within each group while maintaining state variables local to each group's processing function.
• D.Use a grouped_agg Pandas UDF that processes each stock symbol group independently, maintaining state through intermediate aggregation results that get passed between successive UDF calls via broadcast variables.

Comment: *

Name: *

Email: *

Verification: *

A data engineer is configuring a pipeline that will potentially see late-arriving, duplicate records.
In addition to de-duplicating records within the batch, which of the following approaches allows Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from the data engineer to deduplicate data against previously processed records as it is inserted into a Delta table?

• A.Set the configuration delta.deduplicate = true.
• B.VACUUM the Delta table after each batch completes.
• C.Perform an insert-only merge with a matching condition on a unique key.
• D.Perform a full outer join on a unique key and overwrite existing data.
• E.Rely on Delta Lake schema enforcement to prevent duplicate records.

Comment: *

Name: *

Email: *

Verification: *

A data engineer wants to ingest a large collection of image files (JPEG and PNG) from cloud object storage into a Unity Catalog-managed table for analysis and visualization. Which two configurations and practices are recommended to incrementally ingest these images into the table? (Choose two.)

• A.Move files to a volume and read with SQL editor.
• B.Use Auto Loader and set cloudFiles.format to "BINARYFILE".
• C.Use Auto Loader and set cloudFiles.format to "TEXT".
• D.Use Auto Loader and set cloudFiles.format to "IMAGE".
• E.Use the pathGlobFilter option to select only image files (e.g., "*.jpg,*.png").

Comment: *

Name: *

Email: *

Verification: *

A data engineering team is setting up deployment automation. To deploy workspace assets remotely using the Databricks CLI command, they must configure it with proper authentication.
Which authentication approach will provide the highest level of security?

• A.Use a service principal with OAuth token federation.
• B.Use a service principal ID and its OAuth client secret.
• C.Use a service principal and its Personal Access Token.
• D.Use a shared user account and its OAuth client secret.

Comment: *

Name: *

Email: *

Verification: *

A table named user_ltv is being used to create a view that will be used by data analysts on various teams. Users in the workspace are configured into groups, which are used for setting up data access using ACLs.
The user_ltv table has the following schema:
email STRING, age INT, ltv INT
The following view definition is executed:

An analyst who is not a member of the auditing group executes the following query:
SELECT * FROM user_ltv_no_minors
Which statement describes the results returned by this query?

• A.All columns will be displayed normally for those records that have an age greater than 18; records not meeting this condition will be omitted.
• B.All columns will be displayed normally for those records that have an age greater than 17; records not meeting this condition will be omitted.
• C.All age values less than 18 will be returned as null values all other columns will be returned with the values in user_ltv.
• D.All records from all columns will be displayed with the values in user_ltv.
• E.All values for the age column will be returned as null values, all other columns will be returned with the values in user_ltv.

Comment: *

Name: *

Email: *

Verification: *