A new data engineer notices that a critical field was omitted from an application that writes its Kafka source to Delta Lake. This happened even though the critical field was in the Kafka source.
That field was further missing from data written to dependent, long-term storage. The retention threshold on the Kafka service is seven days. The pipeline has been in production for three months.
Which describes how Delta Lake can help to avoid data loss of this nature in the future?

• A.The Delta log and Structured Streaming checkpoints record the full history of the Kafka producer.
• B.Delta Lake schema evolution can retroactively calculate the correct value for newly added fields, as long as the data was in the original source.
• C.Delta Lake automatically checks that all fields present in the source data are included in the ingestion layer.
• D.Data can never be permanently dropped or deleted from Delta Lake, so data loss is not possible under any circumstance.
• E.Ingestine all raw data and metadata from Kafka to a bronze Delta table creates a permanent, replayable history of the data state.Get Latest & Actual Certified-Data-Engineer-Professional Exam's Question and Answers from

Comment: *

Name: *

Email: *

Verification: *

A data engineer needs to productionize a new Spark application written by teammate. This application has numerous external dependencies, including libraries, and requires custom environment variables and Spark configuration parameters to be set. Which two methods will help the data engineer accomplish the task? (Choose two.)

• A.Install libraries on DBFS
• B.Add libraries to compute policies
• C.Use secrets in init scripts to store configuration data
• D.Use compute policies to set system properties, environment variables, and Spark configuration parameters.
• E.Create init scripts on DBFS.

Comment: *

Name: *

Email: *

Verification: *

The data governance team is reviewing code used for deleting records for compliance with GDPR. They note the following logic is used to delete records from the Delta Lake table named users.

Assuming that user_id is a unique identifying key and that delete_requests contains all users that have requested deletion, which statement describes whether successfully executing the above logic guarantees that the records to be deleted are no longer accessible and why?

• A.Yes; Delta Lake ACID guarantees provide assurance that the delete command succeeded fully and permanently purged these records.
• B.No; the Delta cache may return records from previous versions of the table until the cluster is restarted.
• C.Yes; the Delta cache immediately updates to reflect the latest data files recorded to disk.
• D.No; the Delta Lake delete command only provides ACID guarantees when combined with the merge into command.
• E.No; files containing deleted records may still be accessible with time travel until a vacuum command is used to remove invalidated data files.

Comment: *

Name: *

Email: *

Verification: *

A data engineer wants to refactor the following DLT code, which includes multiple table definitions with very similar code.

In an attempt to programmatically create these tables using a parameterized table definition, the data engineer writes the following code.

The pipeline runs an update with this refactored code, but generates a different DAG showing incorrect configuration values for these tables.
How can the data engineer fix this?

• A.Wrap the for loop inside another table definition, using generalized names and properties to replace with those from the inner table definition.
• B.Convert the list of configuration values to a dictionary of table settings, using table names as keys.
• C.Move the table definition into a separate function, and make calls to this function using different input parameters inside the for loop.
• D.Load the configuration values for these tables from a separate file, located at a path provided by a pipeline parameter.

Comment: *

Name: *

Email: *

Verification: *

A data engineer is performing a join operating to combine values from a static userlookup table with a streaming DataFrame streamingDF.
Which code block attempts to perform an invalid stream-static join?

• A.userLookup.join(streamingDF, ["userid"], how="inner")
• B.streamingDF.join(userLookup, ["user_id"], how="outer")
• C.streamingDF.join(userLookup, ["user_id"], how="left")
• D.streamingDF.join(userLookup, ["userid"], how="inner")
• E.userLookup.join(streamingDF, ["user_id"], how="right")

Comment: *

Name: *

Email: *

Verification: *