What is the main difference between the bronze layer and silver layer in a medallion architecture?

• A.Duplicates are removed in bronze, schema is applied in silver
• B.Silver may contain aggregated data
• C.Bronze is raw copy of ingested data, silver contains data with production schema and optimized for ELT/ETL throughput
• D.Bad data is filtered in Bronze, silver is a copy of bronze data

Correct Answer: C

Explanation
Medallion Architecture - Databricks
Exam focus: Please review the below image and understand the role of each layer(bronze, silver, gold) in medallion architecture, you will see varying questions targeting each layer and its purpose.
Sorry I had to add the watermark some people in Udemy are copying my content.
A diagram of a house Description automatically generated with low confidence

Comment: *

Name: *

Email: *

Verification: *

The data engineer team is configuring environment for development testing, and production before beginning migration on a new data pipeline. The team requires extensive testing on both the code and data resulting from code execution, and the team want to develop and test against similar production data as possible.
A junior data engineer suggests that production data can be mounted to the development testing environments, allowing pre production code to execute against production data. Because all users have Admin privileges in the development environment, the junior data engineer has offered to configure permissions and mount this data for the team.
Which statement captures best practices for this situation?

• A.Because access to production data will always be verified using passthrough credentials it is safe to mount data to any Databricks development environment.
• B.All developer, testing and production code and data should exist in a single unified workspace; creating separate environments for testing and development further reduces risks.
• C.In environments where interactive code will be executed, production data should only be accessible with read permissions; creating isolated databases for each environment further reduces risks.
• D.Because delta Lake versions all data and supports time travel, it is not possible for user error or malicious actors to permanently delete production data, as such it is generally safe to mount production data anywhere.

Comment: *

Name: *

Email: *

Verification: *

A table in the Lakehouse namedcustomer_churn_paramsis used in churn prediction by the machine learning team. The table contains information about customers derived from a number of upstream sources. Currently, the data engineering team populates this table nightly by overwriting the table with the current valid values derived from upstream data sources.
The churn prediction model used by the ML team is fairly stable in production. The team is only interested in making predictions on records that have changed in the past 24 hours.
Which approach would simplify the identification of these changed records?

• A.Apply the churn model to all rows in the customer_churn_params table, but implement logic to perform an upsert into the predictions table that ignores rows where predictions have not changed.
• B.Convert the batch job to a Structured Streaming job using the complete output mode; configure a Structured Streaming job to read from the customer_churn_params table and incrementally predict against the churn model.
• C.Calculate the difference between the previous model predictions and the current customer_churn_params on a key identifying unique customers before making new predictions; only make predictions on those customers not in the previous predictions.
• D.Modify the overwrite logic to include a field populated by calling
  spark.sql.functions.current_timestamp() as data are being written; use this field to identify records written on a particular date.
• E.Replace the current overwrite logic with a merge statement to modify only those records that have changed; write logic to make predictions on the changed records identified by the change data feed.

Comment: *

Name: *

Email: *

Verification: *

A data pipeline uses Structured Streaming to ingest data from kafka to Delta Lake. Data is being stored in a bronze table, and includes the Kafka_generated timesamp, key, and value. Three months after the pipeline is deployed the data engineering team has noticed some latency issued during certain times of the day.
A senior data engineer updates the Delta Table's schema and ingestion logic to include the current timestamp (as recoded by Apache Spark) as well the Kafka topic and partition. The team plans to use the additional metadata fields to diagnose the transient processing delays:
Which limitation will the team face while diagnosing this problem?

• A.New fields not be computed for historic records.
• B.Updating the table schema will invalidate the Delta transaction log metadata.
• C.Updating the table schema requires a default value provided for each file added.
• D.Spark cannot capture the topic partition fields from the kafka source.

Comment: *

Name: *

Email: *

Verification: *

In order to facilitate near real-time workloads, a data engineer is creating a helper function to leverage the schema detection and evolution functionality of Databricks Auto Loader. The desired function will automatically detect the schema of the source directly, incrementally process JSON files as they arrive in a source directory, and automatically evolve the schema of the table when new fields are detected.
The function is displayed below with a blank:

Which response correctly fills in the blank to meet the specified requirements?

• A.Option A
• B.Option B
• C.Option C
• D.Option D
• E.Option E

Comment: *

Name: *

Email: *

Verification: *