While investigating a data issue, you wanted to review yesterday's version of the table using below command, while querying the previous version of the table using time travel you realized that you are no longer able to view the historical data in the table and you could see it the table was updated yesterday based on the table history(DESCRIBE HISTORY table_name) command what could be the reason why you can not access this data?
SELECT * FROM table_name TIMESTAMP AS OF date_sub(current_date(), 1)

• A.You currently do not have access to view historical data
• B.By default, historical data is cleaned every 180 days in DELTA
• C.A command VACUUM table_name RETAIN 0 was ran on the table
• D.Time travel is disabled
• E.Time travel must be enabled before you query previous data

Comment: *

Name: *

Email: *

Verification: *

You are currently asked to work on building a data pipeline, you have noticed that you are currently working on a very large scale ETL many data dependencies, which of the following tools can be used to address this problem?

• A.AUTO LOADER
• B.JOBS and TASKS
• C.SQL Endpoints
• D.DELTA LIVE TABLES
• E.STRUCTURED STREAMING with MULTI HOP

Correct Answer: D

Explanation
The answer is, DELTA LIVE TABLES
DLT simplifies data dependencies by building DAG-based joins between live tables. Here is a view of how the dag looks with data dependencies without additional meta data,
1.create or replace live view customers
2.select * from customers;
3.
4.create or replace live view sales_orders_raw
5.select * from sales_orders;
6.
7.create or replace live view sales_orders_cleaned
8.as
9.select sales.* from
10.live.sales_orders_raw s
11. join live.customers c
12.on c.customer_id = s.customer_id
13.where c.city = 'LA';
14.
15.create or replace live table sales_orders_in_la
16.selects from sales_orders_cleaned;
Above code creates below dag

Documentation on DELTA LIVE TABLES,
https://databricks.com/product/delta-live-tables
https://databricks.com/blog/2022/04/05/announcing-generally-availability-of-databricks-delta-live-tables-dlt.htm DELTA LIVE TABLES, addresses below challenges when building ETL processes
1.Complexities of large scale ETL
a.Hard to build and maintain dependencies
b.Difficult to switch between batch and stream
2.Data quality and governance
a.Difficult to monitor and enforce data quality
b.Impossible to trace data lineage
3.Difficult pipeline operations
a.Poor observability at granular data level
b.Error handling and recovery is laborious

Comment: *

Name: *

Email: *

Verification: *

A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then
perform a streaming write into a new table. The code block used by the data engineer is below:
1. (spark.table("sales")
2. .withColumn("avg_price", col("sales") / col("units"))
3. .writeStream
4. .option("checkpointLocation", checkpointPath)
5. .outputMode("complete")
6. ._____
7. .table("new_sales")
8.)
If the data engineer only wants the query to execute a single micro-batch to process all of the available data,
which of the following lines of code should the data engineer use to fill in the blank?

• A..trigger(once=True)
• B..trigger(continuous="once")
• C..trigger(processingTime="once")
• D..processingTime("once")
• E..processingTime(1)

Comment: *

Name: *

Email: *

Verification: *

The data engineering team is using a SQL query to review data completeness every day to monitor the ETL job, and query output is being used in multiple dashboards which of the following ap-proaches can be used to set up a schedule and automate this process?

• A.They can schedule the query to run every day from the Jobs UI.
• B.They can schedule the query to refresh every day from the query's page in Databricks SQL
• C.They can schedule the query to run every 12 hours from the Jobs UI.
• D.They can schedule the query to refresh every day from the SQL endpoint's page in Databricks SQL.
• E.They can schedule the query to refresh every 12 hours from the SQL endpoint's page in Databricks SQL

Comment: *

Name: *

Email: *

Verification: *

Below sample input data contains two columns, one cartId also known as session id, and the second column is called items, every time a customer makes a change to the cart this is stored as an array in the table, the Marketing team asked you to create a unique list of item's that were ever added to the cart by each customer, fill in blanks by choosing the appropriate array function so the query produces below expected result as shown below.
Schema: cartId INT, items Array<INT>
Sample Data

1.SELECT cartId, ___ (___(items)) as items
2.FROM carts GROUP BY cartId
Expected result:
cartId items
1 [1,100,200,300,250]

• A.FLATTEN, COLLECT_UNION
• B.ARRAY_UNION, FLATTEN
• C.ARRAY_UNION, ARRAY_DISTINT
• D.ARRAY_UNION, COLLECT_SET
• E.ARRAY_DISTINCT, ARRAY_UNION

Correct Answer: D

Explanation
COLLECT SET is a kind of aggregate function that combines a column value from all rows into a unique list ARRAY_UNION combines and removes any duplicates, Graphical user interface, application Description automatically generated with medium confidence

Comment: *

Name: *

Email: *

Verification: *