You were asked to identify number of times a temperature sensor exceed threshold temperature (100.00) by each device, each row contains 5 readings collected every 5 minutes, fill in the blank with the appropriate functions.
Schema: deviceId INT, deviceTemp ARRAY<double>, dateTimeCollected TIMESTAMP

SELECT deviceId, __ (__ (__(deviceTemp], i -> i > 100.00)))
FROM devices
GROUP BY deviceId

• A.SUM, COUNT, SIZE
• B.SUM, SIZE, SLICE
• C.SUM, SIZE, ARRAY_CONTAINS
• D.SUM, SIZE, ARRAY_FILTER
• E.SUM, SIZE, FILTER

Correct Answer: E

Explanation
FILER function can be used to filter an array based on an expression
SIZE function can be used to get size of an array
SUM is used to calculate to total by device
Diagram Description automatically generated

Comment: *

Name: *

Email: *

Verification: *

Which of the following developer operations in CI/CD flow can be implemented in Databricks Re-pos?

• A.Merge when code is committed
• B.Pull request and review process
• C.Trigger Databricks Repos API to pull the latest version of code into production folder
• D.Resolve merge conflicts
• E.Delete a branch

Correct Answer: C

Explanation
See the below diagram to understand the role Databricks Repos and Git provider plays when building a CI/CD workflow.
All the steps highlighted in yellow can be done Databricks Repo, all the steps highlighted in Gray are done in a git provider like Github or Azure DevOps

Comment: *

Name: *

Email: *

Verification: *

What could be the expected output of query SELECT COUNT (DISTINCT *) FROM user on this table

• A.3
• B.2
  (Correct)
• C.1
• D.0
• E.NULL

Comment: *

Name: *

Email: *

Verification: *

You are working to set up two notebooks to run on a schedule, the second notebook is dependent on the first notebook but both notebooks need different types of compute to run in an optimal fashion, what is the best way to set up these notebooks as jobs?

• A.Use DELTA LIVE PIPELINES instead of notebook tasks
• B.A Job can only use single cluster, setup job for each notebook and use job dependency to link both jobs together
• C.Each task can use different cluster, add these two notebooks as two tasks in a single job with linear dependency and modify the cluster as needed for each of the tasks
• D.Use a single job to setup both notebooks as individual tasks, but use the cluster API to setup the second cluster before the start of second task
• E.Use a very large cluster to run both the tasks in a single job

Comment: *

Name: *

Email: *

Verification: *

Which of the following scenarios is the best fit for AUTO LOADER?

• A.Efficiently process new data incrementally from cloud object storage
• B.Efficiently move data incrementally from one delta table to another delta table
• C.Incrementally process new data from streaming data sources like Kafka into delta lake
• D.Incrementally process new data from relational databases like MySQL
• E.Efficiently copy data from one data lake location to another data lake location

Correct Answer: A

Explanation
The answer is, Efficiently process new data incrementally from cloud object storage, AU-TO LOADER only supports ingesting files stored in a cloud object storage. Auto Loader cannot process streaming data sources like Kafka or Delta streams, use Structured streaming for these data sources.
Diagram Description automatically generated

Auto Loader and Cloud Storage Integration
Auto Loader supports a couple of ways to ingest data incrementally
1.Directory listing - List Directory and maintain the state in RocksDB, supports incremental file listing
2.File notification - Uses a trigger+queue to store the file notification which can be later used to retrieve the file, unlike Directory listing File notification can scale up to millions of files per day.
[OPTIONAL]
Auto Loader vs COPY INTO?
Auto Loader
Auto Loader incrementally and efficiently processes new data files as they arrive in cloud storage without any additional setup. Auto Loader provides a new Structured Streaming source called cloudFiles. Given an input directory path on the cloud file storage, the cloudFiles source automatically processes new files as they arrive, with the option of also processing existing files in that directory.
When to use Auto Loader instead of the COPY INTO?
*You want to load data from a file location that contains files in the order of millions or higher. Auto Loader can discover files more efficiently than the COPY INTO SQL command and can split file processing into multiple batches.
*You do not plan to load subsets of previously uploaded files. With Auto Loader, it can be more difficult to reprocess subsets of files. However, you can use the COPY INTO SQL command to reload subsets of files while an Auto Loader stream is simultaneously running.

Comment: *

Name: *

Email: *

Verification: *