You have an Azure Storage account that generates 200,000 new files daily. The file names have a format of
{YYYY}/{MM}/{DD}/{HH}/{CustomerID}.csv.
You need to design an Azure Data Factory solution that will load new data from the storage account to an Azure Data Lake once hourly. The solution must minimize load times and costs.
How should you configure the solution? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point

Correct Answer:

Explanation

Box 1: Incremental load
When you start to build the end to end data integration flow the first challenge is to extract data from different data stores, where incrementally (or delta) loading data after an initial full load is widely used at this stage.
Now, ADF provides a new capability for you to incrementally copy new or changed files only by LastModifiedDate from a file-based store. By using this new feature, you do not need to partition the data by time-based folder or file name. The new or changed file will be automatically selected by its metadata LastModifiedDate and copied to the destination store.
Box 2: Tumbling window
Tumbling window triggers are a type of trigger that fires at a periodic time interval from a specified start time, while retaining state. Tumbling windows are a series of fixed-sized, non-overlapping, and contiguous time intervals. A tumbling window trigger has a one-to-one relationship with a pipeline and can only reference a singular pipeline.
Reference:
https://azure.microsoft.com/en-us/blog/incrementally-copy-new-files-by-lastmodifieddate-with-azure-data-factor
https://docs.microsoft.com/en-us/azure/data-factory/how-to-create-tumbling-window-trigger

Comment: *

Name: *

Email: *

Verification: *

You are designing a solution for the ad hoc analysis of data in Azure Databricks notebooks. The data will be stored in Azure Blob storage.
You need to ensure that Blob storage will support the recovery of the data if the data is overwritten accidentally.
What should you recommend?

• A.Enable soft delete.
• B.Add a resource lock.
• C.Enable diagnostics logging.
• D.Use read-access geo-redundant storage (RA-GRS).

Comment: *

Name: *

Email: *

Verification: *

You are planning a design pattern based on the Lambda architecture as shown in the exhibit.

Which Azure service should you use for the hot path?

• A.Azure Synapse Analytics
• B.Azure SQL Database
• C.Azure Cosmos DB
• D.Azure Data Catalog

Correct Answer: C

Explanation
In Azure, all of the following data stores will meet the core requirements supporting real-time processing:
* Apache Spark in Azure Databricks
* Azure Stream Analytics
* HDInsight with Spark Streaming
* HDInsight with Storm
* Azure Functions
* Azure App Service WebJobs
Note: Lambda architectures use batch-processing, stream-processing, and a serving layer to minimize the latency involved in querying big data.

Reference:
https://azure.microsoft.com/en-us/blog/lambda-architecture-using-azure-cosmosdb-faster-performance-low-tco-l
https://docs.microsoft.com/en-us/azure/architecture/data-guide/technology-choices/stream-processing
https://docs.microsoft.com/en-us/azure/cosmos-db/lambda-architecture

Comment: *

Name: *

Email: *

Verification: *

You need to design the image processing solution to meet the optimization requirements for image tag data.
What should you configure? To answer, drag the appropriate setting to the correct drop targets.
Each source may be used once, more than once, or not at all. You may need to drag the split bar between panes or scroll to view content.
NOTE: Each correct selection is worth one point.

Correct Answer:

Comment: *

Name: *

Email: *

Verification: *

Which Azure service and feature should you recommend using to manage the transient data for Data Lake Storage? To answer, select the appropriate options in the answer area.
NOTE: Each correct selection is worth one point.

Correct Answer:

Explanation

Scenario: Stage inventory data in Azure Data Lake Storage Gen2 before loading the data into the analytical data store. Litware wants to remove transient data from Data Lake Storage once the data is no longer in use.
Files that have a modified date that is older than 14 days must be removed.
Service: Azure Data Factory
Clean up files by built-in delete activity in Azure Data Factory (ADF).
ADF built-in delete activity, which can be part of your ETL workflow to deletes undesired files without writing code. You can use ADF to delete folder or files from Azure Blob Storage, Azure Data Lake Storage Gen1, Azure Data Lake Storage Gen2, File System, FTP Server, sFTP Server, and Amazon S3.
You can delete expired files only rather than deleting all the files in one folder. For example, you may want to only delete the files which were last modified more than 13 days ago.
Feature: Delete Activity
Reference:
https://azure.microsoft.com/sv-se/blog/clean-up-files-by-built-in-delete-activity-in-azure-data-factory/

Comment: *

Name: *

Email: *

Verification: *