You're experimenting with Iceberg table formats (vl and v2). Which of the following statements is true regarding their differences?

• A.V2 supports new data types like UUIDs, which are unavailable in V1.
• B.V2 tables are generally less performant than V1 tables due to added metadata overhead.
• C.V2 uses manifest lists instead of manifest files for tracking data files.
• D.V2 introduces mandatory partitioning, while V1 allows for unpartitioned tables.

Comment: *

Name: *

Email: *

Verification: *

You are deploying a Spark application on Kubernetes and need to specify the amount of memory allocated to each Executor. In your PySpark code, which configuration setting will you use?

• A.'spark.executor.memoryoverhead'
• B.'spark.executor.instances'
• C.'spark.executor.memory'
• D.'spark.driver.memory'

Comment: *

Name: *

Email: *

Verification: *

What role do user-defined functions (UDFs) play in schema inference within SQL-based data processing engines?

• A.UDFs restrict the ability of the engine to infer schemas automatically.
• B.They primarily increase the storage requirements for schema metadata.
• C.UDFs can enhance schema inference by providing custom logic for data interpretation.
• D.They eliminate the need for schema inference by pre-defining data schemas.

Comment: *

Name: *

Email: *

Verification: *

What is the impact of caching intermediate data in Spark on iterative algorithms' performance?

• A.It significantly increases the execution time of each iteration.
• B.It improves performance by reducing the need to recompute data in each iteration.
• C.It has no impact on performance but increases storage requirements.
• D.It decreases fault tolerance by storing data in volatile storage.

Comment: *

Name: *

Email: *

Verification: *

You're integrating data quality checks into a complex ETL pipeline with numerous tasks and dependencies. How can you ensure the checks are executed in the correct order and don't interfere with other pipeline tasks?

• A.Schedule the data quality checks as a separate DAG and trigger it after the ETL pipeline completes.
• B.Utilize Airflow upstream/downstream dependencies to define the execution order between check tasks and other pipeline tasks.
• C.Implement a custom script to manage the execution of the data quality checks independently.
• D.Run all tasks (ETL and checks) concurrently, assuming they are independent.

Comment: *

Name: *

Email: *

Verification: *