How can you leverage Spark Streaming for real-time data processing and analytics?

• A.By defining a streaming DataFrame with a window function.
• B.By utilizing Structured Streaming with Kafka as the source and sink.
• C.By implementing custom logic for data ingestion, transformation, and output.
• D.Both A and B.

Comment: *

Name: *

Email: *

Verification: *

Your project involves integrating Spark with a NoSQL database, MongoDB. You need to write a DataFrame 'df into a MongoDB collection named 'orders'. Which PySpark code snippet correctly achieves this?

• A.
• B.
• C.
• D.

Comment: *

Name: *

Email: *

Verification: *

You're working with a Spark application that processes sensitive dat
a. How can you ensure that persisted data remains secure even if accessed from unauthorized sources?

• A.No additional security measures are needed, as Spark handles data security
• B.Encrypt the data before persisting it and decrypt it when needed
• C.Rely on Spark's lineage tracking to prevent unauthorized access
• D.Implement custom access control mechanisms within your application

Comment: *

Name: *

Email: *

Verification: *

Your Spark application encounters performance issues when reading data from a large Hive table. What potential optimization techniques can you explore?

• A.Increase the number of Spark executors without further optimization
• B.Use a different file format for the Hive table, like CSV, for faster parsing
• C.Leverage partition pruning to only read relevant data from the table
• D.Implement custom data compression logic within Spark for improved storage efficiency

Comment: *

Name: *

Email: *

Verification: *

Which feature in Apache Airflow allows you to retry a data quality check task if it fails initially due to transient issues?

• A.catchup
• B.retries parameter in the task definition
• C.SLAs
• D.Branching

Comment: *

Name: *

Email: *

Verification: *