Which of the following Apache Spark benefits helps manage big data processing?

Correct. Apache Spark creates a comprehensive, unified framework to manage big data processing.

How does Apache Spark solve read/write problems encountered by other tools?

Correct. Keeping data in-memory avoids disk I/O, which speeds up processes.

Distributed computing is ________.

Correct. Distributed computing is a group of computers working together with a shared memory.

The three components of Spark architecture are:

Correct. Spark is made up of memory containing data sets, high-level APIs, and a cluster management framework system.

One component of Spark architecture is the executor. Which of the following is true?

Correct. Executors complete tasks on worker nodes and pass results back to the driver.

The three Apache Spark components are data storage, compute interface, and cluster management framework. Which order does the data flow through these components?

Correct. the data from a Hadoop file system flows into the compute interface or API, which then flows into different nodes to perform distributed/parallel tasks.

Which of the following is true of datasets?

Correct. Compile-time type safety means that Spark can detect syntax and semantic errors in production applications before deployment.

Which of the following is NOT a way to create a dataset?

Correct. DataFrames can’t be contained in datasets.

Which of the following features belong to Tungsten?

This answer is correct. Tungsten manages memory explicitly and does not rely on the JVM object model or garbage collection.

Which of the following is true of Tungsten?

Correct. This reduces multiple CPU calls.

Which tool mentioned in the course creates production-ready environments for AI and machine learning?

Correct. IBM Watson provides services, support, and holistic workflows.

AIOps tools use ________ to facilitate IT operations.

Correct. Machine learning helps identify events and patterns as well as helping to report and fix root issues quickly.

Spark Shell simplifies working with data by _________.

Correct. This enables you to start working with data immediately.

Spark dependencies require driver and cluster executor processes to be able to access the application project. Java and Scala applications provide this access with what?

Correct. this is a single JAR file containing all dependencies, so the application is portable through the cluster.

What is the name of the Spark unified interface?

Correct. The spark-submit script is found in the ‘bin/’ directory.

By default, how much memory does Spark use?

Correct. The amount of memory used by a worker can be specified, however.

Why does Spark queue tasks and wait for available cores?

Correct. Once a task is in queue, Spark assigns it to any available executor.

Why does Spark queue tasks and wait for available cores?

Correct. Once a task is in queue, Spark assigns it to any available executor.

Identify common areas where Spark application issues can happen.

Yes! User code, Configuration, Application Dependencies, Resource allocation, and Network Communication are common areas where Spark application issues can happen.

If a task fails due to an error, Spark _____________.

Correct. The cause of an application failure can usually be found in the driver event log.

If a task fails due to a dependency problem, what is the best way to identify the issue?

Correct. These identify which libraries the application loaded.

Which of the following was mentioned in the course as a common application of Big Data?

Correct. Feedback about why this answer is correct

Select the answers that describe the relationship between Big Data and today’s personal assistants including Google, Alexa Siri and others.

Yes! Personal assistants use unstructured data sources including personal data in the form of photos, videos, and texts that people send to each other as the bulk of data collected by consumer goods companies.

Which of the following is NOT a major component of the Internet of Things (IoT) ecosystem?

Correct. A large percentage of IoT devices collect information without user interaction.

Which of the following is true of Big Data?

Correct. Big Data arrives at a massive volume and with little or no structure.

Select the answer that identifies the main components that describe the dimensions of Big Data.

Yes! Four main components describe the dimensions of Big Data.

Which of the following is true of parallel processing?

Correct. Parallel processing works well for Big Data because of its speed and flexibility, among other reasons.

Semi-structured data _________.

Correct. Semi-structured data combines unstructured and structured data.

What has contributed significantly to the launch of the Big Data era?

Correct. Cloud computing offers the flexibility, cost savings, and high capacity that makes Big Data workable.

Which of these is a good situation for using Hadoop?

Correct. Hadoop is a good solution for Big Data work.

Which of the following Hadoop core components prepares the RAM and CPU for Hadoop to run data in batch, stream, interactive, and graph processing?

Correct. Yarn is short for "yet another resource negotiator" and it's one of the most important components because it prepares the RAM and CPU for Hadoop to run data in these types of processing.

Correct. Hadoop has many components that work together.

A driver program is a process that _______________.

Correct. The driver process can be run on a cluster node or another machine.

Increasing Executors and cores _________.

Correct. Tasks run in separate threads until all cores are used.

What happens when Spark performs a shuffle? Select all that apply.

Correct. when Spark performs a shuffle, it redistributes the dataset across the cluster.

What is the Spark property configuration that follows a precedence order, with the highest being configuration set programmatically, then spark-submit configuration and lastly configuration set in the spark-defaults.conf file?

Correct. this task configuration could change so dynamic configuration handles it well.

Which configuration method enables you to adjust settings on a per-machine basis?

Correct. Feedback about why this answer is correct

What are the required additional considerations when deploying Spark applications on top Kubernetes using client mode? Select all that apply.

Correct. the executors must be able to communicate and connect with the driver program. <br> <br> Correct. Use the driver’s pod name to set ‘spark.kubernetes.driver.pod.name’. This will ensure the executor pods are cleaned up along with the driver pod when the application finishes.