AWS Certified Data Engineer - Associate (DEA-C01) Dumps July 2026
PDF + Test Engine
$139 $97
Test Engine
$119 $83
PDF
$99 $69
Here are Amazon Data-Engineer-Associate PDF available features:
302 questions with answers
Updation Date : 16 Jul, 2026
1 day study required to pass exam
100% Passing Assurance
100% Money Back Guarantee
Free 3 Months Updates
Amazon Data-Engineer-Associate Sample Questions
Question # 61
A data engineer needs to create an
AWS Lambda function that converts the format of data from .csv to
Apache Parquet. The Lambda function must run only if a user uploads a
.csv file to an Amazon S3 bucket.Which solution will meet these requirements with the LEAST operational overhead?
A. Create an S3 event notification that has an
event type of s3:ObjectCreated:*. Use a filter rule to generate
notifications only when the suffix includes .csv. Set the Amazon
Resource Name (ARN) of the Lambda function as the destination for the
event notification.
B. Create an S3 event notification that has an event type of s3:ObjectTagging:* for objects that have a tag set to .csv. Set the Amazon Resource Name (ARN) of the Lambda function as the destination for the event notification. C. Create an S3 event notification that has an event type of s3:*. Use a filter rule to generate notifications only when the suffix includes .csv. Set the Amazon Resource Name (ARN) of the Lambda function as the destination for the event notification. D. Create an S3 event notification that has an event type of s3:ObjectCreated:*. Use a filter rule to generate notifications only when the suffix includes .csv. Set an Amazon Simple Notification Service (Amazon SNS) topic as the destination for the event notification. Subscribe the Lambda function to the SNS topic.
Answer: A Explanation:
AWS
Certified Data Engineer - Associate DEA-C01 Complete Study Guide,
Chapter 3: Data Ingestion and Transformation, Section 3.2: S3 Event
Notifications and Lambda Functions, Pages 67-69
Building Batch Data Analytics Solutions on AWS, Module 4: Data Transformation, Lesson 4.2: AWS Lambda, Pages 4-8
AWS
Documentation Overview, AWS Lambda Developer Guide, Working with AWS
Lambda Functions, Configuring Function Triggers, Using AWS Lambda with
Amazon S3, Pages 1-5
Explanation: Option
A is the correct answer because it meets the requirements with the
least operational overhead. Creating an S3 event notification that has
an event type of s3:ObjectCreated:* will trigger the Lambda function
whenever a new object is created in the S3 bucket. Using a filter rule
to generate notifications only when the suffix includes .csv will ensure
that the Lambda function only runs for .csv files. Setting the ARN of
the Lambda function as the destination for the event notification will
directly invoke the Lambda function without any additional steps.Option
B is incorrect because it requires the user to tag the objects with
.csv, which adds an extra step and increases the operational overhead.Option
C is incorrect because it uses an event type of s3:*, which will
trigger the Lambda function for any S3 event, not just object creation.
This could result in unnecessary invocations and increased costs.Option
D is incorrect because it involves creating and subscribing to an SNS
topic, which adds an extra layer of complexity and operational overhead.References:
Question # 62
An airline company is collecting
metrics about flight activities for analytics. The company is conducting
a proof of concept (POC) test to show how analytics can provide
insights that the company can use to increase on-time departures.The
POC test uses objects in Amazon S3 that contain the metrics in .csv
format. The POC test uses Amazon Athena to query the data. The data is
partitioned in the S3 bucket by date.As the amount of data increases, the company wants to optimize the storage solution to improve query performance.Which combination of solutions will meet these requirements? (Choose two.)
A. Add a randomized string to the beginning of the keys in Amazon S3 to get more throughput across partitions.
B. Use an S3 bucket that is in the same account that uses Athena to query the data. C. Use an S3 bucket that is in the same AWS Region where the company runs Athena queries. D. Preprocess the .csvdata to JSON format by fetchingonly the document keys that the query requires. E. Preprocess the .csv data to Apache Parquet format by fetching only the data blocks that are needed for predicates.
Answer: C,E Explanation:
Explanation: Using
an S3 bucket that is in the same AWS Region where the company runs
Athena queries can improve query performance by reducing data transfer
latency and costs. Preprocessing the .csv data to Apache Parquet format
can also improve query performance by enabling columnar storage,
compression, and partitioning, which can reduce the amount of data
scanned and fetched by the query. These solutions can optimize the
storage solution for the POC test without requiring much effort or
changes to the existing data pipeline. The other solutions are not
optimal or relevant for this requirement. Adding a randomized string to
the beginning of the keys in Amazon S3 can improve the throughput across
partitions, but it can also make the data harder to query and manage.
Using an S3 bucket that is in the same account that uses Athena to query
the data does not have any significant impact on query performance, as
long as the proper permissions are granted. Preprocessing the .csv data
to JSON format does not offer any benefits over the .csv format, as both
are row-based and verbose formats that require more data scanning and
fetching than columnar formats like Parquet. References:
Best Practices When Using Athena with AWS Glue
Optimizing Amazon S3 Performance
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide
Question # 63
A data engineer is using Amazon
Athena to analyze sales data that is in Amazon S3. The data engineer
writes a query to retrieve sales amounts for 2023 for several products
from a table named sales_data. However, the query does not return
results for all of the products that are in the sales_data table. The
data engineer needs to troubleshoot the query to resolve the issue.The data engineer's original query is as follows:SELECT product_name, sum(sales_amount)FROM sales_dataWHERE year = 2023GROUP BY product_nameHow should the data engineer modify the Athena query to meet these requirements?
A. Replace sum(sales amount) with count(*J for the aggregation.
B. Change WHERE year = 2023 to WHERE extractlyear FROM sales data) = 2023. C. Add HAVING sumfsales amount) > 0 after the GROUP BY clause. D. Remove the GROUP BY clause
Answer: B Explanation:
Troubleshooting JSON queries - Amazon Athena (Section: JSON related errors)
When I query a table in Amazon Athena, the TIMESTAMP result is empty (Section: Resolution)
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide (Chapter 7, page 197)
Explanation: The
original query does not return results for all of the products because
the year column in the sales_data table is not an integer, but a
timestamp. Therefore, the WHERE clause does not filter the data
correctly, and only returns the products that have a null value for the
year column. To fix this, the data engineer should use the extract
function to extract the year from the timestamp and compare it with
2023. This way, the querywill return the correct results for all of the
products in the sales_data table. The other options are either incorrect
or irrelevant, as they do not address the root cause of the issue.
Replacing sum with count does not change the filtering condition, adding
HAVING clause does not affect the grouping logic, and removing the
GROUP BY clause does not solve the problem of missing products. References:
Question # 64
A manufacturing company collects
sensor data from its factory floor to monitor and enhance operational
efficiency. The company uses Amazon Kinesis Data Streams to publish the
data that the sensors collect to a data stream. Then Amazon Kinesis Data
Firehose writes the data to an Amazon S3 bucket.The company needs to display a real-time view of operational efficiency on a large screen in the manufacturing facility.Which solution will meet these requirements with the LOWEST latency?
A. Use Amazon Managed Service for Apache Flink
(previously known as Amazon Kinesis Data Analytics) to process the
sensor data. Use a connector for Apache Flink to write data to an Amazon
Timestream database. Use the Timestream database as a source to create a
Grafana dashboard.
B. Configure the S3 bucket to send a notification to an AWS Lambda function when any new object is created. Use the Lambda function to publish the data to Amazon Aurora. Use Aurora as a source to create an Amazon QuickSight dashboard. C. Use Amazon Managed Service for Apache Flink (previously known as Amazon Kinesis Data Analytics) to process the sensor data. Create a new Data Firehose delivery stream to publish data directly to an Amazon Timestream database. Use the Timestream database as a source to create an Amazon QuickSight dashboard. D. Use AWS Glue bookmarks to read sensor data from the S3 bucket in real time. Publish the data to an Amazon Timestream database. Use the Timestream database as a source to create a Grafana dashboard.
Answer: C Explanation:
A.
Use Amazon Managed Service for Apache Flink (previously known as Amazon
Kinesis Data Analytics) to process the sensor data. Use a connector for
Apache Flink to write data to an Amazon Timestream database. Use the
Timestream database as a source to create a Grafana dashboard. This
option is similar to option C, but it uses Grafana instead of Amazon
QuickSight to create the dashboard. Grafana is an open source
visualization tool that can also connect to Amazon Timestream, but it
requires additional steps to set up and configure, such as deploying a
Grafana server on Amazon EC2, installing the Amazon Timestream plugin,
and creating an IAM role for Grafana to access Timestream. These steps
can increase the latency and complexity of the solution.
B.
Configure the S3 bucket to send a notification to an AWS Lambda function
when any new object is created. Use the Lambda function to publish the
data to Amazon Aurora. Use Aurora as a source to create an Amazon
QuickSight dashboard. This option is not suitable for displaying a
real-time view of operational efficiency, as it introduces unnecessary
delays and costs in the data pipeline. First, the sensor data is written
to an S3 bucket by Amazon Kinesis Data Firehose, which can have a
buffering interval of up to 900 seconds. Then, the S3 bucket sends a
notification to a Lambda function, which can incur additional invocation
and execution time. Finally, the Lambda function publishes the data to
Amazon Aurora, a relational database that is not optimized for time
series data and can have higher storage and performance costs than
Amazon Timestream .
D. Use AWS Glue bookmarks to read sensor
data from the S3 bucket in real time. Publish the data to an Amazon
Timestream database. Use the Timestream database as a source to create a
Grafana dashboard. This option is also not suitable for displaying a
real-time view of operational efficiency, as it uses AWS Glue bookmarks
to read sensor data from the S3 bucket. AWS Glue bookmarks are a feature
that helps AWS Glue jobs and crawlers keep track of the data that has
already been processed, so that they can resume from where they left
off. However, AWS Glue jobs and crawlers are not designed for real-time
data processing, as they can have a minimum frequency of 5 minutes and a
variable start-up time. Moreover, this option also uses Grafana instead
of Amazon QuickSight to create the dashboard, which can increase the
latency and complexity of the solution .
References:
1: Amazon Managed Streaming for Apache Flink
2: Amazon Timestream
3: Amazon QuickSight
: Analyze data in Amazon Timestream using Grafana
: Amazon Kinesis Data Firehose
: Amazon Aurora
: AWS Glue Bookmarks
: AWS Glue Job and Crawler Scheduling
Explanation: This
solution will meet the requirements with the lowest latency because it
uses Amazon Managed Service for Apache Flink to process the sensor data
in real time and write it to Amazon Timestream, a fast, scalable, and
serverless time series database. Amazon Timestream is optimized for
storing and analyzing time series data, such as sensor data, and can
handle trillions of events per day with millisecond latency. By using
AmazonTimestream as a source, you can create an Amazon QuickSight
dashboard that displays a real-time view of operational efficiency on a
large screen in the manufacturing facility. Amazon QuickSight is a fully
managed business intelligence service that can connect to various data
sources, including Amazon Timestream, and provide interactive
visualizations and insights123.The other options are not optimal for the following reasons:
Question # 65
A data engineer needs to schedule a
workflow that runs a set of AWS Glue jobs every day. The data engineer
does not require the Glue jobs to run or finish at a specific time.Which solution will run the Glue jobs in the MOST cost-effective way?
A. Choose the FLEX execution class in the Glue job properties.
B. Use the Spot Instance type in Glue job properties. C. Choose the STANDARD execution class in the Glue job properties. D. Choose the latest version in the GlueVersion field in the Glue job properties.
Answer: A Explanation:
Introducing AWS Glue Flex jobs: Cost savings on ETL workloads
Serverless Data Integration – AWS Glue Pricing
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide (Chapter 5, page 125)
Explanation: The
FLEX execution class allows you to run AWS Glue jobs on spare compute
capacity instead of dedicated hardware. This can reduce the cost of
running non-urgent or non-time sensitive data integration workloads,
such as testing and one-time data loads. The FLEX execution class is
available for AWS Glue 3.0 Spark jobs. The other options are not as
cost-effective as FLEX, because they either use dedicated resources
(STANDARD) or do not affect the cost at all (Spot Instance type and
GlueVersion). References:
Question # 66
A company is building an analytics
solution. The solution uses Amazon S3 for data lake storage and Amazon
Redshift for a data warehouse. The company wants to use Amazon Redshift
Spectrum to query the data that is in Amazon S3.Which actions will provide the FASTEST queries? (Choose two.)
A. Use gzip compression to compress individual files to sizes that are between 1 GB and 5 GB.
B. Use a columnar storage file format. C. Partition the data based on the most common query predicates. D. Split the data into files that are less than 10 KB. E. Use file formats that are not
Answer: B,C Explanation: Explanation: Amazon
Redshift Spectrum is a feature that allows you to run SQL queries
directly against data in Amazon S3, without loading or transforming the
data. Redshift Spectrum can query various data formats, such as CSV,
JSON, ORC, Avro, and Parquet. However, not all data formats are equally
efficient for querying. Some data formats, such as CSV and JSON, are
row-oriented, meaning that they store data as a sequence of records,
each with the same fields. Row-oriented formats are suitable for loading
and exporting data, but they are not optimal for analytical queries
that often access only a subset ofcolumns. Row-oriented formats also do
not support compression or encoding techniques that can reduce the data
size and improve the query performance.On the other hand, some
data formats, such as ORC and Parquet, are column-oriented, meaning that
they store data as a collection of columns, each with a specific data
type. Column-oriented formats are ideal for analytical queries that
often filter, aggregate, or join data by columns. Column-oriented
formats also support compression and encoding techniques that can reduce
the data size and improve the query performance. For example, Parquet
supports dictionary encoding, which replaces repeated values with
numeric codes, and run-length encoding, which replaces consecutive
identical values with a single value and a count. Parquet also supports
various compression algorithms, such as Snappy, GZIP, and ZSTD, that can
further reduce the data size and improve the query performance.Therefore,
using a columnar storage file format, such as Parquet, will provide
faster queries, as it allows Redshift Spectrum to scan only the relevant
columns and skip the rest, reducing the amount of data read from S3.
Additionally, partitioning the data based on the most common query
predicates, such as date, time, region, etc., will provide faster
queries, as it allows Redshift Spectrum to prune the partitions that do
not match the query criteria, reducing the amount of data scanned from
S3. Partitioning also improves the performance of joins and
aggregations, as it reduces data skew and shuffling.The other
options are not as effective as using a columnar storage file format and
partitioning the data. Using gzip compression to compress individual
files to sizes that are between 1 GB and 5 GB will reduce the data size,
but it will not improve the query performance significantly, as gzip is
not a splittable compression algorithm and requires decompression
before reading. Splitting the data into files that are less than 10 KB
will increase the number of files and the metadata overhead, which will
degrade the query performance. Using file formats that are not supported
by Redshift Spectrum, such as XML, will not work, as Redshift Spectrum
will not be able to read or parse the data. References:
Amazon Redshift Spectrum
Choosing the Right Data Format
AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide, Chapter 4: Data Lakes and Data Warehouses, Section 4.3: Amazon Redshift Spectrum
Question # 67
A data engineer must build an
extract, transform, and load (ETL) pipeline to process and load data
from 10 source systems into 10 tables that are in an Amazon Redshift
database. All the source systems generate .csv, JSON, or Apache Parquet
files every 15 minutes. The source systems all deliver files into one
Amazon S3 bucket. The file sizes range from 10 MB to 20 GB. The ETL
pipeline must function correctly despite changes to the data schema.Which data pipeline solutions will meet these requirements? (Choose two.)
A. Use an Amazon EventBridge rule to run an AWS
Glue job every 15 minutes. Configure the AWS Glue job to process and
load the data into the Amazon Redshift tables.
B. Use an Amazon EventBridge rule to invoke an AWS Glue workflow job every 15 minutes. Configure the AWS Glue workflow to have an on-demand trigger that runs an AWS Glue crawler and then runs an AWS Glue job when the crawler finishes running successfully. Configure the AWS Glue job to process and load the data into the Amazon Redshift tables. C. Configure an AWS Lambda function to invoke an AWS Glue crawler when a file is loaded into the S3 bucket. Configure an AWS Glue job to process and load the data into the Amazon Redshift tables. Create a second Lambda function to run the AWS Glue job. Create an Amazon EventBridge rule to invoke the second Lambda function when the AWS Glue crawler finishes running successfully. D. Configure an AWS Lambda function to invoke an AWS Glue workflow when a file is loaded into the S3 bucket. Configure the AWS Glue workflow to have an on-demand trigger that runs an AWS Glue crawler and then runs an AWS Glue job when the crawler finishes running successfully. Configure the AWS Glue job to process and load the data into the Amazon Redshift tables. E. Configure an AWS Lambda function to invoke an AWS Glue job when a file is loaded into the S3 bucket. Configure the AWS Glue job to read the files from the S3 bucket into an Apache Spark DataFrame. Configure the AWS Glue job to also put smaller partitions of the DataFrame into an Amazon Kinesis Data Firehose delivery stream. Configure the delivery stream to load data into the Amazon Redshift tables.
Answer: A,B Explanation: Explanation: Using
an Amazon EventBridge rule to run an AWS Glue job or invoke an AWS Glue
workflow job every 15 minutes are two possible solutions that will meet
the requirements. AWS Glue is a serverless ETL service that can process
and load data from various sources to various targets, including Amazon
Redshift. AWS Glue can handle different data formats, such as CSV,
JSON, and Parquet, and also support schema evolution, meaning it can
adapt to changes in the data schema over time. AWS Glue can also
leverage Apache Spark to perform distributed processing and
transformation of large datasets. AWS Glue integrates with Amazon
EventBridge, which is a serverless event bus service that can trigger
actions based on rules and schedules. By using an Amazon EventBridge
rule, you can invoke an AWS Glue job or workflow every 15 minutes, and
configure the job or workflow to run an AWS Glue crawler and then load
the data into the Amazon Redshift tables. This way, you can build a
cost-effective and scalable ETL pipeline that can handle data from 10
source systems and function correctly despite changes to the data
schema.The other options are not solutions that will meet the
requirements. Option C, configuring an AWS Lambda function to invoke an
AWS Glue crawler when a file is loaded into the S3 bucket, and creating a
second Lambda function to run the AWS Glue job, is not a feasible
solution, as it would require a lot of Lambda invocations
andcoordination. AWS Lambda has some limits on the execution time,
memory, and concurrency, which can affect the performance and
reliability of the ETL pipeline. Option D, configuring an AWS Lambda
function to invoke an AWS Glue workflow when a file is loaded into the
S3 bucket, is not a necessary solution, as you can use an Amazon
EventBridge rule to invoke the AWS Glue workflow directly, without the
need for a Lambda function. Option E, configuring an AWS Lambda function
to invoke an AWS Glue job when a file is loaded into the S3 bucket, and
configuring the AWS Glue job to put smaller partitions of the DataFrame
into an Amazon Kinesis Data Firehose delivery stream, is not a
cost-effective solution, as it would incur additional costs for Lambda
invocations and data delivery. Moreover, using Amazon Kinesis Data
Firehose to load data into Amazon Redshift is not suitable for frequent
and small batches of data, as it can cause performance issues and data
fragmentation. References:
AWS Glue
Amazon EventBridge
Using AWS Glue to run ETL jobs against non-native JDBC data sources
[AWS Lambda quotas]
[Amazon Kinesis Data Firehose quotas]
Question # 68
A media company uses software as a
service (SaaS) applications to gather data by using third-party tools.
The company needs to store the data in an Amazon S3 bucket. The company
will use Amazon Redshift to perform analytics based on the data.Which AWS service or feature will meet these requirements with the LEAST operational overhead?
A. Amazon Managed Streaming for Apache Kafka (Amazon MSK)
B. Amazon AppFlow C. AWS Glue Data Catalog D. Amazon Kinesis
Answer: B Explanation:
Amazon AppFlow
Amazon AppFlow | SaaS Integrations List
Get started with data integration from Amazon S3 to Amazon Redshift using AWS Glue interactive sessions
Explanation: Amazon
AppFlow is a fully managed integration service that enables you to
securely transfer data between SaaS applications and AWS services like
Amazon S3 and AmazonRedshift. Amazon AppFlow supports many SaaS
applications as data sources and targets, and allows you to configure
data flows with a few clicks. Amazon AppFlow also provides features such
as data transformation, filtering, validation, and encryption to
prepare and protect your data. Amazon AppFlow meets the requirements of
the media company with the least operational overhead, as it eliminates
the need to write code, manage infrastructure, or monitor data
pipelines. References:
Question # 69
A company receives a daily file
that contains customer data in .xls format. The company stores the file
in Amazon S3. The daily file is approximately 2 GB in size.A data
engineer concatenates the column in the file that contains customer
first names and the column that contains customer last names. The data
engineer needs to determine the number of distinct customers in the
file.Which solution will meet this requirement with the LEAST operational effort?
A. Create and run an Apache Spark job in an AWS
Glue notebook. Configure the job to read the S3 file and calculate the
number of distinct customers.
B. Create an AWS Glue crawler to create an AWS Glue Data Catalog of the S3 file. Run SQL queries from Amazon Athena to calculate the number of distinct customers. C. Create and run an Apache Spark job in Amazon EMR Serverless to calculate the number of distinct customers. D. Use AWS Glue DataBrew to create a recipe that uses the COUNT_DISTINCT aggregate function to calculate the number of distinct customers.
Answer: D Explanation:
It does not require you to
write any code, as DataBrew provides a graphical user interface that
lets you explore, transform, and visualize your data. You can use
DataBrewto concatenate the columns that contain customer first names and
last names, and then use the COUNT_DISTINCT aggregate function to count
the number of unique values in the resulting column2.
It does
not require you to provision, manage, or scale any servers, clusters, or
notebooks, as DataBrew is a fully managed service that handles all the
infrastructure for you. DataBrew can automatically scale up or down the
compute resources based on the size and complexity of your data and
recipes1.
It does not require you to create or update any AWS
Glue Data Catalog entries, as DataBrew can automatically create and
register the data sources and targets in the Data Catalog. DataBrew can
also use the existing Data Catalog entries to access the data in S3 or
other sources3.
Option A is incorrect because it suggests
creating and running an Apache Spark job in an AWS Glue notebook. This
solution has the following disadvantages:
It requires you to
write code, as AWS Glue notebooks are interactive development
environments that allow you to write, test, and debug Apache Spark code
using Python or Scala. You need to use the Spark SQL or the Spark
DataFrame API to read the S3 file and calculate the number of distinct
customers.
It requires you to provision and manage a development
endpoint, which is a serverless Apache Spark environment that you can
connect to your notebook. You need to specify the type and number of
workers for your development endpoint, and monitor its status and
metrics.
It requires you to create or update the AWS Glue Data
Catalog entries for the S3 file, either manually or using a crawler. You
need to use the Data Catalog as a metadata store for your Spark job,
and specify the database and table names in your code.
Option
B is incorrect because it suggests creating an AWS Glue crawler to
create an AWS Glue Data Catalog of the S3 file, and running SQL queries
from Amazon Athena to calculate the number of distinct customers. This
solution has the following disadvantages:
It requires you to
create and run a crawler, which is a program that connects to your data
store, progresses through a prioritized list of classifiers to determine
the schema for your data, and then creates metadata tables in the Data
Catalog. You need to specify the data store, the IAM role, the schedule,
and the output database for your crawler.
It requires you to
write SQL queries, as Amazon Athena is a serverless interactive query
service that allows you to analyze data in S3 using standard SQL. You
need to use Athena to concatenate the columns that contain customer
first names and last names, and then use the COUNT(DISTINCT) aggregate
function to count the number of unique values in the resulting column.
Option
C is incorrect because it suggests creating and running an Apache Spark
job in Amazon EMR Serverless to calculate the number of distinct
customers. This solution has the following disadvantages:
It
requires you to write code, as Amazon EMR Serverless is a service that
allows you to run Apache Spark jobs on AWS without provisioning or
managing any infrastructure. You need to use the Spark SQL or the Spark
DataFrame API to read the S3 file and calculate the number of distinct
customers.
It requires you to create and manage an Amazon EMR
Serverless cluster, which is a fully managed and scalable Spark
environment that runs on AWS Fargate. You need to specify the cluster
name, the IAM role, the VPC, and the subnet for your cluster, and
monitor its status and metrics.
It requires you to create or
update the AWS Glue Data Catalog entries for the S3 file, either
manually or using a crawler. You need to use the Data Catalog as a
metadata store for your Spark job, and specify the database and table
names in your code.
References:
1: AWS Glue DataBrew - Features
2: Working with recipes - AWS Glue DataBrew
3: Working with data sources and data targets - AWS Glue DataBrew
[4]: AWS Glue notebooks - AWS Glue
[5]: Development endpoints - AWS Glue
[6]: Populating the AWS Glue Data Catalog - AWS Glue
[7]: Crawlers - AWS Glue
[8]: Amazon Athena - Features
[9]: Amazon EMR Serverless - Features
[10]: Creating an Amazon EMR Serverless cluster - Amazon EMR
[11]: Using the AWS Glue Data Catalog with Amazon EMR Serverless - Amazon EMR
Explanation: AWS
Glue DataBrew is a visual data preparation tool that allows you to
clean, normalize, and transform data without writing code. You can use
DataBrew to create recipes that define the steps to apply to your data,
such as filtering, renaming, splitting, or aggregating columns. You can
also use DataBrew to run jobs that execute the recipes on your data
sources, such as Amazon S3, Amazon Redshift, or Amazon Aurora. DataBrew
integrates with AWS Glue Data Catalog, which is a centralized metadata
repository for your data assets1.The solution that meets the
requirement with the least operational effort is to use AWS Glue
DataBrew to create a recipe that uses the COUNT_DISTINCT aggregate
function to calculate the number of distinct customers. This solution
has the following advantages:
Question # 70
A data engineer is configuring an
AWS Glue job to read data from an Amazon S3 bucket. The data engineer
has set up the necessary AWS Glue connection details and an associated
IAM role. However, when the data engineer attempts to run the AWS Glue
job, the data engineer receives an error message that indicates that
there are problems with the Amazon S3 VPC gateway endpoint.The data engineer must resolve the error and connect the AWS Glue job to the S3 bucket.Which solution will meet this requirement?
A. Update the AWS Glue security group to allow inbound traffic from the Amazon S3 VPC gateway endpoint.
B. Configure an S3 bucket policy to explicitly grant the AWS Glue job permissions to access the S3 bucket. C. Review the AWS Glue job code to ensure that the AWS Glue connection details include a fully qualified domain name. D. Verify that the VPC's route table includes inbound and outbound routes for the Amazon S3 VPC gateway endpoint.
Answer: D Explanation:
Troubleshoot the AWS Glue error “VPC S3 endpoint validation failed”
Amazon VPC endpoints for Amazon S3
[AWS Certified Data Engineer - Associate DEA-C01 Complete Study Guide]
Explanation: The
error message indicates that the AWS Glue job cannot access the Amazon
S3 bucket through the VPC endpoint. This could be because the VPC’s
route table does not have the necessary routes to direct the traffic to
the endpoint. To fix this, the data engineer must verify that the route
table has an entry for the Amazon S3 service prefix
(com.amazonaws.region.s3) with the target as the VPC endpoint ID. This
will allow the AWS Glue job to use the VPC endpoint to access the S3
bucket without going through the internet or a NAT gateway. For more
information, see Gateway endpoints. References:
