Data Ingestion and Transformation on AWS

Course content

1. 1Data engineering on AWSComing
2. 2Batch ingestion patternsComing
3. 3Streaming ingestion with Kinesis and Amazon MSKComing
4. 4AWS Glue jobs and crawlersComing
5. 5Transforming data with Spark on Amazon EMRComing
6. 6Lightweight transformation with LambdaComing
7. 7File formats: CSV, JSON, Parquet and othersComing
8. 8Orchestration with Step Functions and Amazon MWAAComing
9. 9Event-driven pipelinesComing
10. 10Programming concepts for data pipelinesComing
11. 11Pipeline cost and performanceComing
12. 12Case study: an ingestion pipeline for sales dataComing

Requirements

• AWS-204 Storage on AWS: S3, EBS, EFS and Backup
• AWS-206 Serverless and Event-Driven Applications

Recommended background. You can start any course at any time.

Description

Batch and streaming ingestion, transformation with AWS Glue, Amazon EMR and Lambda, and orchestrating data pipelines.