Week 7 Worklog

Week 7 Objectives:

  • Master AWS Data Engineering & Analytics services: S3, Kinesis, Glue, Athena, QuickSight, Redshift.
  • Understand and implement the full pipeline: Ingest → Store → Catalog → Transform → Analyze → Visualize → Serve.
  • Explore Amazon DynamoDB from fundamentals to advanced serverless & event-driven designs.
  • Work with AWS Glue (interactive, GUI, and DataBrew) for data cleaning and transformation.
  • Utilize Cloud9, CloudShell, and SDK tools to manage data engineering workloads.
  • Build QuickSight dashboards and improve them with advanced visual features.

Tasks to be carried out this week:

DayTaskStart DateCompletion DateReference Material
1Perform Kinesis → Glue → Athena → QuickSight – Lab35:
+ 3.x: Create S3 bucket, Delivery Stream, Sample Data
+ 4.x: Glue Crawler, Data Check
+ 5.x: Code explanation, S3 output, session connect
+ 6.x: Analyze with Athena & Visualize with QuickSight
+ 7: Cleanup		20/10/202520/10/2025AWS Study Group
2DynamoDB – Lab39:
+ 1–3: Hands-on DynamoDB, explore console
+ 4: Backup
+ 6: Advanced Design Patterns
+ 7: Global Serverless Application
+ 8: Event-driven Architecture		21/10/202521/10/2025AWS Study Group
3DynamoDB Cost Optimization – Lab40:
+ 2.x: Preparing & building DB
+ 3.x: Data, cost, tagging, allocation, usage
+ 3.5: Additional query features
+ 4: Cleanup		22/10/202522/10/2025AWS Study Group
4CloudShell – SDK – Cloud9 – DataBrew – Lab60 & Lab70:
+ Lab60: CloudShell, Console, SDK usage
+ Lab70: Cloud9 → dataset download → upload to S3
+ DataBrew profiling, cleaning, transforming		23/10/202523/10/2025AWS Study Group
5Full Analytics Pipeline – Lab72 & Lab73:
+ Lab72: Prep → Ingest → Store → Catalog → Glue Interactive → Glue GUI → DataBrew → EMR → Athena → Kinesis Analytics → QuickSight → Lambda → Redshift
+ Lab73: Build Dashboard → Improve Dashboard → Create Interactive Dashboard		24/10/202524/10/2025AWS Study Group

Week 7 Achievements:

  • Successfully built an end-to-end analytics pipeline from ingestion to dashboard visualization.
  • Used Glue Crawler for Data Cataloging, Athena for interactive SQL analytics, and QuickSight for BI visualization.
  • Developed deep understanding of DynamoDB: partition key design, advanced patterns, backup, cost allocation, global replication.
  • Built serverless and event-driven architectures using DynamoDB Streams and Lambda.
  • Leveraged CloudShell, SDK, and Cloud9 for data engineering workflows.
  • Applied DataBrew to clean and transform datasets visually.
  • Used EMR and Kinesis Analytics for large-scale processing and streaming analytics.
  • Built and enhanced professional BI dashboards using QuickSight with interactive features.
  • Gained comprehensive insight into AWS Data Analytics ecosystem and cross-service integrations.