Data Engineer • February-2023 to May-2024
Description: Developed an ETL pipeline from scratch for data ingestion into AWS Redshift from client APIs, encompassing synchronous API calling, preprocessing data through Glue Jobs, handling large datasets, managing data, implementing backups, and orchestrating Apace Airflow DAGs Python | Apache Airflow | AWS(S3, ECS, Cloudwatch, Glue, Redshift, IAM, EMR, Athena) | SQL | JQ | Docker