Data Engineer • August-2023
Description: Data Migration Project: Aug 2023- current ● Spearheaded apivotal role in a data migration project, ensuring seamless transition and accuracy in data transfer. ● Implemented robust processes to validate and cleanse data, guaranteeing its quality and integrity throughout the migration. Data Processing and Validation: ● Executedcomprehensive data processing and validation procedures, leveraging a suite of tools including Python,teradata, SQL, PySpark, Pandas, Numpy,DBT,clickhouse,nifi ● Significantly enhanced data quality and reduced processing time through the implementation of effective methodologies. Quality Assurance and Risk Management: ● Appliedrigorous methodology and governance to minimize both business and technical risks, ensuring adherence to quality assurance standards and data reconciliation. ● Ledreverse engineering exercises for legacy ETL and data management solutions, optimizing processes and ensuring compliance with industry best practices. Business-Centric Approach: ● Collaborated with cross-functional teams to understand business problems, aligning technology and analytical tools to deliver comprehensive solutions. ● Utilized big data technology to ingest and analyze data streams, aligning with business goals and objectives. Develop dashboards and design tables to help in data quality