Senior BigData Engineer • July-2023 to September-2024
Description: ● Conducted quantitative analysis and developed custom reports for clients in the healthcare industry. ● Collaborated with cross-functional terms to develop and deliver client presentations. ● Developed and executed sql queries to extract and manipulate healthcare data from various sources. ● Designed and implemented statistical models to identify patient risk factors and predict advance events. ● Familiarity with big data technologies such as Apache Hadoop, Apache Spark, and Apache Kafka for processing and analyzing large volumes of data. ● Tech used - Python, AWS Glue, AWS Lambda, AWS RDS, AWS GatewayApi,Pyspark,AWS DMS,AWS Sagemaker
Senior Bigdata Engineer • September-2021 to July-2023
Capgemini
Description: ● Data ingestion layer: A connection with data sources such as Actual Digital Sales, NDDC Forecasted Demand, Promo information, Product information, ELC Inventory and other sources. ● Data (pre-processing) layer: Responsible for processing the ingested data and makes sure the structure and quality is usable for model input and business review. ● Model development/Training layer: This layer is connected to the data layer and provides an independent place to train and improve models. It also includes a parameter tuning function to optimize the model’s hyper parameters. Once a model is trained it should be passed on to the back testing layer for validation. ● Tuning/back-testing layer: This layer is responsible for back testing a model and comparing it to another model over time. Once the model is back-tested and the results are as desired, the newly created model can be pushed into (pre)-production. ● Automated ML pipeline/ Inference layer: This layer is used to make the predictions. This layer will use a created model and use production data to make predictions. ● Designed and developed ETL pipelines using AWS Glue for ingesting, cleaning, and transforming data from various sources into Redshift and Snowflake data warehouses. ● Conducted performance tuning and optimization of SQL queries in Redshift and ● Snowflake to enhance data processing e ciency. ● Utilized Python and R for data analysis and statistical modeling, enabling data-driven decision-making for business stakeholders. ● Python, PySpark, AWS Glue, AWS Sagemaker, AWS Lambda, Snowflake, AWS RDS,Spark ML.