SUMMARY

Experienced Data Engineer, Python developer with 7+ years of experience in building scalable and distributed data processing solutions. Proficient in developing complex ETL pipelines using PySpark, SQL, NoSQL. Committed to delivering high-quality work and continuously improving skills to stay current with new technologies.

EXPERIENCE

NTT Data - Bangalore, India - Senior Data Engineer

Oct 2020 - PRESENT

Designed and developed various ETL pipelines using PySpark to process large datasets in distributed environments.
Optimized Spark jobs by tuning performance parameters and implementing best practices for partitioning and caching data to reduce complexity.
Worked on cost optimization for various pipelines and processes and achieved 70% reduction in cost and 85% reduction in time and > 90% cluster utilization.
Designed and implemented a data pipeline system which can support various clients with isolated billing for each client.
Worked on crucial projects with 30+ customers from across the globe with varying deadlines.
Built various dashboards to monitor various aspects of ETL progress and statistics of data.
Trained and supervised 4 members, providing guidance & training.

Almug - Bangalore, India - Data Engineer

Oct 2019 - Oct 2020

Developed multiple data extraction pipelines and techniques for web scraping.
Developed an efficient mechanism to extract coordinates of all elements on a target webpage using JavaScript and Python.
Implemented custom-made distributed systems using physical servers for dynamic load-balancing.
Supervised 2 junior developers.

HCL - Chennai, India - Python Developer

Feb 2018 - Aug 2019

Developed web applications for clients from the financial and healthcare domain.
Worked on building and improving systems to ensure data security and user level access management.
Closely worked with ML teams in developing web application counterparts for ML backend applications.

KGISL - Coimbatore, India - Python Developer

Feb 2017 - Dec 2017

Developed and maintained web applications built on Python, Flask, SQL.
Worked on migrating various financial and healthcare applications from Java to Python.

EDUCATION

JNTU, Anantapur - Bachelor of Technology (BTech) in Computer Science and Engineering

Aug 2012 - Apr 2016