Opportunity brief
About the Role:
We are looking for a Data Engineering Lead to join our AMS team supporting a large scale analytics platform for a major Oil & Gas enterprise. You will lead the data engineering track, managing ETL pipelines, data transformations, and ensuring platform reliability.
Key Technologies:
- AWS Glue, AWS Lambda, Amazon Redshift, Aurora PostgreSQL
- MSK Kafka, AWS SQS, AWS Transfer Family
- Apache Airflow (orchestration)
- SQL, Python
What You'll Do:
- Lead a team of 3 data engineers in maintaining & enhancing large-scale data assets
- Own ETL pipeline performance, monitoring & optimization
- Handle production incidents (24x7 rotation for P1 issues)
- Drive enhancements within defined SLA timelines
- Collaborate with client business advisory team
- Create & maintain runbooks and documentation
Requirements:
- 8+ years in Data Engineering with leadership experience
- Expert in AWS data services (Glue, Redshift, Aurora)
- Strong SQL and Python skills
- Experience managing production data pipelines at scale
- Ability to work from Mumbai (client site as required)
- Available for weekend/night on-call rotation
Nice to Have: - Experience in Oil & Gas domain / large enterprise analytics