Data Engineer, Specialist

Founded in 1975, Vanguard is one of the world's leading investment management companies. The firm offers investments, advice, and retirement services to tens of millions of individual investors around the globe-directly, through workplace plans, and through financial intermediaries.

Vanguard India

Vanguard's office in India is a significant milestone in our global expansion. We are committed to establishing an enduring technology center in Hyderabad, Telangana and are excited to be adding talent who will focus on Artificial Intelligence (AI), mobile, and cloud-based technologies that drive our business outcomes and deliver a world-class experience for our clients.

Role Summary

The Lead Data Engineer, Specialist is responsible for designing and delivering scalable data platforms, cloud-native data pipelines, and analytics solutions. This is a hands-on technical leadership role requiring deep data engineering expertise, strong problem-solving skills, and the ability to lead a team of Data and AI Engineers.

The role requires a strong product mindset to understand business needs, translate them into sustainable technical solutions, and take end-to-end ownership of delivery, quality, reliability, and continuous improvement. The individual will collaborate with business and technology stakeholders to deliver secure, high-quality, and future-ready data solutions while contributing to modern AI-assisted engineering practices.

Responsibilities

• Lead and mentor a team of Data and AI Engineers, providing technical direction, coaching, and support.
• Lead solution architecture and technical design, ensuring alignment with business objectives and product priorities.
• Provide technical leadership through architecture reviews, design reviews, code reviews, and engineering best practices.
• Design, develop, and optimize scalable data pipelines that ingest, process, and transform large datasets from diverse sources.
• Build cloud-based data platforms and storage solutions to support analytical and operational workloads.
• Design and optimize data models, schemas, data quality controls, and governance frameworks.
• Implement batch and real-time data processing solutions using modern orchestration and streaming technologies.
• Partner with product owners, business stakeholders, analysts, architects, and engineering teams to deliver data-driven solutions.
• Take end-to-end ownership of solution design, development, deployment, production reliability, and continuous improvement.
• Evaluate technical options and make informed decisions considering performance, scalability, security, cost, and maintainability.
• Implement CI/CD, automation, testing, observability, and DevOps practices to improve engineering productivity and platform reliability.
• Apply AI-assisted development practices and modern engineering approaches to improve quality, efficiency, and innovation.
• Support production operations, troubleshoot complex issues, perform root-cause analysis, and implement preventive actions.
• Participate in application-support rotations and work in shifts or outside regular business hours when required during critical incidents, major releases, or other business-critical situations.

• Bachelor's degree in Computer Science, Information Technology, Engineering, Computer Applications, or a related discipline from a recognized institution.
• Minimum of 8 years of relevant technology experience.
• At least 4 years of hands-on experience designing and developing scalable data pipelines, ETL or ELT frameworks, and database or data platform solutions.
• Demonstrated experience in leading technical solutions and guiding engineering teams through design, development, deployment, and production support.
• Experience working in Agile product environments with strong ownership and accountability.

• Cloud Platforms: AWS, including S3, Glue, Lambda, Kinesis, Redshift, Athena, Step Functions, SNS/SQS, IAM, CloudWatch, and EMR
• Data Engineering: Redshift, Databricks, dbt, Spark
• Programming: Python, PySpark, Pandas, and Advanced SQL
• Orchestration and Streaming: Airflow and Kafka
• DevOps and Infrastructure: Docker, Terraform, GitHub Actions, CodePipeline, ECS, ECR, and CI/CD
• Analytics and Integration: Power BI and REST APIs
• Engineering Practices: Git, automated testing, monitoring, observability, data quality, and governance

• Strong product mindset with the ability to connect technical decisions to business and customer outcomes.
• Ability to lead technical solution design and drive delivery from concept through production.
• Strong end-to-end ownership of deliverables, technical decisions, quality, reliability, and support.
• Proven ability to lead, mentor, and develop engineers while remaining technically hands-on.
• Strong analytical, problem-solving, troubleshooting, and debugging skills.
• Effective communication and stakeholder management across technical and non-technical teams.
• Ability to manage ambiguity, balance competing priorities, and proactively address risks and dependencies.
• Willingness to support critical production incidents and application-support requirements outside regular working hours when needed.