Associate Specialist , Data Engineering

• Build, enhance, and support

data pipelines using defined technical designs and backlog requirements.

• Develop and maintain

, and SQL across data lake, lakehouse, and warehouse environments.

• Work closely with Data Analysts, Data Scientists, senior engineers, tech leads, and product managers to understand requirements and deliver

.

• Implement

, validations, reconciliations, and basic anomaly checks to improve trust and usability of data outputs.

• Run, monitor, and troubleshoot pipelines using orchestration and observability tools such as

, scheduling, logging, monitoring, and alerting.

• Follow engineering practices including

.

• Support BI and analytics use cases by applying

.

• Write and tune

queries for data profiling, transformations, validations, debugging, and performance improvements.

• Use

, and CloudWatch to support data engineering workloads while following security practices such as IAM, encryption, and least privilege.

• Contribute to cloud resource provisioning and environment configuration using

, with guidance from senior engineers.

• Package, deploy, and support workloads using

and related runtime configurations, including ECS/Fargate where applicable.

• Use

pipelines.

• Develop scalable data processing logic on

, ACID transactions, and schema evolution.

• Use Jupyter/Databricks notebooks for exploration, debugging, and PoCs; convert validated logic into reusable modules, tests, and deployment-ready pipelines.

• Participate in

delivery ceremonies, provide task-level estimates, share progress updates, and raise risks or dependencies early.

• Create and maintain technical documentation such as pipeline specifications, data contracts, runbooks, and support notes.

• Bachelor's degree in computer science, Engineering, or a related field, or equivalent practical experience.

•

of hands-on experience in data engineering, including building or supporting production data pipelines and ETL/ELT workflows.

• Practical experience with

, and CloudWatch; understanding of IAM, encryption, and cloud security basics.

• Hands-on experience with

.

• Strong

skills for joins, window functions, data profiling, transformations, validations, and performance tuning.

• Good working knowledge of

, including Spark fundamentals such as partitioning, shuffle, caching, file formats, debugging, and optimization.

• Understanding of

.

• Exposure to

, code reviews, branching, release practices, and engineering quality standards.

• Working knowledge of

for deployment, runtime configuration, and cloud environment support.

• Ability to work in Agile teams, communicate clearly, collaborate with cross-functional stakeholders, and take ownership of assigned deliverables.

• Exposure to data quality or testing frameworks such as Collibra, Immuta, and basic awareness of data governance practices including catalog, lineage, and access controls.

• Exposure to orchestration tools such as Airflow or modern table formats such as Delta, Iceberg.

• AWS certification such as Developer or Solutions Architect, or equivalent demonstrated cloud experience.