AVP, Data Platform Development, LME

HKEX

CN-Shenzhen-HyQPosted 4 days agoDiscoveredMatch locked

Full time6+ years exp

Apply

Location

CN-Shenzhen-HyQ

Shift

Standard - 40 Hours (China)

Scheduled Weekly Hours

Worker Type

Permanent

Job Summary

Support the development, enhancement, and day‑to‑day operation of LME technology systems, including trading, market data, and supporting platforms. This entry‑level engineering role focuses on delivering high‑quality software, ensuring system stability, and collaborating with cross‑functional teams across LME Technology.

Role Summary

Design, build, and operate critical subsystems of the data platform. Take ownership of major components - streaming, batch, storage, or serving - drive them from design to production, and continuously improve their reliability and performance.

Responsibilities

•

Streaming Infrastructure

Kafka cluster operations (broker tuning, partition rebalancing, monitoring, disaster recovery); manage schema registry and Kafka Connect connectors •

Batch & Streaming Compute

Build and optimize Spark batch jobs and Flink/Spark Structured Streaming pipelines; contribute reusable job frameworks and tuning guides •

Storage & Lakehouse

Manage Iceberg tables (compaction, snapshot expiration, orphan file cleanup, schema evolution); operate MinIO at scale (lifecycle rules, tiering, performance tuning) •

Query & Serving

Deploy and operate Trino clusters (connector config, resource groups, query monitoring); manage StarRocks/ClickHouse clusters (sharding, replication, materialized views) •

Orchestration

Build and maintain Airflow/Dagster DAGs for platform operations; extend custom operators and sensors •

Platform Observability

Implement monitoring with OpenTelemetry, Prometheus, Grafana, and Loki across all stack layers; build dashboards and alerting rules •

Kubernetes Operations

Write Helm charts, manage operator lifecycles, configure resource quotas, node affinity, pod disruption budgets •

CI/CD & GitOps

Own ArgoCD application sets, Helm-based deployments, and promotion pipelines from dev to prod

Participate in on-call rotation; write post-mortems and runbooks
Mentor mid-level engineers through pairing, design discussions, and code reviews

Required Skills & Experience

6+ years in data/platform engineering or backend infrastructure
Strong Kubernetes: Helm chart authoring, RBAC, network policies, storage (PV/PVC), operators
Solid Kafka: topic design, consumer group management, offset management, monitoring lag, Kafka Connect, schema registry (Apicurio or Confluent)
Solid Spark: Dataframe/Dataset API, Spark SQL, performance tuning, troubleshooting in production
Working knowledge of Flink or Spark Structured Streaming for real-time pipelines
Practical Iceberg experience: table maintenance, time-travel, catalog integration
Hands-on Trino or Presto: connector configuration, query tuning, resource group management
Experience with an OLAP engine: StarRocks, ClickHouse, or Doris - table design, ingestion pipelines, query optimization
Proficient in Python and either Scala or Java
Solid CI/CD and GitOps: ArgoCD or Flux, Helm, Docker
Airflow or Dagster for pipeline orchestration

Nice to Have

OpenShift-specific: SCC, Routes, ImageStreams, BuildConfigs
dbt project experience for data transformation and modeling
Data quality frameworks: Great Expectations, Soda, or Deequ
Kafka Streams or ksqlDB for stream processing
Exposure to DataHub/Atlas for data discovery and lineage

Company Introduction

ITD SZ 港交所科技（深圳）有限公司，是2016年12月28日于深圳市前海自贸区成立的外商独资企业。作为港交所的技术子公司，港交所科技（深圳）有限公司主要是为集团及其附属公司提供计算机软件、计算机硬件、信息系统、云存储、云计算、物联网和计算机网络的开发、技术服务、技术咨询、技术转让；经济信息咨询、企业管理咨询、商务信息咨询、商业信息咨询、信息系统设计、集成、运行维护；数据库管理、大数据分析；以承接服务外包方式提供系统应用管理和维护、信息技术支持管理、数据处理等信息技术和业务流程外包服务。

Not included in the source posting: benefits.

Skills

kafkasparkairflowci-cdkubernetesdbtdockerjavapythonscalasqlgrafana

Who can apply

The employer didn't state any visa, work authorization, citizenship or clearance requirements in this posting. Confirm with the employer before applying.

Read automatically from the employer's posting text. Always confirm with the employer — requirements can change after a job is published.