Senior Technical Lead
HCLTech
City Of London, United KingdomPosted about 1 month agoDiscoveredMatch locked
Full-time
Role
- Configure Databricks Auto Loader or Apache Spark Structured Streaming to securely subscribe to the relevant SAP AEM topics.
- Develop the PySpark/SQL data pipelines to process the incoming JSON payloads through the Delta Lake layers.
- Carry out data modelling in Databricks.
- Expose the Delta tables to the visualisation layer.
- Ensure that the data pipelines are optimised.
Skillsets
:
- Expert in Apache Spark Structured Streaming and Databricks Auto Loader.
- Fluent in PySpark and Databricks SQL.
- Proven experience building Databricks architecture patterns.
Familiarity with Unity Catalog for data governance and access control.
Role
- Configure Databricks Auto Loader or Apache Spark Structured Streaming to securely subscribe to the relevant SAP AEM topics.
- Develop the PySpark/SQL data pipelines to process the incoming JSON payloads through the Delta Lake layers.
- Carry out data modelling in Databricks.
- Expose the Delta tables to the visualisation layer.
- Ensure that the data pipelines are optimised.
Skillsets
:
- Expert in Apache Spark Structured Streaming and Databricks Auto Loader.
- Fluent in PySpark and Databricks SQL.
- Proven experience building Databricks architecture patterns.
Familiarity with Unity Catalog for data governance and access control.
Role
- Configure Databricks Auto Loader or Apache Spark Structured Streaming to securely subscribe to the relevant SAP AEM topics.
- Develop the PySpark/SQL data pipelines to process the incoming JSON payloads through the Delta Lake layers.
- Carry out data modelling in Databricks.
- Expose the Delta tables to the visualisation layer.
- Ensure that the data pipelines are optimised.
Skillsets
:
- Expert in Apache Spark Structured Streaming and Databricks Auto Loader.
- Fluent in PySpark and Databricks SQL.
- Proven experience building Databricks architecture patterns.
Familiarity with Unity Catalog for data governance and access control.
Role
- Configure Databricks Auto Loader or Apache Spark Structured Streaming to securely subscribe to the relevant SAP AEM topics.
- Develop the PySpark/SQL data pipelines to process the incoming JSON payloads through the Delta Lake layers.
- Carry out data modelling in Databricks.
- Expose the Delta tables to the visualisation layer.
- Ensure that the data pipelines are optimised.
Skillsets
:
- Expert in Apache Spark Structured Streaming and Databricks Auto Loader.
- Fluent in PySpark and Databricks SQL.
- Proven experience building Databricks architecture patterns.
Familiarity with Unity Catalog for data governance and access control.
Not included in the source posting: about the role, what you'll do, benefits.
Skills
databrickssparksqlsap
Who can apply
The employer didn't state any visa, work authorization, citizenship or clearance requirements in this posting. Confirm with the employer before applying.
Read automatically from the employer's posting text. Always confirm with the employer — requirements can change after a job is published.