Apache Spark Technical Specialist - Scala, Python
HCLTech
Sydney, New South Wales, AustraliaPosted 21 days agoDiscoveredMatch locked
Full-time
Job Summary
This role involves working as a Technical Specialist with expertise in apache spark, scala, and python. The primary responsibilities include developing, implementing, and maintaining data pipelines, performing data analysis, and ensuring the scalability and efficiency of data processing tasks using apache spark and related technologies.
Key Responsibilities
- Develop and implement data processing applications using apache spark, scala, and python.
- Design and maintain scalable data pipelines to process large volumes of data efficiently.
- Perform data analysis and optimization to enhance the performance of data processing tasks.
- Collaborate with cross functional teams to understand business requirements and translate them into technical solutions.
- Troubleshoot and resolve issues related to data processing and data quality.
- Stay updated with the latest advancements in apache spark, scala, and python to continuously improve data processing capabilities.
Skill Requirements
- Proficiency in apache spark for big data processing and analytics.
- Strong programming skills in scala for developing scalable applications.
- Advanced knowledge of python programming for data manipulation and analysis.
- Experience with data processing frameworks and tools in the apache spark ecosystem.
- Understanding of distributed computing principles and parallel data processing.
- Excellent problem-solving skills and ability to work in a fast paced environment.
- Strong communication skills and ability to collaborate effectively with team members.
Other Requirements
null
Not included in the source posting: about the role, benefits.
Skills
sparkpythonscala
Who can apply
The employer didn't state any visa, work authorization, citizenship or clearance requirements in this posting. Confirm with the employer before applying.
Read automatically from the employer's posting text. Always confirm with the employer — requirements can change after a job is published.