Data Engineer

We are working with a company that specialises in building robust and scalable data solutions. They are seeking a highly skilled professional to join their dynamic team and contribute to cutting-edge cloud-native data platforms.

The Role

• Design, develop, and maintain scalable ETL pipelines using AWS Glue and Apache Spark (PySpark).
• Build and orchestrate data workflows using AWS Step Functions.
• Design and implement S3 Data Lake architectures following AWS best practices.
• Develop and deploy containerized applications on Amazon EKS using Kubernetes.
• Build event-driven data processing solutions using Amazon SQS, AWS Lambda, and KEDA for auto-scaling.
• Manage metadata using AWS Glue Data Catalog.
• Implement data validation and governance using AWS Glue Data Quality.
• Monitor applications and data pipelines using Amazon CloudWatch.
• Optimize ETL jobs, Spark workloads, and Kubernetes deployments for performance and scalability.
  
  

What You'll Need

• Strong experience in building scalable cloud-native data platforms and ETL pipelines on AWS.
• Hands-on expertise with AWS Glue, Spark ETL, Step Functions, Amazon EKS, Kubernetes, Lambda, S3 Data Lake architectures, SQS, KEDA, Glue Data Catalog, Glue Data Quality, and CloudWatch.
• Proficiency in Apache Spark (PySpark).
• Experience with containerized applications and Kubernetes.
• Ability to implement data validation and governance.
• Strong analytical and problem-solving skills.
  
  

What's On Offer

• Opportunity to work with cutting-edge cloud technologies.
• Contribute to scalable cloud-native data platforms.
• Full-time engagement with a dynamic team.
• Remote work flexibility.
  
  

Apply via Haystack today!