AWS Data Cloud Engineer - Johannesburg
Salary Negotiable
Johannesburg, Gauteng
Johannesburg,
Gauteng
more than 14 days ago
03-03-2025 12:13:23 PM
We are seeking a highly skilled AWS Data Cloud Engineer (Big Data Engineer) to join our dynamic team. The ideal candidate will be responsible for designing and implementing large-scale enterprise data solutions using AWS cloud technologies. You will work on migrating and optimizing on premise data warehouses to AWS, building robust data pipelines, and enabling advanced analytics solutions.
Key Responsibilities:
Enterprise Data Solutions: Design and implement large-scale data solutions using AWS services, including AWS Glue, AWS Step Functions, AWS Redshift, AWS Lambda, AWS Athena, AWS Lake Formation, Spark, and Python.
Cloud Data Migration: Analyze, re-architect, and migrate on premise data warehouses to AWS cloud platforms, leveraging AWS-native and third-party services.
Data Pipeline Development: Design and build production-grade data pipelines for data ingestion, transformation, and integration using PySpark, Python, and SQL.
Analytical Infrastructure: Develop and maintain analytical data infrastructure to support ad-hoc queries, reporting, and large-scale data analysis.
Data Integration: Interface with other technology teams to extract, transform, and load (ETL) data from various structured and unstructured sources.
Innovation & Optimization: Continuously research the latest big data and visualization technologies to enhance capabilities and improve efficiency.
Advanced Analytics Support: Collaborate with data science and engineering teams to implement advanced analytics, machine learning models, and predictive analytics.
Process Automation: Improve ongoing reporting and analytics workflows through automation and self-service capabilities.
Required Experience & Skills:
Advanced Data Engineering Expertise: Strong knowledge of modern data engineering practices and handling large-scale distributed data processing.
Cloud-Based Data Platforms: Experience in designing, developing, and maintaining data solutions on AWS Cloud using AWS-native services.
Big Data Technologies: Hands-on experience with AWS Glue, PySpark, SQL, and distributed computing frameworks.
SQL & Data Processing: Expertise in SQL for complex queries, data transformation, and optimization.
Root Cause Analysis: Ability to perform root cause analysis on data and process issues to improve operational efficiency.
Data Infrastructure & Pipelines: Strong skills in building ETL processes, metadata management, and workload optimization.
Agile & DevOps Practices: Understanding of agile methodologies, software engineering best practices, version control, CI/CD, and cloud deployment.
Additional Experience (Preferred, but Not Required):
Experience with AWS Step Functions and Control-M for workflow management.
Familiarity with AWS services like EC2, EMR, RDS, and DynamoDB.
Experience working in cross-functional teams in a dynamic environment.
A successful track record of manipulating, processing, and extracting insights from large datasets.
Minimum Qualifications:
Bachelor’s Degree in Computer Science, Information Technology, or a related field.
Proven experience with AWS Athena, AWS Glue, PySpark, AWS DynamoDB, AWS Redshift, AWS Lambda, and AWS Step Functions.
Proficiency in SQL, Python, and PySpark.
Experience in cloud computing platforms and services.
Strong understanding of software engineering best practices, including coding standards, code reviews, testing, DevOps, and system operations.
Job Type:
Contract
Workplace Type:
Hybrid
Location:
Gauteng
Experience Level:
Intermediate to Senior