A minimum of 7+ years of proven experience as a Data Engineer with a strong background in data lakehouse architecture, ETL processes, and AWS integration.
Hands-on experience with data engineering tools and technologies (e.g. AWS Glue).
Strong experience with cloud platforms (mainly in AWS) and containerization (e.g., Docker, Kubernetes).
Strong SQL skills for data manipulation and querying.
Excellent problem-solving and communication skills.
Knowledge of data governance principles and data privacy regulations is a plus.
Proficiency in programming languages such as Python, Scala, or Java is a plus.
Your role and responsibilities
Data Lakehouse Architecture: – Design and maintain scalable and efficient data lakehouse architecture. – Combine the advantages of data lakes and data warehouses. – Implement best practices for data partitioning, indexing, and storage optimization.
Data Pipelines: – Develop robust data pipelines for data ingestion, transformation, and loading. – Utilize various data sources, including S3, AWS Glue, PostgreSQL, RedShift, Lambda, Cloudwatch, AWS Athena, MFT, and AWS Batch. – Ensure fault tolerance, scalability, and performance of data pipelines.
ETL Processes: – Create and manage ETL processes for data cleaning, transformation, and enrichment. – Implement data quality checks and monitoring to maintain data integrity.
Reverse ETL: – Build reverse ETL processes to enable data consumption by operational systems and downstream applications. – Facilitate the flow of insights and data back into transactional systems.
Data Governance: – Implement data governance policies and ensure compliance with data privacy and security regulations. – Collaborate with data stewards to define data lineage and ownership.
What you'll get...
Salary in $s
A friendly, flexible working environment with WFH option
Exposure to working with international clients (Singapore)
Growing firm, plenty of opportunities for career progression
