Data Engineer

Position : Data Engineer

• Migrate data pipelines from existing data acquisition framework to the new GDP data acquisition framework

• Configure,develop and deliver data ingestion scripts for loading data into T1 data layer

• Develop and manage ETL/ELT workflows, ensuring data quality, integrity, and reliability.

• Integrate and automate data quality checks and validation processes within data pipelines.

• Deploy and manage containerized applications using Docker and orchestrate workloads on Kubernetes.

• Work with modern data lake and warehouse technologies such as Iceberg

• Implement real-time data streaming solutions using Kafka.

• Orchestrate complex workflows using Airflow.

• Integrate with data catalog and governance tools such as Datahub and Ranger.

• Collaborate with cross-functional teams to understand business requirements and deliver data solutions.

• Ensure security, compliance, and best practices in data management and governance.

• Strong proficiency in Linux, Python, and Shell scripting.

• Hands-on experience with Docker, Kubernetes, and container orchestration.

• Hands-on experience with Minio and Azure Data Lake Storage (ADLS) using S3 protocols.

• Experience with Apache Iceberg, Kafka, Airflow, Datahub, Trino, and Ranger.

Work Location : Singapore.

Interested applicants- Please send us resume at [email protected]