Data Engineer

Company Overview

TECHCOM SOLUTIONS, founded in 2003, is a Singapore-based IT total solutions provider serving government agencies, MNCs, system integrators, educational institutions, and BFSI across Singapore and APAC. We specialize in outsourcing, managed services, and staffing solutions.

Job Summary

You will develop and optimize data integration pipelines and ETL/ELT codes using Teradata SQL, Informatica, and Apache Spark to support data warehousing and analytics. You will also automate workflows and enhance application performance to drive efficient data operations.

Responsibilities

• Develop ETL/ELT codes using Teradata SQL, Informatica PowerCenter, Apache Spark, QueryGrid, and Trino to transform and load data into Teradata data warehouses and datamarts
• Optimize and tune performance of complex data applications to reduce resource consumption and improve efficiency
• Design and implement data pipeline orchestration frameworks using GCFR, shell scripting, and BTEQ to automate end-to-end data workflows
• Integrate automation frameworks with control systems to ensure reliable and scalable data processing
• Apply CDC (Change Data Capture) concepts, preferably with Java-based implementations, to support real-time data updates
• Utilize Apache Spark and Hadoop ecosystem tools (Hive, Kafka, NiFi, Iceberg, Impala) to manage and process large-scale data sets
• Write and maintain shell scripts and BTEQ scripts to support data operations and automation tasks

Preferred competencies and qualifications

• 6+ years of relevant experience in data engineering and ETL development