Data Engineer for WSG with 4 years experience

Duration: 12 months, renewable

We are looking for an experienced Data Engineer to support the development and maintenance of WSG data products within MOM's SeDAAP. We require a team of four developers.

The detailed key responsibilities include:

1) Data Ingestion and Processing:

1. Ingest data from identified datasets (such as Career Coach 4.0 (CC 4.0) career coaching data, EDG (Enterprise Division Group) programme and grant data, Central Provident Fund (CPF) individual employment data) into MOM's SeDAAP.
2. Perform data extraction, cleaning, transformation, and flow.
3. Conduct web scraping when necessary for data extraction.
4. Integrate and ingest both internal WSG and external datasets (e.g., from DataHive) into MOM's SeDAAP.

2) Stakeholder Collaboration:

1. Work closely with various stakeholders on all data engineering related activities.
2. Gather requirements, conduct clarifications, implement and deploy solutions, perform testing, and provide support maintenance across all project environments.
3. Collaborate with Project Managers, Frontend Developers, UX Designers, and Data Analysts to build scalable, data-driven products that support lighthouse projects.

3) Data Architecture and Pipeline Development:

1. Design data models and develop data pipelines for data warehouses, data lakes, and operational data stores.
2. Create data marts and other data products to support secure analysis using various analytical tools.
3. Design, build, launch, and maintain efficient and reliable large-scale batch and real-time data pipelines using appropriate data processing frameworks.

4) Documentation and Best Practices:

1. Provide system documentation on design and development work.
2. Ensure proper coding standards, version control, release management, and change management of programs.
3. Work in an Agile environment practicing Continuous Integration and Delivery.
4. Engage in pair programming and code review processes with fellow developers.

5) Policy and Integration:

1. Address evolving policy changes and reporting needs.
2. Accommodate architecture adjustments resulting from technology upgrades.
3. Ensure smooth integration with other relevant systems.
4. Integrate and collate data silos in a scalable and compliant manner.

6) Backend Development:

1. Develop backend APIs and work on databases to support applications.

7) SeDAAP Integration and Cloud Migration:

1. Execute the migration of WSG's data into MOM's SeDAAP hosted in a cloud-based environment.
2. Ensure data integrity during migration and position the data within the SeDAAP architecture.
3. Collaborate with MOM's DevOps team and stakeholders to ensure seamless integration of applications and systems within SeDAAP.
4. Coordinate development efforts to align with MOM's automation processes.
5. Optimize platform use for specific needs and collaborate with MOM's DevOps for maintenance and support within SeDAAP.

Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related field.
• 4 years of experience in data engineering, backend development, or cloud migration.
• Prior experience with government-related data platforms like DataHive, CPF, or MOM’s SeDAAP is a plus.