Senior/ Data Engineer - DSC/EZ

Job Overview

The Data Engineer supports the design, implementation and maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable and secure manner.

The Data Engineer focuses on defining optimal solutions to data collection, processing and warehousing. The job holder designs, codes and tests data systems and works on implementing those into the internal infrastructure.

Job Responsibilities

• Identify suitable data structures based on business needs to ensure availability and accessibility of data.
• Determine technical system requirements based on data needs.
• Keep abreast of latest technologies and products in database and data processing software, and technologies.
• Build scalable data pipelines to extract, transform, load and integrate data.
• Develop codes and scripts to process structured and unstructured data in real-time from a variety of data sources.
• Test data pipelines for scalability and reliability to process high data volume, variety and velocity.
• Implement and monitor data security and privacy measures on data solutions.
• Consolidate and create data storage solutions for storage and retrieval of information.
• Monitor data system performance.
• Assist in the integration of data systems with existing infrastructure.
• Develop tools to improve data flows between internal and/or external systems and the data warehouse.
• Automate the data collection and analysis processes, data releasing and reporting tools.
• Test data system configurations to increase efficiency.