Data Engineering Lead (M-2025-0039)

Position Overview

The data engineering lead designs systems to facilitate access to and finding of information. He/She plans, designs, develops, and tests internal information-delivery solutions and data models with the focus on providing a positive user experience. He/She works with end users to specify requirements, create and implement designs to meet internal and client-facing objectives. He/She develops information management standards and practices.

He/She integrates diverse needs and perspectives from internal and external clients, possesses a creative mind to develop new and fresh ideas and solutions. He/She is able to influence key stakeholders and spearhead a data-driven approach to resolve business issues.

He/She holds responsibilities associated with data sourcing and preparation, data storage, reporting, analytics, data exploration, and information delivery. He/She works with senior management to understand and prioritize data and information requirements. He/She oversees the development of testing methodology and criteria, standards, policies, and procedures for the structure and attributes of the business intelligence tools and systems.

He/She holds responsibilities to build data flow channels and processing systems to extract, transform, load and integrate data from various sources. He/She assists with the integration of data systems into existing infrastructure.

Role & Responsibilities

Build data ingestion pipelines

1. Build data pipelines to extract, transform, load and integrate data from various sources.
2. Develop complex code, scripts, and data processing pipelines to process structured and unstructured data in near real-time.
3. Clean, standardize, and harmonize data for analytics use cases.
4. Validate the data sources and methods for data ingestion, and establish data transfer protocols such as API integration, SFTP pipelines, and message queue implementation.

Manage Data Engineering Team

1. Oversee work allocations and staffing within a team.
2. Develop staff through ongoing coaching, mentoring, and career discussions.
3. Define common goals, direction, and accountability among staff.
4. Drive effective performance management practices within the department in accordance with company policies and procedures.

Set data standards and governance

1. Develop standards, policies, and procedures for the form, structure, and attributes of the data tools and systems.
2. Create long-term, lean data governance initiatives that serve to improve data quality across all systems over time.
3. Provide guidance on best practices related to data governance.
4. Educate and train staff in data standards and governance.

Identify business needs

1. Collaborate with stakeholders to understand needs for data structure, availability, integrity, quality, scalability, and accessibility.
2. Translate data business needs into technical system requirements.
3. Recommend adoption of technologies, strategies, and products in database and data processing software.
4. Contribute to defining data retention policies.

Manage projects

1. Plan project timelines and resources needed.
2. Deliver projects in line with agreed standards, providing fit for purpose solutions within time, quality, and budget constraints.
3. Apply project management tools and processes to ensure project is cost-effective.
4. Identify any project risks.
5. Prepare project status reports for stakeholders.
6. Communicate project objectives at critical junctions to obtain buy-in from all stakeholders.

Requirements

• Minimum 10 years of experience in delivering data warehouse or advanced analytics solutions.
• Minimum 5 years of relevant hands-on experience in delivering end-to-end industry analytics solutions, from conceptualization to deployment, within the industry.
• Demonstrate good, in-depth knowledge in BI/Big data hardware/software products, frameworks, and methodologies.
• Degree/Master in Computer Science, Information Technology, Computer Engineering or equivalent.
• Comfortable working with and staffing senior management, proficient in usage of Office Productivity Suite Software.
• Possess good verbal and written communication, analytical and conflict resolution skills with proven ability to translate complex, technical subjects into clear and concise communications to a variety of key stakeholders of different levels.
• Strong ability to handle ambiguity and high work pressure.
• Versatile in working independently as well as an effective team player.