Lead Data Engineer

A Lead Data Engineer works closely with a multidisciplinary Agile team to build high quality data pipelines driving analytic solutions. These solutions will generate insights allowing to advance data-driven decision-making capabilities. This role requires deep understanding of data architecture, data engineering, data analysis, reporting, and a basic understanding of data science techniques and workflows. The ideal candidate is a skilled data/software engineer with experience creating data products supporting analytic solutions. They work as part of a technical, cross-functional analytics team.

KEY ACCOUNTABILITIES:

1. Design, develop, optimize, and maintain data architecture and pipelines that adhere to Extract, Transform, and Load (ETL) principles and business goals.
2. Solve complex data problems to deliver insights that help our business achieve their goals.
3. Create data products for analytics and data scientist team members to improve their productivity.
4. Advise, consult, mentor, and coach other data and analytic professionals on data standards and practices.
5. Foster a culture of sharing, re-use, design for scale stability, and operational efficiency of data and analytical solutions.
6. Lead the evaluation, implementation, and deployment of emerging tools and processes for analytic data engineering in order to improve our productivity as a team.
7. Develop and deliver communication and education plans on analytic data engineering capabilities, standards, and processes.
8. Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives.
9. Develop and deliver some data science courses as part of advance analytics and digital academy.
10. Perform other duties as assigned.

QUALIFICATIONS & SKILLS:

Minimum Qualifications:

1. Bachelor's degree or MS required; Computer Science, MIS, or Engineering preferred.
2. 5 years of experience working in a data engineering or architecture role, 7+ preferred.
3. Expertise in SQL and data analysis and experience with programming languages (Python, R, or Scala preferred).

Minimum Experience:

1. Experience developing and maintaining data warehouses in big data solutions.
2. Should have domain knowledge of metal and mining or energy or heavy industry or oil and gas experience or a combination of either industries.
3. Good knowledge of programming languages and developing data analytics models (R, Python, Spark, Java, Scala, etc.).
4. Experience with dashboard development and BI tools such as Tableau, Power BI, Looker, Shiny, Qlik, etc.
5. Experience with developing solutions on cloud computing services and infrastructure in the data and analytics space (preferred).
6. Experience in data architecture including the ability to define data retention policies, monitor performance, and advise any necessary infrastructure changes.
7. Good knowledge of data and analytics, e.g. dimensional modeling, ETL, reporting tools, data governance and warehousing, structured and unstructured data.
8. Practical knowledge in traditional ETL tools (Informatica, Alteryx, SAP data services, etc.) as well as big database systems like the Hadoop ecosystem and cloud technologies (Azure, Spark, etc.).
9. Database development experience using Hadoop or BigQuery and experience with a variety of relational, NoSQL, and cloud database technologies.
10. Big Data Development experience using Hive, Impala, Spark and familiarity with Kafka (preferred), familiarity with different operating systems (preferred).
11. Exposure to Machine Learning, data science, computer vision, AI, statistics, and/or applied mathematics.
12. Good working knowledge of industry 4.0 applications, big data, autonomous robots, system integration, cloud computing, additive manufacturing, augmented reality, etc.
13. Experience in automation & control systems and training in similar fields is valued.

AUTHORITY/DECISION MAKING:

1. Thinking within broadly defined policies, standards, and objectives. The determination of what needs to be done in applying policies is largely left up to the incumbent who must establish the plan, determine the priorities, and prescribe the processes needed to achieve the objectives.
2. Subject to general direction and broadly defined functional policy objectives. Actions which will impact other functional or operating areas usually require approval before they may be implemented.

Agile/Digital skills:

1. Experience working on collaborative Agile product teams.

Individual skills:

1. Self-motivated with strong problem-solving and learning skills.
2. Flexibility to changes in work direction as the project develops.
3. Excellent communication, listening, and influencing skills.

Mindset & Behaviors:

1. Flexibility in quickly adjusting positions based on stakeholders' input.
2. Ability to work at an abstract level and gain consensus.