Python Developer - Data Engineer

Python Developer - Data Engineer

This is with a large IT services firm for a major US bank in Canada.

Candidate must have legal work status for Canada.

Client Location is Mississauga ON.

Hybrid - In-Office 3 days per week.

Perm/FT role, salary and benefits at market rate.

We are seeking a passionate and highly skilled Python Data Engineer to spearhead the development and maintenance of our cutting-edge data infrastructure. As a leader in our data team, you'll play a pivotal role in designing, building, and optimizing data pipelines that empower our organization with insightful and actionable data. If you thrive in a fast-paced environment, possess a strong collaborative spirit, and are eager to make a significant impact, we encourage you to apply.

Objectives of this role

1. Design, architect, and implement robust and scalable data pipelines using Python and related technologies (Airflow, PySpark, PyFlink are a plus).
2. Champion best practices for data engineering, code quality, testing, and deployment.
3. Mentor and guide a team of talented data engineers, fostering a collaborative and high-performing team culture.
4. Collaborate closely with Data Scientists, Data Analysts, and business stakeholders to translate complex business requirements into efficient data solutions.
5. Continuously research and implement new technologies and best practices to improve the efficiency and scalability of our data platform.
6. Take ownership of the deployment and monitoring of data pipelines and related infrastructure on cloud platforms such as Openshift, ECS, or Kubernetes.

Responsibilities

1. Lead the design and development of data pipelines for ingestion, transformation, and loading of data from various sources (databases, APIs, streaming platforms) into our data warehouse/lake.
2. Write optimized and maintainable SQL queries and leverage SQLAlchemy for efficient database interaction.
3. Implement robust data quality checks and monitoring systems to ensure data integrity and accuracy.
4. Develop comprehensive documentation and contribute to knowledge sharing within the team.
5. Contribute to the design and implementation of data governance policies and procedures.

Required skills and qualifications

1. 5+ years of hands-on experience in a Data Engineering role, with a strong proficiency in Python (version 3.6+).
2. Extensive experience working with relational databases and writing complex SQL queries.
3. Proven expertise with SQLAlchemy or similar ORM libraries.
4. Experience with workflow management tools like Airflow (experience with PySpark or PyFlink is a major plus).
5. Solid understanding of data warehousing concepts and experience working with large datasets.
6. Ability to guide and mentor junior developers, fostering a collaborative team environment.
7. Strong communication skills, both written and verbal, with the ability to explain complex technical concepts to both technical and non-technical audiences.
8. Experience deploying and managing applications on cloud platforms like Openshift, ECS, or Kubernetes.

Preferred skills and qualifications

1. Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
2. Experience with data visualization tools and techniques.
3. Familiarity with agile development methodologies.
4. Contributions to open-source projects or active participation in the data engineering community.