Data Engineer

Role: Data Engineer

Location: Mississauga, ON / Montreal, QC / Iselin NJ 3 Days Hybrid from Day 1

Mode: Contract

Must Have:

• Snowflake
• SQL
• Python
• ADF

Job Description:

This position is for a Cloud Data Engineer with a background in Python, Pyspark, SQL, and data warehousing for enterprise-level systems. The position calls for someone that is comfortable working with business users along with business analyst expertise.

Major Responsibilities:

• Build and optimize data pipelines for efficient data ingestion, transformation, and loading from various sources while ensuring data quality and integrity.
• Design, develop, and deploy Spark programs in the Databricks environment to process and analyze large volumes of data.
• Experience with Delta Lake, DWH, Data Integration, Cloud, Design, and Data Modeling.
• Proficient in developing programs in Python and SQL.
• Experience with Data Warehouse Dimensional data modeling.
• Working with event-based/streaming technologies to ingest and process data.
• Working with structured, semi-structured, and unstructured data.
• Optimize Databricks jobs for performance and scalability to handle big data workloads.
• Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.
• Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.
• Proficient in writing SQL queries and programming including stored procedures and reverse engineering existing processes.
• Perform code reviews to ensure fit to requirements, optimal execution patterns, and adherence to established standards.

Skills:

• 5+ years Python coding experience.
• 5+ years SQL Server based development of large datasets.
• 5+ years experience with developing and deploying ETL pipelines using Databricks Pyspark.
• Experience in any cloud data warehouse like Synapse, Big Query, Redshift, Snowflake.
• Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
• Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills.
• Experience with Cloud-based data architectures, messaging, and analytics.
• Cloud certification(s).
• Any experience with Airflow is a plus.

Job Type: Fixed term contract
Contract length: 24 months

Pay: $50.00-$70.00 per hour

Expected hours: 40 per week

Work Location: In person