Senior Data Architect

Job Title: Senior Data Architect
Location: BC or ON, with preference in BC – if the candidate is on the East Coast they will be working at PST hours
Job ID: 78884

What you will do:

• Cloud Data Architecture Design: Lead the design, implementation, and optimization of data solutions within Azure Cloud, using technologies such as Azure Databricks, Apache Spark, Unity Catalog, and Purview. Build architectures that are secure, scalable, compliant, and cost-efficient, with an emphasis on data governance at every stage.
• Data Governance Framework: Work in conjunction with the Data Governance team to implement and enforce robust data governance frameworks across the data lifecycle, ensuring that all data processes align with regulatory requirements (e.g., PIPEDA, OSFI, FINTRAC), internal policies, and industry best practices. Utilize Unity Catalog & Purview for cataloging, lineage, and compliance tracking to ensure data integrity, traceability, and accessibility.
• Data Pipeline Architecture & CI/CD Integration: Architect automated, scalable, and governance-compliant data pipelines using Apache Spark and Databricks Notebooks. Implement CI/CD practices to ensure seamless deployment, version control, testing, and monitoring of data workflows.
• Cloud Migration Strategy: Collaborate with cross-functional teams to plan and execute the migration of existing data infrastructure to Azure Cloud, ensuring that governance, security, and compliance are maintained throughout the migration process.
• Data Access & Security Management: Leverage Unity Catalog to define, enforce, and monitor data access policies, ensuring that data is available only to authorized users and teams. Work closely with security and compliance teams to integrate security controls and encryption mechanisms into the architecture.
• Databricks & Spark Implementation: Design, implement, and manage data pipelines using Databricks Notebooks and Apache Spark for batch and real-time data processing. Ensure the architecture scales and performs optimally while maintaining data governance standards.
• Data Quality and Lineage: Work with the data engineering and analytics teams to implement strong data quality checks and ensure that data lineage is fully traceable through Purview and other tools, enabling both technical and business users to trust the data being processed and consumed.
• Collaboration & Leadership: Partner with data engineering, data science, business intelligence, and IT teams to ensure seamless integration of cloud data architecture and governance frameworks into the broader enterprise ecosystem. Lead by example and mentor junior team members on best practices related to cloud data governance and architecture.
• Automation and Optimization: Implement best practices for automating data pipeline deployments and monitoring using CI/CD pipelines. Continuously optimize data workflows and query performance while ensuring compliance with governance policies.
• Documentation & Knowledge Sharing: Document all data models, architecture, and best practices. Share knowledge with team members and stakeholders to promote a culture of learning, transparency, and continuous improvement in data governance practices. Contribute to the documentation of data governance processes.

What you must have:

• Bachelor’s or Master’s Degree in Computer Science, Information Technology, Engineering, or a related field.
• 7+ years of experience in data architecture, data engineering, or a related field, with significant hands-on experience in cloud-based solutions.
• Expertise in Azure Cloud, with significant experience in Azure Databricks, Apache Spark, Unity Catalog, and Purview for data governance and security.
• Proven experience designing and implementing data governance frameworks that ensure data is properly classified, accessible, and compliant with security and privacy regulations.
• Hands-on experience in developing and managing CI/CD pipelines for automated data workflows and deployments.
• Experience with data cataloging, metadata management, and data lineage in a cloud environment, especially using Purview.
• Preferred certifications: Azure Data Engineer Associate or Azure Solutions Architect Expert certification, Databricks Certified Associate Developer or Data Engineer certification.
• Experience influencing senior leadership and driving data strategy at the enterprise level.
• Experience in the financial services industry and familiarity with the regulatory environment governing financial data would be an asset.

Technical Skills:

• Proficiency in Python & PySpark for data engineering, scripting, and automation within Databricks Notebooks and Azure.
• Strong experience with Apache Spark for distributed data processing in both batch and streaming contexts.
• Expertise in data governance, including data classification, metadata management, access controls, and compliance management using Purview and Unity Catalog.
• Familiarity with CI/CD tools such as Azure DevOps, Git, and Jenkins for deploying and managing data pipelines and workflows.
• Solid understanding of data security frameworks and experience working in highly regulated environments such as banking or finance.

You are:

• A strong communicator: You have the strong communication and collaboration skills to engage with cross-functional teams, including data engineering, business intelligence, compliance, and security teams.
• An analytical problem-solver: You have excellent problem-solving skills, with a focus on designing scalable and compliant data solutions. You're also able to work in a dynamic, fast-paced environment and lead complex data transformation projects.
• A leader: You have the ability to mentor and guide junior team members on best practices for cloud data governance, architecture, and engineering.

Salary Range: $81,200 – $122,000/annually

For more information about TEEMA and to consider other career opportunities, please visit our website at www.teemagroup.com