Sr. Azure Data Developer

3 days ago Be among the first 25 applicants

Get AI-powered advice on this job and more exclusive features.

Direct message the job poster from Themesoft Inc.

Talent Acquisition Manager at Themesoft Inc.

Location: CALGARY, ALBERTA CANADA

Skills:

Required skills:

• Python - Strong proficiency, unit testing expertise and good knowledge of packages for data, like Pandas, SQLAlchemy and Alembic.
• SQL - strong proficiency in DDL and DML, including window function (ex.: lag), CTEs, sub-queries, joins, optimization, and performance profiling. Know how SQL behaves in different platforms, like Spark, Postgresql, etc…
• Spark - Pyspark, SparkSQL, Batch and Streaming processing, partitioning/liquid clustering, delta tables, parquet
• Databricks - Workflows/Jobs, Clusters, SQL Warehouse, Unity Catalog, Performance profiling, log analysis.
• Azure EventHubs or a similar streaming solution understand how to best consume data from streamings for aggregations and parallelization/scaling.
• Postgresql - Queries, indexes(different types of indexes), performance profiling, JSON columns.
• Data Modeling: Dimensional Modeling (and experience with this model being used in BI tools), Normal Forms.
• Containerization (ex.: Docker)
• Familiar with Azure data services
• Infra as code/ CICD tools: Kubernetes, Argo, Cross Plane, Terraform or similar.
• Migrations - versioning for database, for example, Alembic (preferred), Flyway, Liquibase, etc.…
• SQL and NoSQL databases, and selecting the best fit for different use cases.
• Logging and being able to query logs using KQL(Azure) or similar.

Desired skills:

• SQL Server - Queries, indexes, performance analysis, table/index options and maintenance.
• DBT with Databricks
• PowerBI - Semantic models
• MLFlow - track experiments, register models

Qualifications and experience:

• Computer Science degree or equivalent education and experience
• Professional experience with data ingestion, ETL, and ELT for structured and unstructured data.
• Strong proficiency in Python (especially data packages like pandas, numpy, etc) and SQL for analytics, database development, and data modelling
• Experience with DevOps and CICD for data.
• Experience working with one or more cloud platforms to implement data-intensive applications, Azure is preferred.
• Strong understanding of Agile methodologies and experience on an agile team
• Keen collaborator and clear communicator, responsive to service needs and operational demands
• Passionate about the impact of emerging technologies and learning new skills and ideas
• Self-starter who embraces a culture of transparency and trust and thrives in an environment of rapid change and growth
• Experience in solving problems quickly and effectively when faced with business impact.
• Applying and guiding the team in best practices and the best way to achieve the expected results and keeping maintainability of the code base.
• Ability to focus on the value delivered and keep the team on the right track to achieve the desired value.

Responsibilities:

• Support, maintain, optimize and create ETL/ELT pipelines, both batch and streaming, in Databricks(PySpark, DatabricksSQL), Python, SQL and/or DBT.
• Build, Design and Model data objects.
• Proficient in Dimensional Modeling and Database normalization (Normal Form)
• Learns the business rules and why they matter, understands the business needs.
• Write and perform tests for data flow
• Work closely with solution owners to deliver the needed data accurately
• Work with a team with different skill sets and roles (back and front-end developers, data scientists, business specialists, DevOps, etc…) to deliver the best solution possible
• Collaborate with Platform teams to use the available infrastructure as efficiently as possible, including on the CICD deployment.
• Participate in planning, estimates, and other necessary work to prepare the project plan and report on its progress.
• Keen collaborator and clear communicator, responsive to the needs and operational demands of the solutions.
• Do code reviews, train the team in best practices, leading by example.
• Support/solve production issues and post measures to avoid the problem.
• Helped unblock other team members, keep the team pace, and keep the team effective in delivering value to the business.