If you are interested kindly send the following documents to hrsmss@smsoftconsulting.com at your earliest convenience.
Job Title: ETL Developer
Client: One of the Ministry in Ontario
Work Location: Toronto, Ontario, Hybrid
Hours per day or Week: 7.25 hours per day
Security Level: CRJMC
Must haves
- Design, Develop and Implement data pipelines using Azure Data Factory (ADF), stored procedure, and Informatica PowerCenter performing extraction, transformation, and loading activities.
- Develop Azure CI/CD pipeline to automate ADF release.
- Extensive experience in design, development, and implementation with Azure Data Factory, Databricks, Informatica including - Data movement, ETL pipelines.
- Experience with CI/CD (DevOps) pipelines and concepts, including Azure Resource Management, GitHub Repo, Source code version control.
- Extensive experience in designing and developing SQL Server Stored Procedure, Oracle PL/SQL programs.
- Knowledge of Databases -Oracle, Microsoft SQL Server database and tools.
Description
Deliverables section:
- Scope of Services and Deliverables - The Services and Deliverables to be provided by the Vendor will include the following:
- Participate in the review of project, work request and maintenance release artefacts - Use Case, System Requirement Specification, etc.
- Deliver detailed system design documents and any other supporting documentation conforming to Ministry standards.
- Design logical and physical data models using Sybase Power Designer.
- Produce transformation mapping document.
- Implement data pipelines using Azure Data Factory (ADF), stored procedure, and Informatica PowerCenter performing extraction, transformation, and loading activities.
- Implement solutions using Logic App and Function App.
- Generate structured JSON file using ADF/ SQL.
- Develop Azure CI/CD pipeline to automate ADF release.
- Develop complex Oracle PL/SQL program to fulfill project requirements.
- Implement complex data conversion, e.g., binary to character, EBCDIC to UTF.
- Implement complex data transformation(s) for derived and calculated values.
- Integrate data sets from diverse source systems.
- Implement ADF for initial data load and incremental load.
- Promote ADF and Informatica ETL/ELT through all Ministry environments including Development, Integration Testing, QA, UAT, Production.
- Resolve and troubleshoot ADF pipeline, stored procedure, and Informatica workflow technical problems.
- Monitor and analyze incident, provide timely resolution.
- Collaborate with IT Professionals throughout the Software Development Life Cycle.
- Document information from diverse business area stakeholders and subject matter experts.
- Optimize performance of ADF pipeline, Azure SQL and Synapse databases, and Informatica workflow.
- Monitor application functionality and performance on daily basis.
- Provide effective knowledge transfer to Ministry staff at all stages of this assignment.
Experience and Skill Set Requirements
ETL Development-Data Integration -40%
- Extensive experience in gathering requirements and business process knowledge in order to design correct and high-quality data transformation.
- Extensive experience in design, development, and implementation with Azure Data Factory, Databricks, including:
- Self-Hosted Integration Runtime.
- Data movement, ETL pipelines.
- Performance tuning on large data volumes.
- Complex data transformations.
- ADF Debugger to validate data transformation.
- Azure Roles.
- Experience with Informatica, including:
- Workflow Manager.
- Repository Manager.
- Designer.
- Workflow Monitor.
- Experience with developing solutions using Logic App and Function App.
- Experience in rapid application development (RAD) methodologies.
- Experience in maintaining and improving existing ETL processes.
- Experience in investigating data to identify potential issues within ETL pipelines.
- Experience with Business Intelligence tools IBM Cognos and Microsoft PowerBI.
Azure Technologies -40%
- Experience with CI/CD (DevOps) pipelines and concepts, including:
- Azure Resource Management.
- GitHub Repo.
- Source code version control.
- Branching, pull requests, build, release, multiple parallel development repositories merge.
- Experience with Azure DevOps, including:
- Azure Boards.
- Azure Test Plans.
- Azure Pipelines.
- Azure Repos.
- Azure Artefacts.
Oracle PL/SQL Development- 10%
- Extensive experience in designing and developing SQL Server Stored Procedure, Oracle PL/SQL programs.
Database Technologies- 10%
- Experience with Oracle, Microsoft SQL Server database and tools.
- Experience with relational and hierarchical database technologies.
- Extensive knowledge of Azure SQL DB and Synapse platforms, including:
- Performance tuning on large data volume.
Note:
This position is currently listed as Hybrid; however, the consultant will be remote due to COVID-19 related Work from Home (WFH) direction. Once the I&IT cluster staff are required to return to the office, the resource under this request will be required to work onsite as well.
- Hybrid: 3 days onsite / 2 days remote.