Data Engineer

Job Description: Data Engineer
Location: Dubai

Job Summary: We are looking for an experienced Data Engineer. The suitable candidate should have demonstrated experience in designing and implementing ETL/ELT solutions on-premises modern platforms to support Enterprise data warehouse, Data Mart, and advanced analytics capabilities. Success in this role comes from marrying a strong data engineering background with product and business expertise to deliver scalable data pipelines.

Role Purpose:

1. Design, implement, and maintain robust, scalable, and high-performance ETL/ELT pipelines to ingest, process, and load data from various sources.
2. Develop automated processes for deploying, testing, and managing data pipelines and orchestrate workflows using tools.
3. Deep understanding of data warehousing and data model concepts.
4. Analyze data requirements, complex source data, data models, and determine the best methods for extracting, transforming, and loading the data into the data staging warehouse and other system integration projects.
5. Analyze business requirements and outline solutions.
6. Develop and deploy ETL/ELT job workflows with reliable error/exception handling and rollback.
7. Proactively communicate innovative ideas, solutions, and capabilities over and above the specific task request.
8. Work closely with data architects, data analysts, and business stakeholders to understand data requirements and develop solutions that meet business needs.
9. Collaboratively work with a team and independently. Continuously strive for high-performing business solutions.
10. Perform and coordinate unit and system integration testing.
11. Optimize data storage and retrieval to enhance performance and efficiency. Improve query performance and reduce latency for analytics and reporting.
12. Document data pipelines, architecture, and processes. Establish and promote best practices for data engineering within the organization.

Requirements

Key Requirements and Qualifications:

1. Minimum 6 years of data engineering relevant experience.
2. Proficiency in Advanced SQL and experience with any ETL/ELT tools.
3. Knowledge of data modeling, data warehousing, and data lake architecture.
4. Hands-on with HTAB databases like Single Store.
5. Familiarity with orchestration tools like Stone Branch or similar workflow management tools.
6. Experience in the healthcare/insurance industry is preferred.
7. Bachelor's degree in computer science or any related field with programming exposure.

Skills and Competencies:

1. Strong analytical and problem-solving skills with attention to detail.
2. Experience with CI/CD tools like Jenkins and Bitbucket.
3. Experience in developing flows using batch, real-time, and streaming processes using PySQL/PySpark preferred.
4. Experience in data streaming technology like Oracle Golden Gate or similar.
5. Ability to work with multiple projects and work streams at one time. Must be able to deliver results based upon project deadlines.
6. Strong interpersonal and communication skills.