BIG-DATA-ENGINEER

Job Description

Must have Skills: Azure Data Factory, Azure Table storage

We are seeking a skilled Data Engineer with a minimum of 8+ years of experience to join our team on-site in Abu Dhabi. The ideal candidate will have a strong background in batch data processing, developing and maintaining scalable ETL/ELT pipelines, and working with cloud platforms like Azure. This role focuses on building efficient data solutions that ensure high performance and data quality.

Key Responsibilities:

1. Design, develop, and maintain scalable batch ETL/ELT pipelines using Azure Data Factory and other Azure services.
2. Collaborate with data architects and cross-functional teams to translate business requirements into technical solutions.
3. Ensure data processing pipelines are optimized for performance, reliability, and cost-effectiveness.
4. Perform data integration, cleansing, and validation to ensure high data quality.
5. Monitor and troubleshoot batch processing jobs to ensure timely and accurate data delivery.
6. Ensure compliance with data governance and security standards.

Qualifications:

1. Minimum of 8+ years of experience in Data Engineering, focusing on batch data processing.
2. Hands-on experience with Azure data services (Azure Data Factory, Data Lake, etc.).
3. Strong understanding of ETL processes, data integration, and data modeling.
4. Experience with data storage solutions such as Azure Data Lake and SQL databases.
5. Ability to work on-site in Abu Dhabi and collaborate with local teams.
6. Strong problem-solving and communication skills.