Data Engineer

We are seeking a talented and driven Data Engineer to join our team. In this role, you will be responsible for building, maintaining, and optimizing data pipelines, utilizing Airflow and ETL processes, while managing infrastructure on Google Cloud Platform (GCP). You will collaborate closely with the data team to ensure the quality, consistency, and security of data, and play a key role in supporting data-driven decisions within the organization.

Responsibilities:

• Build, maintain, and optimize data pipelines using Airflow and ETL processes to ensure data flows seamlessly across systems.
• Manage and monitor the infrastructure on Google Cloud Platform (GCP) to ensure scalability and reliability of data systems.
• Ensure the quality, consistency, and integrity of data across platforms and systems.
• Manage and optimize Kafka for real-time data streaming, ensuring low-latency data processing.
• Implement security measures and ensure compliance with data privacy regulations and industry standards.
• Collaborate with the data team to design and create dashboards using Redash to present key data insights.
• Document data architecture, pipeline workflows, and processes to ensure clarity and maintainability.

Requirements:

• Proven experience as a Data Engineer or in a similar data-related role.
• Strong knowledge of data pipeline frameworks such as Airflow and ETL processes.
• Experience with Google Cloud Platform (GCP) services such as BigQuery, Cloud Storage, etc.
• Proficiency in working with Kafka for real-time data streaming and processing.
• Strong understanding of data quality, data consistency, and data security practices.
• Experience with dashboard tools such as Redash or similar business intelligence tools.
• Solid understanding of database management and data storage principles.
• Strong problem-solving and analytical skills.
• Ability to collaborate effectively with cross-functional teams.