Data Engineer

We are seeking a skilled Data Engineer to design, implement, and maintain robust data pipelines, ensuring seamless data integration, storage optimization, and high data quality. The ideal candidate will work closely with senior data engineers to develop efficient data solutions, optimize performance, and contribute to innovations that enhance data reliability and accessibility.

Key Responsibilities

1. Design, develop, and maintain scalable ETL/ELT pipelines to facilitate seamless data extraction, transformation, and loading from diverse sources.
2. Implement and execute architectural planning for data solutions, ensuring smooth integration of data products and applications.
3. Ensure data validation and quality assurance, proactively identifying and rectifying data anomalies, inconsistencies, and errors.
4. Develop and optimize data models for both transactional and reporting use cases, refining structures for improved performance.
5. Manage and optimize data storage systems, including databases, data warehouses, and data applications to support business needs.
6. Contribute to internal innovations by developing data tools and products to streamline workflows and enhance data engineering processes.
7. Monitor system performance, identify bottlenecks, and implement solutions to improve data retrieval and storage efficiency.
8. Stay updated on emerging data engineering tools and technologies, evaluating new solutions to enhance productivity.
9. Troubleshoot and resolve technical issues related to data pipelines, databases, and integration processes.
10. Maintain comprehensive documentation for data pipelines, processes, and procedures to support knowledge sharing and troubleshooting.
11. Collaborate with security teams to ensure data solutions comply with organizational and regulatory standards.
12. Engage with data consumers to facilitate effective data usage, gather feedback, and drive continuous improvement.

Required Qualifications & Skills

1. Bachelors or Masters degree in Computer Science, Data Engineering, Information Systems, or a related field.
2. Proven experience in data pipeline development, data integration, and ETL/ELT processes.
3. Proficiency in SQL, Python, or Scala for data processing and manipulation.
4. Experience with data modeling and database design for both transactional and analytical use cases.
5. Hands-on experience with big data technologies (Hadoop, Spark, Kafka) and cloud-based data platforms (AWS, Azure, or GCP).
6. Strong knowledge of data warehousing concepts and relational databases (PostgreSQL, MySQL, SQL Server, etc.).
7. Experience with data orchestration tools (Apache Airflow, Prefect, or similar).
8. Familiarity with CI/CD pipelines and DevOps practices in data engineering.
9. Strong problem-solving skills with the ability to analyze and resolve technical data challenges.
10. Excellent communication and documentation skills to collaborate with cross-functional teams.

Preferred Qualifications

1. Experience with data governance and security best practices.
2. Knowledge of streaming data technologies (Kafka, Flink, etc.).
3. Exposure to machine learning model deployment in data pipelines.