Data Engineer II

• Produce reliable data for stakeholders by reviewing collected data APIs, troubleshooting, and processing data into the datamart.
• Propose and collect requirements from users for data pipeline creation, sourcing data based on needs, and establishing data pipelines, cataloging, and formatting.
• Execute the creation of machine learning pipelines, gathering user requirements, sourcing data, developing pipelines, and cataloging destinations for formatted data.
• Develop APIs by collecting requirements, sourcing relevant data, creating pipelines, and cataloging and formatting data accordingly.
• Improve structured planning for model deployment through research, testing, and documenting technical specifications (TSD).
• Maintain discussions with stakeholders, provide documentation, and collect feedback.
• Optimize and implement integrations effectively based on stakeholder feedback and testing results.
• Design, build, and manage scalable data pipelines and ETL processes.
• Develop and maintain the data infrastructure (e.g., data warehouses, data lakes).
• Monitor and maintain data pipelines to ensure real-time and batch data processing.

Requirements

• At least have 3+ years experience in the same position
• Strong experience with SQL and relational databases (e.g., MySQL, PostgreSQL).
• Experience with ETL tools and data pipeline frameworks (e.g., Apache Airflow, AWS Glue).
• Proficiency in a programming language such as Python, Java, or Scala.
• Experience with cloud platforms (AWS, Azure, GCP) and related data services.
• Experience with version control (e.g., Git) and CI/CD pipelines.
• Strong problem-solving and troubleshooting skills.
• Familiarity with data governance, security, and privacy principles.

At Hypefast, we are committed to creating a diverse and inclusive workplace. We believe that diverse perspectives drive innovation and excellence. We welcome candidates from all backgrounds and employment is decided on the basis of qualifications, merit, and business need.