Design and develop ETL Processes: Create and maintain ETL pipelines to extract, transform, and load data from multiple sources into target systems.
Data Transformation: Clean, transform, and format data to meet business requirements and ensure consistency.
API Development: Design, develop, and manage APIs to facilitate data extraction, sharing, and exchange from various platforms and third-party services.
Data Quality Assurance: Monitor data quality and integrity, troubleshoot issues, and implement solutions to maintain high data standards.
Testing and Debugging: Ensure reliability and quality of the ETL process and API. This includes unit, integration, and end-to-end testing.
Performance Optimization: Optimize ETL/API processes for efficiency and scalability, ensuring minimal downtime and maximum performance.
Documentation: Maintain comprehensive documentation of ETL/API processes, data mappings, and API integrations.
Collaboration: Work closely with data engineers, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs.
Support Training and API/ETL Level 2 Support: Provide training and support to level 1 API/ETL production support, ensuring they can effectively use the API, the API management platform, and the ETL pipeline.
Troubleshooting: Promptly identify and resolve ETL pipeline and API integration issues.
Requirement
Minimum 3 years of experience as an API or ETL developer
Strong knowledge of SQL and experience with ETL tools such as Informatica, SSIS, Talend, and Pentaho; API management tools such as Postman, MuleSoft, or Insomnia; and web technologies, protocols, and knowledge of SQL and NoSQL database and how they relate to API development.
Proven experience developing ETL pipelines, APIs, web services, and integrating applications systems.
Excellent analytical and problem-solving skills to troubleshoot and resolve data issues.
Ability to work collaboratively in a team environment.