Senior Data Engineer (Datawarehouse)

Job Purpose:
As a Senior Data Engineer, you will lead efforts to manage, secure, and transform data across various systems into a centralized data warehouse. This role involves ensuring data accuracy, building and optimizing datasets, developing APIs, and providing data-driven insights to stakeholders. You will also be responsible for overseeing integration efforts and automating processes to enhance data accessibility and reliability.

Key Responsibilities:

• Oversee the management, backup, and security of data from multiple sources, ensuring seamless integration into the data warehouse.
• Design, develop, and maintain APIs to enable efficient access for business users.
• Lead data integration projects with internal and third-party systems.
• Build, manage, and optimize datasets tailored to business needs and analytics requirements.
• Validate data integrity and ensure accuracy across all data sources.
• Generate reports and insights for stakeholders based on analytical requirements.
• Automate data workflows and improve data processing efficiency.
• Mentor junior team members and support ad hoc tasks as necessary.

Qualifications:

Required Skills and Experience

• Experience: Minimum of 3 years in a similar role with relevant responsibilities or as a backend developer.
• Programming Languages: Proficiency with at least one of the following, with a minimum of 3 years of experience:
  • Perl (Preferred)
  • PHP
  • Python
• Database Proficiency: Strong experience with querying and optimizing:
  • PostgreSQL
  • MySQL
  • MSSQL
• Systems Experience: Minimum 3 years working with Linux-based systems (Debian).
• Data Warehouse & ETL: Solid experience with Google BigQuery, including data partitioning, clustering, and indexing.
• ETL Tools: Proficiency with ETL and data transformation tools (e.g., Pentaho, Apache NiFi, or similar tools).
• API & Authentication Knowledge: Expertise with REST API and JWT.
• Soft Skills: Strong communication and problem-solving skills; must be willing to work on-site.

Nice to Have

• Advanced knowledge of Google Cloud Platform (GCP) infrastructure and Google APIs.
• Experience with data analytics, data management, and BI tools.
• Background with Vertex AI.
• Familiarity with Open API and OData Service administration.
• Experience with Tableau.
• Advanced knowledge of BigQuery optimization techniques.