Data Engineer

Job Responsibilities

1. Create and develop data ingestion from various sources: RDBMS, Rest API, Kafka, Text File, and Spreadsheet. Design, develop, optimize, and maintain data architecture and pipelines.
2. Work with Core Data Engineering / Data Warehousing team to utilize existing frameworks for the implementation of these data pipelines.
3. Drive the prioritization, strategy, and focus to solve user problems.
4. Maintain and optimize data pipelines.
5. Participate in code reviews and follow best practices for development and documentation of data pipelines.
6. Continuously learn and adapt to new technologies and methodologies within the data engineering landscape.

Qualifications

1. Experience as a Data Engineer or Data Analyst is preferable.
2. Excellent command of programming languages, preferably in Python.
3. Familiarity with managing a serverless data warehouse like BigQuery or Redshift.
4. Familiarity with schedulers like Airflow & Airbyte.
5. Experience working with GitHub and Docker.
6. Deep knowledge of SQL database design (MySQL, Redshift, PostgreSQL).
7. Understanding of how to optimize data retrieval and develop dashboards, reports, and other visualizations for stakeholders.
8. Good communication skills to work across departments.