Data Engineer

The Data Engineer intern will be participating in exciting projects covering the end-to-end data lifecycle - from raw data integrations with primary and third-party systems, through advanced data modelling, to state-of-the-art data visualisation and development of innovative data products.

You will have the opportunity to learn how to build and work with both batch and real-time data processing pipelines. You will work in a modern cloud-based data warehousing environment alongside a team of diverse, intense and interesting co-workers. You will liaise with other departments - such as product & tech, the core business verticals, trust & safety, finance and others - to enable them to be successful.

Key Responsibilities Include:

1. Raw data integrations with primary and third-party systems
2. Data warehouse modelling for operational and application data layers
3. Development in Amazon Redshift cluster
4. SQL development as part of agile team workflow
5. ETL design and implementation in Matillion ETL
6. Design and implementation of data products enabling data-driven features or business solutions
7. Data quality, system stability and security
8. Coding standards in SQL, Python, ETL design
9. Building data dashboards and advanced visualisations in Periscope Data with a focus on UX, simplicity and usability
10. Working with other departments on data products - i.e. product & technology, marketing & growth, finance, core business, advertising and others
11. Being part and contributing towards a strong team culture and ambition to be on the cutting edge of big data

Requirements

• Bachelor's degree in computer science, engineering, math, physics or any related quantitative field
• Knowledge of relational and dimensional data models
• Knowledge of terminal operations and Linux workflows
• Ability to communicate insights and findings to a non-technical audience
• Good SQL skills across a variety of relational data warehousing technologies especially in cloud data warehousing (e.g. Amazon Redshift, Google BigQuery, Snowflake, Vertica, etc.)
• Attention to details and analytical thinking
• Entrepreneurial spirit and ability to think creatively; highly-driven and self-motivated; strong curiosity and strive for continuous learning

Desired Candidate Profile

1. Data Architecture and Infrastructure
   • Designing Data Pipelines: Develop, construct, and maintain efficient data pipelines that enable the movement and transformation of large datasets between various systems and storage solutions.
   • Building Data Warehouses: Create data storage solutions like data warehouses or data lakes that allow easy access, retrieval, and analysis of data from various sources (e.g., transactional databases, cloud platforms).
   • Database Design and Optimization: Design databases, ensuring that they are scalable, secure, and optimized for both performance and storage.
2. Data Collection and Integration
   • Data Integration: Integrate data from a variety of sources such as APIs, databases, flat files, cloud storage, and real-time data streams into centralized systems.
   • ETL Processes (Extract, Transform, Load): Develop and maintain ETL processes to clean, transform, and load raw data into usable formats for analytics.
   • Data Governance: Ensure that data is accurate, consistent, and secure by enforcing data governance practices, and maintaining data quality standards.
3. Data Transformation and Processing
   • Data Cleaning: Process and clean raw data to remove inconsistencies, errors, or duplicates, ensuring that the data used for analysis is reliable and of high quality.
   • Data Transformation: Transform data into a structured format suitable for analysis, reporting, and further processing by data scientists and analysts.
4. Performance and Scalability
   • Optimization: Continuously monitor, optimize, and troubleshoot data pipelines and storage solutions to ensure they perform efficiently at scale, especially as data volumes grow.
   • Automation: Automate repetitive tasks like data loading and monitoring to reduce manual effort and improve the efficiency of data processing.
5. Collaboration with Data Scientists and Analysts
   • Collaborate on Data Needs: Work closely with data scientists and analysts to understand their data requirements and provide them with clean, organized, and ready-to-use datasets.
   • Provide Data Access: Ensure that analysts and other users can easily access and query the data they need by setting up efficient querying tools and user interfaces.
6. Cloud Platforms and Big Data
   • Cloud Solutions: Leverage cloud-based platforms (e.g., AWS, Google Cloud Platform, Microsoft Azure) for scalable data storage and computing resources.
   • Big Data Technologies: Implement and manage big data technologies (e.g., Hadoop, Spark, Kafka) to process and analyze large datasets across distributed systems.