IC4 - Sr Data Engineer

Objective of the Role

Leads the design, development, and optimization of advanced data solutions, ensuring scalability, reliability, and alignment with business objectives. This role plays a critical part in defining data architectures, mentoring team members, and driving the adoption of best practices throughout the data engineering lifecycle. With a strong focus on performance, security, and governance, ensures that data solutions contribute to a data-driven culture and deliver measurable business value.

Main Responsibilities

• Lead the design and implementation of robust and scalable data pipelines, enabling efficient ingestion, processing, and storage of structured, semi-structured, and unstructured data.
• Collaborate with cross-functional teams to gather requirements, define technical solutions, and align data architectures with business needs.
• Develop and optimize data models using industry-standard techniques to support analytics, reporting, and AI-driven solutions.
• Implement advanced data processing workflows, including batch, microbatch, near real-time, and real-time pipelines, to ensure seamless data availability and performance.
• Drive the adoption of data governance and privacy strategies, ensuring compliance with security and regulatory requirements throughout the data lifecycle.
• Design and maintain data architectures, contributing to the scalability and reliability of data lakes, warehouses, and real-time streaming platforms.
• Develop monitoring processes and data quality metrics to ensure the reliability, integrity, and completeness of data used for decision-making.
• Actively contribute to all phases of the data engineering lifecycle, adhering to Agile methodologies and ensuring timely delivery of high-quality solutions.
• Mentor and guide junior and mid-level engineers, fostering a collaborative and inclusive team environment.
• Maintain and document data architectures, processes, and technical workflows to ensure continuity and knowledge sharing.
• Lead the integration of APIs and external data sources, optimizing data flows and ensuring compatibility with existing architectures.
• Manage version control and repository practices to maintain clean, reliable, and maintainable codebases.
• Promote an autonomous work culture by encouraging self-management, accountability, and proactive problem-solving among team members.
• Serve as a Spin Culture Ambassador to foster and maintain a positive, inclusive, and dynamic work environment that aligns with the company's values and culture.

Required Knowledge and Experience

• Minimum 5 - 6 years of experience in data engineering or related fields, with proven expertise in designing and implementing scalable data solutions.
• Advanced technical expertise in software development life cycle (SDLC) methodologies.
• Advanced understanding of design patterns.
• Able to work autonomously and provide support to less qualified professionals.
• Advanced experience in the use of Python, Java, and data related frameworks.
• Advanced knowledge of dimensional data modeling and advanced techniques in this field, including structured, semi-structured, and unstructured data storage (non-SQL), as well as ETL, ELT construction.
• Advanced knowledge in file processing: CSV, JSON, Parquet, Yaml, images, and videos.
• Advanced knowledge in data processing: batch, microbatch, near real-time, and real-time.
• Advanced understanding and experience to contribute and implement data privacy strategies.
• Advanced understanding and experience in implementing and maintaining robust data governance strategies.
• Advanced knowledge in Data Engineering Lifecycle (also known as Data Processing Lifecycle).
• Advanced knowledge in processing different types of internal and external data sources with API management, for subsequent processing, storage, and analysis.
• Advanced knowledge in understanding Architecture: Business, Solutions, Data, etc.
• Advanced mastery in version control: Git, Github, Gitlab.
• Advanced in business vision (business domain), to sensitively identify how data contributes to the business.
• Advanced mastery and evidence with projects using the Google Cloud Platform (GCP) Stack for Data.
• Advanced mastery and evidence with projects using the Amazon Web Services (AWS) Stack for Data.
• Advanced experience in developing and optimizing SQL and related queries.
• Able to effectively and exceptionally socialize the design of the data solution without supervision.
• Advanced experience in Project Management with Agile methodology (Scrum or Kanban).
• Experience in collaborating on projects to achieve OKRs, warning of risks, and providing business value.
• Advanced competence in transparently communicating project status.