Senior Data Engineer

Senior Data Engineer

Job Overview:

We are seeking an experienced and highly skilled Senior Data Engineer with full-stack expertise to join our dynamic Data Engineering team. You will play a crucial role in designing, building, and enhancing web-based data applications. This hands-on engineering role will focus on aggregating data from various sources, processing and transforming it, and ensuring it is delivered to internal and external stakeholders. Your work will directly impact the scalability and effectiveness of our data infrastructure and proprietary analytical tools.

Key Responsibilities:

• Translate business needs into robust, scalable web solutions by designing and developing APIs, front-end interfaces, and tools for real-time data ingestion and transformation.
• Leverage a mix of open-source and proprietary big data technologies to improve data reliability, performance, and overall quality.
• Collaborate with internal teams and stakeholders to ensure best practices in data governance and ensure the integrity of data systems during implementation.
• Act as a technical expert on emerging big data technologies, providing insights and recommendations for continuous improvement.
• Work on large-scale commercial software systems and databases, applying your deep knowledge of Business Intelligence, Analytics, and Big Data tools.
• Develop in production environments using Java, Scala, and Spark programming languages.
• Ensure proficiency in SQL, along with scripting expertise in languages such as Python, Shell, etc.
• Work with big data processing frameworks such as MapReduce, Spark, HBase, and NoSQL.
• Apply your expertise in data management technologies to process and analyze large volumes of data efficiently.
• Contribute to both back-end and web UI development, ensuring seamless integration and performance.
• Provide clear, analytical insights and problem-solving solutions to complex challenges.
• Actively participate in a collaborative team environment, efficiently managing multiple tasks and delivering on time.

Requirements:

• A minimum of 7 years of proven experience in software development, specifically in building large-scale commercial software systems and database platforms.
• Must pass technical proficiency assessments as part of the recruitment process.
• Expertise in Business Intelligence, Analytics, and Big Data processes; experience in web, video, mobile, or adtech domains is a strong advantage.
• Proficiency in Java, Scala, and Spark programming languages.
• Strong SQL skills are essential.
• Solid scripting knowledge in languages such as Python, Shell, or similar.
• Experience with big data processing frameworks like MapReduce, Spark, HBase, and NoSQL.
• Expert knowledge of data management technologies and the software engineering tools needed to handle large-scale data processing.
• Background in Web UI, middle-tier, and backend development.
• A strong analytical mindset with the ability to think critically and provide actionable solutions.
• A team player with the ability to handle multiple tasks simultaneously and meet deadlines.
• A degree in Computer Science, Engineering, Statistics, Mathematics, or a related field is required.