Senior Data Engineer

The Data Engineer will have a knack for data analysis, data manipulation, and data modelling and will be responsible for understanding and driving the overall technical vision and planning of a client's organisation and translating business needs into technical strategy.

Required Qualifications:

• Tertiary degree, diploma or certificate in a related field (BSc Computer Science, B.IT or Informatics related degrees).

Experience and Knowledge:

• 8-10 years’ working experience as a Data Engineer / Database Developer.
• Experience in data mining, large scale data modelling, and business requirements gathering / analysis.
• Understanding and working experience in data integration and transformation.
• Experience implementing data modelling methodologies like Dimensional Modeling and / or Data Vault.
• Working knowledge of data quality processes and master data management.
• Experience implementing design support systems using Database Management Systems (DBMS) such as SQL Server or Oracle.
• Proficiency in designing and implementing data integration and ETL solutions using SSIS, Azure Data Factory and / or SQL Server stored procedures.
• Understanding of several Big Data technologies like Hadoop, MapReduce, and Spark as well as event processing or message ingestion services like Kafka, Event Hub, and Stream Analytics.
• Experience in database query languages such as T-SQL, ANSI SQL, PL / SQL.
• Some experience developing software solutions using Visual Basic, C++, C#, Java, or Python.
• Experience using SQL Server Management Studio and Visual Studio.
• Experience implementing solutions using Azure SQL databases, Azure Synapse (Previously SQL Data Warehouse), Azure Storage Accounts (Data Lake), and / or Databricks.
• Analytical mind and business acumen.
• Additional skills in the following will be taken into consideration: Tableau, Power BI, strong math skills (e.g. statistics, algebra), Scala, Python, or R.

Key Responsibilities:

• Identify valuable data sources and automate collection processes.
• Undertake preprocessing of structured and unstructured data.
• Analyze large amounts of information to discover trends and patterns.
• Data Modelling (Relational and Star Schema).
• Data Warehouse Design - Build and Development.
• Database Administration.
• Database Performance Tuning and Optimisation.
• Present information using data visualization techniques.
• Propose solutions and strategies to business challenges.
• Collaborate with engineering and product development teams.
• An understanding and hands-on experience on Hadoop / Spark based distributed storage and computing frameworks.
• Real-Time analytics and batch processing.
• Strong experience in architecting analytical applications in cloud environments such as Amazon Web Services and Microsoft Azure.