Senior Data Architect

This range is provided by Cititec Talent. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

Base pay range

Our client, a leading commodities trading firm, is going through a huge technology and data transformation and is seeking an experienced Senior Data Architect to design and implement a greenfield data ecosystem across on-premise and cloud for trading technology.

Key Responsibilities:

1. Data Lake Design & Architecture
2. Design and implement a scalable data lake using Apache Iceberg for efficient data storage and processing.
3. Architect hybrid on-premise and cloud database solutions that optimise performance and cost.
4. Define data partitioning, schema evolution, and versioning strategies to enable efficient query execution.

Data Ingestion & Processing:

1. Implement high-performance data ingestion pipelines for structured and unstructured data.
2. Utilise Apache Arrow Flight for optimised in-memory data transport and real-time analytics.
3. Work with technologies like Kafka, Spark, and Airflow for streaming and batch processing.

Data Governance & Quality:

1. Establish data quality frameworks to ensure accuracy, completeness, and consistency.
2. Implement metadata management, lineage tracking, and audit logging.
3. Define security and compliance policies for sensitive financial and trading data.
4. Leverage cloud-native services (AWS/GCP/Azure) and on-premise databases to maximise efficiency.
5. Collaborate with DevOps teams to enable CI/CD for data pipelines.

Key Skills & Experience:

1. Strong experience in Data Architecture for large-scale financial or commodities trading environments.
2. Deep understanding of Apache Iceberg (table format for big data).
3. Expertise in Apache Arrow & Arrow Flight for high-speed in-memory data transfer.
4. Hands-on experience with cloud (AWS/GCP/Azure) and on-prem databases (PostgreSQL, MySQL, Oracle, etc.).
5. Strong background in data ingestion frameworks (Kafka, Flink, Spark, Airflow).
6. Knowledge of data governance, security, and regulatory compliance.
7. Proficiency in Python, SQL, and distributed data processing frameworks.
8. Previous experience in commodities trading or capital markets.

Preferred Technologies & Tools:

1. Big Data & Storage: Apache Iceberg, Delta Lake, Hadoop, Parquet, ORC
2. Data Processing: Apache Spark, Flink, Dask
3. Data Quality & Governance: Great Expectations, dbt, Collibra

Seniority level

Director

Employment type

Full-time

Job function

Information Technology

Industries

Oil and Gas, Investment Management, and Investment Banking