Lead Data Engineer
- Conceptualize, maintain and improve the data architecture
- Evaluating design and operational cost-benefit trade-offs within systems
- Design, build, and launch collections of data models that support multiple use cases across different products or domains
- Solve our most challenging data integration problems, optimising ELT pipelines, frameworks, sourcing from structured and unstructured data sources
- Implementing CI/CD frameworks
- Create and contribute to frameworks that improve the accuracy, efficiency and general data integrity
- Design and execute 'best-in-class' schema design
- Implementing other potential data tools
- Define and manage refresh schedules, load-balancing and SLA for all data sets in allocated areas of ownership
- Collaborate with engineers, product managers, and data analysts to understand data needs, identify and resolve issues and help set best practices for efficient data capture
- Determine and implement the data governance model and processes within ownership realm (GDPR, PPI, etc)
- Taking ownership of the data engineering process - from project scoping, design, communication, execution and conclusion
- Support and strengthen data infrastructure together with data team and engineering
- Support organisation in understanding the importance of data and advocate for best-in-class infrastructure
- Mentoring, educating team members on best-in-class DE practices
- Priorising workload effectively
- Support quarterly and half-year planning from Data Engineering perspective
- University degree; ideally in data engineering, software engineering, computer science-engineering, numerate or similar
- +7 years of data engineering experience or equivalent
- Expert experience building data warehouses and ETL pipelines
- Expert experience of SQL, python, git, dbt (incl. query efficiency and optimization)
- Expert experience of Cloud Data Platforms (AWS, Snowflake and/or Databricks) → Qualification preferred, not mandatory
- Significant experience of Automation and Integrations tools (FiveTran, Airflow, Astronomer or similar)
- Significant experience with IoC tools (Terraform, Docker, Kubernetes or similar)
- Significant experience with CI/CD tools (Jenkins, GitHub Actions, CircleCI or similar)
- Experience with real time data pipelines (AWS Kinesis, Kafka, Spark)
- Experience with observability tools (Metaplane, MonteCarlo, Datadog or similar)
- Experience within FinTech/Finance/FX preferred
Recommended Jobs
Hollister Co. - Assistant Manager, Foubert's Place - Soho
Abercrombie & Fitch Co. is a global, digitally led omnichannel specialty retailer of apparel and accessories catering to kids through millennials with assortments curated for their specific lifesty...
Commercial Bank Global Business Execution Partner - VP
Citi’s Commercial Bank (“CCB”) is a rapidly growing business, targeting emerging, mid- and larger-sized companies with annual revenue generally up to $3 billion. We are looking for a dynamic individ...
Temporary Butlers
Our client, with a residence in St James’s Park, London, is looking for Temporary Butlers to work at a Formal Household. The property is a large operation with over 15 members of staff jointly ensurin...
Lead Mechanical Engineer - HVAC / Data Centres Job Details | Black & Veatch Family of Companies
Lead Mechanical Engineer - HVAC / Data Centres Date: May 29, 2025 Location: Glasgow, GB London, GT LON, GB Company: Black & Veatch Family of Companies Job Summary The Team Black...
Band 7/8 - Oncology Pharmacist - Chelsea
Job Title: Oncology Pharmacist Banding: Band 7/8 Location: Chelsea, Greater London Hours: 37.5 hours per week Start Date: ASAP Rate: £30 – £43 per hour Duration: End of June ...
Live-out Housekeeper
Formal household in Regent’s Park. 4-storey, 5-bedroom home and in addition, maintaining three mews properties. The family comprises a couple, their two children, and their dog. On occasion, ext...
Consultant - Cost Assurance
Turner & Townsend is a global professional services company with over 22,000 people in more than 60 countries. Working with our clients across real estate, infrastructure, energy and natural r...
Supplier Technical Manager
Supplier Technical Manager PR/013075 £45,000 - £55,000 West London The Role Reporting directly to the Head of Technical, the candidate will have strong communication and interpersonal skills...
Director, EMEA Channel - UK
Description At Gigamon, our purpose is to protect the hybrid networks and data of the largest, most complex organizations on the planet. Certified as a Great Place to Work, we offer a deep observab...
Trainee Recruitment Consultant - Supportive colleagues!
Trainee Recruitment Consultant Wayman Education is looking for driven and enthusiastic Recruitment Consultants to join a rapidly expanding recruitment company. We want to be the premier nat...