Data Engineer
- Conceptualize, maintain and improve the data architecture
- Evaluating design and operational cost-benefit trade-offs within systems
- Design, build, and launch collections of data models that support multiple use cases across different products or domains
- Solve our most challenging data integration problems, optimising ELT pipelines, frameworks, sourcing from structured and unstructured data sources
- Implementing CI/CD frameworks
- Create and contribute to frameworks that improve the accuracy, efficiency and general data integrity
- Design and execute 'best-in-class' schema design
- Implementing other potential data tools
- Define and manage refresh schedules, load-balancing and SLA for all data sets in allocated areas of ownership
- Collaborate with engineers, product managers, and data analysts to understand data needs, identify and resolve issues and help set best practices for efficient data capture
- Determine and implement the data governance model and processes within ownership realm (GDPR, PPI, etc)
- Taking ownership of the data engineering process - from project scoping, design, communication, execution and conclusion
- Support and strengthen data infrastructure together with data team and engineering
- Support organisation in understanding the importance of data and advocate for best-in-class infrastructure
- Mentoring, educating team members on best-in-class DE practices
- Prioritising workload effectively
- Support quarterly and half-year planning from Data Engineering perspective
- University degree; ideally in data engineering, software engineering, computer science-engineering, numerate or similar
- +4 years of data engineering experience or equivalent
- Expert experience building data warehouses and ETL pipelines
- Expert experience of SQL, python, git, dbt (incl. query efficiency and optimization)
- Expert experience of Cloud Data Platforms (AWS, Snowflake and/or Databricks) → Qualification preferred, not mandatory
- Significant experience of Automation and Integrations tools (FiveTran, Airflow, Astronomer or similar)
- Significant experience with IoC tools (Terraform, Docker, Kubernetes or similar)
- Significant experience with CI/CD tools (Jenkins, GitHub Actions, CircleCI or similar)
- Experience with real time data pipelines (AWS Kinesis, Kafka, Spark)
- Experience with observability tools (Metaplane, MonteCarlo, Datadog or similar)
- Experience within FinTech/Finance/FX preferred
Recommended Jobs
Physics Teacher
Physics Teacher – Lambeth – Help Students See the Universe Differently Imagine a classroom buzzing with curiosity, where students are captivated by the fundamental laws that govern our universe. An ...
Background Cast aged 20-40 needed for Short TV Film. 2- 4 days work. Paid
Payment is subject to Talent Talks commission, this being 15% for subscribed Talent Talks members and 20% for non-subscribed Talent Talks members.
Network Engineer (IT)
Network Engineer Position Description Secure Innovation is part of CGI’s Space, Defence and Intelligence business unit, focused primarily on the delivery of contemporary and innovative technical...
Operations Manager
London Heritage Quarter (LHQ) is a collaborative alliance of four Business Improvement Districts (BIDs) spanning Victoria, Victoria Westminster, Whitehall, and Northbank. Stretching from Belgravia to...
Electrical Shift Engineer
Electrical - Multi skilled shift engineer I'm helping a major FM service provider find a shift engineer on a large contract in central London. £47,000 - £50,000 + overtime 4 on 4 off days and night...
Customer Service Advisor (Charlton)
Customer Service Advisor Charlton Our client is one of the fastest growing companies in the UK in their field. They are currently looking for a Customer Service Advisor to join the team i...
Senior Machine Learning Engineer
Woven by Toyota is enabling Toyota's once-in-a-century transformation into a mobility company. Inspired by a legacy of innovating for the benefit of others, our mission is to challenge the current sta...
Legal & Compliance Manager
About Us Fideres delivers effective economic analysis and testimony, turning data into courtroom advantage. As global litigation experts, we help law firms secure new cases and offer unbiased, con...
UK Sales Manager - Pilates Equipment Brand
Remote/Field Based £35k plus bonus Join a Leading Pilates Equipment Brand - Exciting UK Sales Manager Opportunity Are you passionate about fitness and wellness, with a drive to grow sales and...
Conference & Banqueting Coordinator
Accor, the world-leading hospitality group, provides unique hotel stays and experiences in over 5,500 properties, throughout 110 countries. With more than 45 hotel brands, restaurants, bars, e...