Senior Data Engineer

Luxoft
London

We're seeking a highly skilled and motivated Senior Data Engineer to join our growing data team. In this role, you'll architect and maintain robust, scalable data pipelines and infrastructure that power our analytics, machine learning, and business intelligence initiatives. You'll work with cutting-edge technologies like Python, PySpark, AWS EMR, and Snowflake, and collaborate across teams to ensure data is clean, reliable, and actionable.

Responsibilities

Build and maintain scalable ETL pipelines using Python and PySpark to support data ingestion, transformation, and integration

Develop and optimize distributed data workflows on AWS EMR for high-performance processing of large datasets

Design, implement, and tune Snowflake data warehouses to support analytical workloads and reporting needs

Partner with data scientists, analysts, and product teams to deliver reliable, well-documented datasets

Ensure data integrity, consistency, and accuracy across multiple sources and systems

Automate data workflows and processes to improve efficiency and reduce manual intervention

Monitor pipeline performance, identify bottlenecks, and resolve issues proactively

Apply best practices in CI/CD, version control (e.g., Git), and infrastructure-as-code (e.g., Terraform, CloudFormation)

Enforce data security, compliance, and governance standards in line with industry regulations

Mentor junior engineers, conduct code reviews, and foster a culture of continuous learning and knowledge-sharing

Skills

Must have

5+ years of experience in data engineering or software development

Strong proficiency in Python and PySpark

Hands-on experience with AWS services, especially EMR, S3, Lambda, and Glue

Deep understanding of Snowflake architecture and performance tuning

Solid grasp of data modeling, warehousing concepts, and SQL optimization

Familiarity with CI/CD tools (e.g., Jenkins, GitHub Actions) and infrastructure-as-code

Experience with data governance frameworks and security best practices

Excellent communication and collaboration skills

Nice to have

N/A

Other

Languages

English: C1 Advanced

Seniority

Senior

Posted 2025-09-24

Recommended Jobs

Paediatric Community Speech and Language Therapist

Pulse
London

Paediatric Community Speech and Language Therapist – Hillingdon, London Banding: Band 6/7 Location: Hillingdon Start Date: ASAP Hourly Rate: £24.00–£32.00 an hour Working Hours: …

View Details
Posted 2025-09-09

Case Officer - Education G7

London Borough of Havering
Havering, Greater London

Job Category : Social Care & Health Non-Qualified Client Location : Mercury House, London Borough of Havering Hours Per Week :36.00 Start Date : Immediate Start               Start Time :09…

View Details
Posted 2025-08-29

IT Engineer

Intercontinental Exchange Holdings, Inc.
London

Job Description: Job Purpose Intercontinental Exchange, Inc. (ICE) is a leading operator of global exchanges, clearing houses, data, and listings services. We connect businesses around the world t…

View Details
Posted 2025-09-05

PAYROLL & RECRUITMENT LEAD

London

Location: London or Liverpool (6 days per month in office) Salary: Up to £42,000 + excellent benefits Contract: 12 months Fixed Term We are seeking an experienced and ambitious operations le…

View Details
Posted 2025-09-12

SEN Teacher

Connex Education
Sutton, Greater London

SEN Teacher Role: Primary School Teacher with QTS Rate of pay: £750 - £900 Per Week Location: Sutton Coldfield - Birmingham About the role Connex Education is looking for a passionate and co…

View Details
Posted 2025-09-20

Junior .NET Developer

Noir
London

Junior .NET Developer - Online Film Rental Company - London (Tech stack: Junior .NET Developer, .NET 9, C#, Web API 2, Entity Framework, Microservices, Azure, Programmer, Full Stack Engineer, Archite…

View Details
Posted 2025-05-19

Part-time Nanny to care for 1 child, Job ID J1D5FC

Little Ones UK Ltd
Wimbledon, Greater London

A wonderful family in Wimbledon area is looking for a Part-time Nanny to provide proper childcare for their precious baby. They are looking for someone fun and outdoorsy who will become part of the f…

View Details
Posted 2025-08-08

Operations Manager (Site Management & Logistics)

Stratford, Greater London

Job Summary:Company: DF Concerts and EventsDepartment: Concert & Event OperationsLocation: Glasgow and onsite and concerts and eventsReports to: Senior Operations ManagerWorking Hours: Full time plus…

View Details
Posted 2025-09-12

Commercial Lines Pricing

IPS Group
London

I am currently working with a highly innovative and market leading consultancy who are looking to rapidly expand their Commercial Lines Pricing consulting team. They are searching for an experienced …

View Details
Posted 2025-08-29

Renaissance Associate (Daily rate pay)

Renaissance Learning UK
London

Job Summary To be an ambassador for Renaissance by hosting and delivering online webinars and forums to our customers and prospects. Responsibilities Deliver presentations at events online (and…

View Details
Posted 2025-09-25