Senior Data Engineer

Luxoft
London

We're seeking a highly skilled and motivated Senior Data Engineer to join our growing data team. In this role, you'll architect and maintain robust, scalable data pipelines and infrastructure that power our analytics, machine learning, and business intelligence initiatives. You'll work with cutting-edge technologies like Python, PySpark, AWS EMR, and Snowflake, and collaborate across teams to ensure data is clean, reliable, and actionable.

Responsibilities

Build and maintain scalable ETL pipelines using Python and PySpark to support data ingestion, transformation, and integration

Develop and optimize distributed data workflows on AWS EMR for high-performance processing of large datasets

Design, implement, and tune Snowflake data warehouses to support analytical workloads and reporting needs

Partner with data scientists, analysts, and product teams to deliver reliable, well-documented datasets

Ensure data integrity, consistency, and accuracy across multiple sources and systems

Automate data workflows and processes to improve efficiency and reduce manual intervention

Monitor pipeline performance, identify bottlenecks, and resolve issues proactively

Apply best practices in CI/CD, version control (e.g., Git), and infrastructure-as-code (e.g., Terraform, CloudFormation)

Enforce data security, compliance, and governance standards in line with industry regulations

Mentor junior engineers, conduct code reviews, and foster a culture of continuous learning and knowledge-sharing

Skills

Must have

5+ years of experience in data engineering or software development

Strong proficiency in Python and PySpark

Hands-on experience with AWS services, especially EMR, S3, Lambda, and Glue

Deep understanding of Snowflake architecture and performance tuning

Solid grasp of data modeling, warehousing concepts, and SQL optimization

Familiarity with CI/CD tools (e.g., Jenkins, GitHub Actions) and infrastructure-as-code

Experience with data governance frameworks and security best practices

Excellent communication and collaboration skills

Nice to have

N/A

Other

Languages

English: C1 Advanced

Seniority

Senior

Posted 2025-09-24

Recommended Jobs

Waiter - Eagle bar

Rosewood Hotel Group
London

Press space or enter keys to toggle section visibility The Chancery Rosewood, a Rosewood Hotel Group property, is housed in the iconic former U.S. Embassy on Grosvenor Square, reimagined…

View Details
Posted 2025-10-03

Team Executive Assistant

Oriel Partners
London

5 Days Office Key Duties for the Team EA  ~ Comprehensive diary management for the Corporate Credit team, including scheduling calls, internal/external meetings, video conference calls, travel boo…

View Details
Posted 2025-07-01

Lead Technical Writer - NPI

Stratford, Greater London

Collaborate with Service, R&D, Engineering, Manufacturing, Business Management, and Marketing to create comprehensive support documents and procedures for new products. Develop product-specific FR…

View Details
Posted 2025-10-21

Anti-Social Behaviour Complex Case Officer

Uxbridge, Greater London

Anti-Social Behaviour Complex Case Officer needed in Uxbridge The rate is £25.61ph PAYE This is a temporary role The reference number is:                   RQ1610374 The successful candida…

View Details
Posted 2025-10-09

Junior PR Executive

The Opportunity Hub UK
London

A dynamic and well-regarded PR agency is seeking a charismatic and motivated Junior PR Executive to join the team. This is an exciting full-time opportunity for someone with a passion for theatre and …

View Details
Posted 2025-09-02

Finance Job-Mandarin speaking Settlement Officer -London-rj

People First Recruitment
Central London

Please follow us on WeChat to see all our Cantonese and Mandarin jobs, interview tips and London news: Your New Job Title: Mandarin speaking Settlement Officer , London The Skills You'll …

View Details
Posted 2025-09-12

Financial Advisor Malaysia

Prestige IFA Jobs
London

My new client in Malaysia has a small, yet fully regulated and licensed brokerage in Malaysia. If you are an advisor in the region look at a positive change, then this opportunity is well worth consi…

View Details
Posted 2025-10-12

Admin Assistant (BR5)

Bromley Council
Bromley, Greater London

Job Category :Admin / Clerical Location : Saxon Centre, Bromley Council Start Date : Immediate Start Start Time : 10:00 End Time : 18:00 Salary: £10.57 Administrative Officer within ou…

View Details
Posted 2025-09-10

Vehicle Damage Assessor

Stratford, Greater London

As a Vehicle Damage Assessor, you'll be at the heart of our repair process-assessing damaged vehicles, producing accurate repair estimates using Audatex, and working collaboratively with our producti…

View Details
Posted 2025-10-24