Senior Data Engineer
We're seeking a highly skilled and motivated Senior Data Engineer to join our growing data team. In this role, you'll architect and maintain robust, scalable data pipelines and infrastructure that power our analytics, machine learning, and business intelligence initiatives. You'll work with cutting-edge technologies like Python, PySpark, AWS EMR, and Snowflake, and collaborate across teams to ensure data is clean, reliable, and actionable.
Responsibilities
Build and maintain scalable ETL pipelines using Python and PySpark to support data ingestion, transformation, and integration
Develop and optimize distributed data workflows on AWS EMR for high-performance processing of large datasets
Design, implement, and tune Snowflake data warehouses to support analytical workloads and reporting needs
Partner with data scientists, analysts, and product teams to deliver reliable, well-documented datasets
Ensure data integrity, consistency, and accuracy across multiple sources and systems
Automate data workflows and processes to improve efficiency and reduce manual intervention
Monitor pipeline performance, identify bottlenecks, and resolve issues proactively
Apply best practices in CI/CD, version control (e.g., Git), and infrastructure-as-code (e.g., Terraform, CloudFormation)
Enforce data security, compliance, and governance standards in line with industry regulations
Mentor junior engineers, conduct code reviews, and foster a culture of continuous learning and knowledge-sharing
Skills
Must have
5+ years of experience in data engineering or software development
Strong proficiency in Python and PySpark
Hands-on experience with AWS services, especially EMR, S3, Lambda, and Glue
Deep understanding of Snowflake architecture and performance tuning
Solid grasp of data modeling, warehousing concepts, and SQL optimization
Familiarity with CI/CD tools (e.g., Jenkins, GitHub Actions) and infrastructure-as-code
Experience with data governance frameworks and security best practices
Excellent communication and collaboration skills
Nice to have
N/A
Other
Languages
English: C1 Advanced
Seniority
Senior
Recommended Jobs
Paediatric Community Speech and Language Therapist
Paediatric Community Speech and Language Therapist – Hillingdon, London Banding: Band 6/7 Location: Hillingdon Start Date: ASAP Hourly Rate: £24.00–£32.00 an hour Working Hours: …
Case Officer - Education G7
Job Category : Social Care & Health Non-Qualified Client Location : Mercury House, London Borough of Havering Hours Per Week :36.00 Start Date : Immediate Start Start Time :09…
IT Engineer
Job Description: Job Purpose Intercontinental Exchange, Inc. (ICE) is a leading operator of global exchanges, clearing houses, data, and listings services. We connect businesses around the world t…
PAYROLL & RECRUITMENT LEAD
Location: London or Liverpool (6 days per month in office) Salary: Up to £42,000 + excellent benefits Contract: 12 months Fixed Term We are seeking an experienced and ambitious operations le…
SEN Teacher
SEN Teacher Role: Primary School Teacher with QTS Rate of pay: £750 - £900 Per Week Location: Sutton Coldfield - Birmingham About the role Connex Education is looking for a passionate and co…
Junior .NET Developer
Junior .NET Developer - Online Film Rental Company - London (Tech stack: Junior .NET Developer, .NET 9, C#, Web API 2, Entity Framework, Microservices, Azure, Programmer, Full Stack Engineer, Archite…
Part-time Nanny to care for 1 child, Job ID J1D5FC
A wonderful family in Wimbledon area is looking for a Part-time Nanny to provide proper childcare for their precious baby. They are looking for someone fun and outdoorsy who will become part of the f…
Operations Manager (Site Management & Logistics)
Job Summary:Company: DF Concerts and EventsDepartment: Concert & Event OperationsLocation: Glasgow and onsite and concerts and eventsReports to: Senior Operations ManagerWorking Hours: Full time plus…
Commercial Lines Pricing
I am currently working with a highly innovative and market leading consultancy who are looking to rapidly expand their Commercial Lines Pricing consulting team. They are searching for an experienced …
Renaissance Associate (Daily rate pay)
Job Summary To be an ambassador for Renaissance by hosting and delivering online webinars and forums to our customers and prospects. Responsibilities Deliver presentations at events online (and…