Data Engineer
Responsibilities:
- Develop and maintain Python-based systems deployed across remote platforms.
- Contribute to and improve data pipelines, ensuring reliable and efficient system updates.
- Build and enhance features for real-time data analysis and system monitoring to ensure high uptime and efficiency.
- Collaborate with data scientists and engineers to support advanced analytics and machine learning workflows.
- Support the migration of our codebase toward machine learning capabilities by building scalable, maintainable solutions.
- Analyze system logs and performance to debug issues and optimize operations using forensic analysis tools.
- Bachelor's or Master's degree in Computer Science, Mathematics, Data Analytics, or a related field.
- 3+ years of experience developing and deploying production-grade Python software.
- 3+ years of experience with Python and high-performance data libraries such as Polars and Pandas.
- Proficiency with JavaScript, SQL, and KQL.
- Experience with Extract, Transform, Load (ETL), Data Streaming, and Reconciliation.
- Experience building and maintaining deployment pipelines, including DevOps tools like Ansible, and containerization with Docker.
- Proficiency with cloud platforms (AWS or Azure) for deploying and scaling data systems.
- Highly desired experience with Azure, particularly Lakehouse and Eventhouse architectures.
- Experience with relevant infrastructure and tools including NATS, Power BI, Apache Spark/Databricks, and PySpark.
- Hands-on experience with data warehousing methodologies and optimization libraries (e.g., OR-Tools).
- Experience with log analysis, forensic debugging, and system performance tuning.
- Exposure to cloud-based systems.
- Familiarity with Agile/SCRUM methodologies in collaborative development workflows.
- Proficient English communication skills, both written and verbal, with the ability to engage diverse audiences effectively.
- Excellent analytical and problem-solving skills, with the ability to contribute effectively in a collaborative team environment.
Recommended Jobs
AI Engineer
AI Engineer London - mandatory 3 days/week Contract role : 6+ Months – Inside Ir35 Job Description Role Purpose The purpose of this role is to develop minimum viable product (MVP) ...
Professional Development Expert - Data Engineering (Level 5)
About Corndel Corndel is a unique, multi award-winning UK management and technology training provider that focuses on delivering the highest quality programmes throughout an employee’s career. We ...
Audit Manager - Natural Resources & Energy
Ideas | People | Trust We’re BDO. An accountancy and business advisory firm, providing the advice and solutions entrepreneurial organisations need to navigate today’s changing world. We work wi...
Year 6 Teacher - Part-Time, Permanent Position in...
A highly respected independent primary school in Westminster is seeking a dedicated and passionate Year 6 Teacher. This part-time, permanent position is set to begin in September 2025, offering an ex...
Inspirational English Teacher Wanted in Ealing!...
Our flourishing school in Ealing, renowned for its vibrant learning community, is seeking an inspirational English Teacher to join our dynamic and supportive English department. We are looking fo...
Planning Associate Director - BUPA
OUR COMPANY Initiative celebrates diversity in an inclusive environment where all of our 6,000+ strong talent across 90+ markets feel seen, heard, and valued. Our vision is to create not only a ...
Campaign and Planning Executive
What’s the role about? As our Campaign and Planning Executive you will be ensuring that all marketing projects are planned and delivered on time against seasonal objectives, whilst supporting our M...
Hospitality Staff - ExCel London Exhibition Centre
Here at ExCeL London Hospitality we provide catering services to one of the biggest and most dynamic exhibitions centers in the world, the ExCeL! Are you enthusiastic about providing outstanding ...
Legal PA
Legal PA Law Firm of the Year Winner is seeking to hire a pro-active and well organised Legal PA to join their Private Client Tax team. The successful candidate will provide secretarial and admini...
Content Manager
Edgewell is not just a company, but a vibrant global community of 6,800 visionaries, doers, and makers . Our family of over 25 personal care brands serves people in more than 50 countries . We are...