Staff Data Engineer, AI Evaluation
About us Founded in 2017, Wayve is the leading developer of Embodied AI technology. Our advanced AI software and foundation models enable vehicles to perceive, understand, and navigate any complex environment, enhancing the usability and safety of automated driving systems. Our vision is to create autonomy that propels the world forward. Our intelligent, mapless, and hardware-agnostic AI products are designed for automakers, accelerating the transition from assisted to automated driving. In our fast-paced environment big problems ignite us-we embrace uncertainty, leaning into complex challenges to unlock groundbreaking solutions. We aim high and stay humble in our pursuit of excellence, constantly learning and evolving as we pave the way for a smarter, safer future. At Wayve, your contributions matter. We value diversity, embrace new perspectives, and foster an inclusive work environment; we back each other to deliver impact. Make Wayve the experience that defines your career! Impact expected Wayve's machine learning-first approach relies on high-quality, well-structured data. The Evaluation Workflows and Measurement teams build tools and pipelines that power model evaluation at scale. As we scale our evaluation approaches and tooling, we need to process massive volumes of test data efficiently and reliably. This Data Engineer will be embedded in the AI Evaluation division to ensure our evaluation and analytics pipelines are robust, performant, and future-proof. Their work will strengthen our data foundations for fast decision-making, accelerate the availability of large-scale image and video analytics, and help us rapidly integrate and leverage data from external partners - enabling faster iteration across both offline and on-road evaluation.
Challenges you will own
- Build scalable and reliable data and analytics pipelines to process and enrich over 1 million hours of driving video data annually and supply mission-critical data to stakeholders across the business.
- Unlock rapid insights by architecting and optimising analytics pipelines that drive company wide development and decision-making.
- Collaborate across functions - including research engineers, simulation experts, robotics engineers, data scientists and safety drivers - to deliver and visualise enriched data.
- Improve pipeline observability, validation, and fault tolerance for production-grade robustness.
- Enable LLM-driven workflows by shaping data to be AI-consumable (e.g. chunking, embeddings, metadata).
- Reduce tech debt and simplify orchestration across Flyte, Databricks, and Azure-based infrastructure.
- Design and optimise distributed data pipelines to handle large-scale video and image data processing.
- Re-design and optimise existing analytics pipelines.
- Collaborate with the data platform team to integrate pipelines with Databricks for governance and compliance - and unlock massive scale for offline evaluation from third party datasets.
- Shape evaluation data to support future use cases like Retrieval-Augmented Generation (RAG) and natural language analytics.
- Proficiency in Python and SQL, with experience in frameworks like Pandas, PySpark, and NumPy for large-scale data processing.
- Expertise in debugging and optimising distributed systems with a focus on scalability and reliability.
- Proven ability to design and implement scalable, fault-tolerant ETL pipelines with minimal manual intervention.
- Knowledge of data modelling best practices, including the medallion architecture or comparable frameworks.
- Experience in workflow orchestration using Flyte, dbt, Airflow, or Prefect.
- Strong understanding of unit, integration, and data validation testing using tools like Pytest or Great Expectations.
- Familiarity with cloud infrastructure (preferably Azure) for managing pipelines and storage
- Ability to collaborate closely with stakeholders to understand requirements and shape data pipelines to meet user needs effectively.
- 5+ years of experience in a data engineering or similar role
- Experience with Docker, Kubernetes, Databricks
- Familiarity with shaping data for AI/LLM-based systems
Recommended Jobs
Band6:Locum Community Speech and Language Therapist: London
Job Title: Lo cum Community Speech and Language Therap ist Banding : 6 Location : London Start: ASAP Duration : Full time Rate : £28 –£32 Working hours : Monday – Frid ay , 08:3…
Band 7 Locum Paediatric Physiotherapist - London
Job Title: Locum Paediatric Physiotherapist Banding: Band 7 Location: London Working Hours: Monday – Friday 8:00 AM – 4:00 PM or 8:30 AM – 4:30 PM Start Date: ASAP Duration: 3 month…
Corporate Paralegal
Corporate Paralegal Prestigious US firm is hiring a new Paralegal to join the Corporate, Private Equity and M&A group in London. Up to £55,000Exceptional benefits including a free on-site gymHigh l…
Trading Infrastructure Engineer
Trading Infrastructure Engineer Quant Capital is urgently looking for an Trading Infrastructure Engineerto join our high profile client. Our client is a very successful and stable technolog…
Employment Associate
Job Details This award-winning, Top Tier Legal 500 ranked practice, located in the City, prides themselves on having an excellent quality of work; they now seek an Employment Associate 2-4+ PQE …
Senior Client Advisor
Church's is seeking a Senior Client Advisor in London to serve as a brand ambassador, maximizing sales and fostering customer loyalty. The role involves providing an exceptional in-store experience, m…
Senior Practitioner - East London - Adult Safeguarding
Liquid Personnel is currently recruiting for a Senior Practitioner to join a busy Adults Safeguarding Team based in Hackney. Benefits of the role: Great Learning and Development opportunity as …
Bartender - Scarfes Bar (50 Best Bars)
Press space or enter keys to toggle section visibility Role We are currently looking for a Bartender to join our team of associates in Scarfes Bar at Rosewood London. Our Bartende…
In-house Employment Lawyer
A fantastic opportunity to join a top international law firm based in London as an In-house Employment Lawyer. Key Responsibilities Provide expert legal advice on employment matters, particular…
Oracle ERP - Finance And/Or SCM Lead Consultant
About Us You want to boost your career and collaborate with expert, talented colleagues to solve and deliver against our clients' most important challenges? We are growing and are looking for peop…