Staff Data Engineer, AI Evaluation
About us Founded in 2017, Wayve is the leading developer of Embodied AI technology. Our advanced AI software and foundation models enable vehicles to perceive, understand, and navigate any complex environment, enhancing the usability and safety of automated driving systems. Our vision is to create autonomy that propels the world forward. Our intelligent, mapless, and hardware-agnostic AI products are designed for automakers, accelerating the transition from assisted to automated driving. In our fast-paced environment big problems ignite us-we embrace uncertainty, leaning into complex challenges to unlock groundbreaking solutions. We aim high and stay humble in our pursuit of excellence, constantly learning and evolving as we pave the way for a smarter, safer future. At Wayve, your contributions matter. We value diversity, embrace new perspectives, and foster an inclusive work environment; we back each other to deliver impact. Make Wayve the experience that defines your career! Impact expected Wayve's machine learning-first approach relies on high-quality, well-structured data. The Evaluation Workflows and Measurement teams build tools and pipelines that power model evaluation at scale. As we scale our evaluation approaches and tooling, we need to process massive volumes of test data efficiently and reliably. This Data Engineer will be embedded in the AI Evaluation division to ensure our evaluation and analytics pipelines are robust, performant, and future-proof. Their work will strengthen our data foundations for fast decision-making, accelerate the availability of large-scale image and video analytics, and help us rapidly integrate and leverage data from external partners - enabling faster iteration across both offline and on-road evaluation.
Challenges you will own
- Build scalable and reliable data and analytics pipelines to process and enrich over 1 million hours of driving video data annually and supply mission-critical data to stakeholders across the business.
- Unlock rapid insights by architecting and optimising analytics pipelines that drive company wide development and decision-making.
- Collaborate across functions - including research engineers, simulation experts, robotics engineers, data scientists and safety drivers - to deliver and visualise enriched data.
- Improve pipeline observability, validation, and fault tolerance for production-grade robustness.
- Enable LLM-driven workflows by shaping data to be AI-consumable (e.g. chunking, embeddings, metadata).
- Reduce tech debt and simplify orchestration across Flyte, Databricks, and Azure-based infrastructure.
- Design and optimise distributed data pipelines to handle large-scale video and image data processing.
- Re-design and optimise existing analytics pipelines.
- Collaborate with the data platform team to integrate pipelines with Databricks for governance and compliance - and unlock massive scale for offline evaluation from third party datasets.
- Shape evaluation data to support future use cases like Retrieval-Augmented Generation (RAG) and natural language analytics.
- Proficiency in Python and SQL, with experience in frameworks like Pandas, PySpark, and NumPy for large-scale data processing.
- Expertise in debugging and optimising distributed systems with a focus on scalability and reliability.
- Proven ability to design and implement scalable, fault-tolerant ETL pipelines with minimal manual intervention.
- Knowledge of data modelling best practices, including the medallion architecture or comparable frameworks.
- Experience in workflow orchestration using Flyte, dbt, Airflow, or Prefect.
- Strong understanding of unit, integration, and data validation testing using tools like Pytest or Great Expectations.
- Familiarity with cloud infrastructure (preferably Azure) for managing pipelines and storage
- Ability to collaborate closely with stakeholders to understand requirements and shape data pipelines to meet user needs effectively.
- 5+ years of experience in a data engineering or similar role
- Experience with Docker, Kubernetes, Databricks
- Familiarity with shaping data for AI/LLM-based systems
Recommended Jobs
Sous Chef - Full Time
Salary: £15.80 per hour Shift hours: Full Time Sous chef Monday to Friday 40 hours per week £15.80 per Hour Spice Up Your Next Career Move! Are you a talented and enthusiastic Sous Ch…
AI Agent Engineer - Consultant/Snr Consultant level
Infosys Consulting is at the forefront of applied AI innovation, delivering real-world business value through the convergence of AI agents, machine learning, and modern enterprise architecture. As pa…
New Business Manager
Role: New Business Manager – North London Location: Remote (must live within region and be able to travel to clients regularly) Following a period of exciting growth and development at Roya…
Nursery Manager
Our client a stunning and welcoming nursery situated in East London, offering fulltime and flexible childcare for children aged 6months to 5 years. They aim to provide a safe, healthy, fun, stimulatin…
Account Manager - Northwest London
Role: Regional Account Manager – Northwest London Location: Remote (must live within region and be able to travel to clients regularly) Compensation: Competitive (dependant on experience),…
Associate Temporary Works Engineer
My client, a leading and award winning civil structural outfit based in central London, require an Associate on a permanent basis. As part of a well established and specialist team of structural en…
FT/PT Baristas required across SE London
Hi! Please read the whole text and respond to the list of questions at the bottom. Browns of Brockley is a neighbourhood coffee shop established in 2009, with two other shops in Forest Hill and …
Senior Pipeline Engineer
WHAT MAKES US EPIC? At the core of Epic's success are talented, passionate people. Epic prides itself on creating a collaborative, welcoming, and creative environment. Whether it's building award-w…
Network Systems Engineer (Pre-Sales)
Company Description Arista Networks is an industry leader in data-driven, client-to-cloud networking for large data center, campus and routing environments. What sets us apart is our relentless pu…
SEND Teaching Assistant
Experienced SEND Care Team Leader Available – Post-19 Provision – Harrow Location: Harrow (available to work across nearby boroughs) Availability: Monday to Friday, 8.30 am to 4.00 pm Start D…