Principal AI & Data Engineer

Job Description

We are looking for a Principal AI & Data Engineer to join our team in London, UK in a hybrid working mode. In this role, you will design, build and productionize enterprise-grade AI and GenAI solutions on Azure and Databricks platforms. You will develop scalable AI applications, intelligent agents and data-driven solutions powered by large language models and modern AI frameworks. The role requires hands-on expertise in AI data engineering, cloud technologies and GenAI adoption while driving innovation, reliability and scalability across enterprise environments.

Responsibilities

• Develop and evaluate AI, GenAI and Agentic AI solutions using enterprise AI platforms and frameworks
• Build and optimize Retrieval-Augmented Generation (RAG) applications, including embedding strategies, prompt engineering and evaluation processes
• Design, develop and deploy AI agents using frameworks such as LangChain, LangGraph, AutoGen and similar technologies
• Create scalable AI workflows using Databricks, MLflow, Azure AI Foundry and cloud-native services
• Build and maintain robust data pipelines using modern data engineering practices and streaming technologies
• Deploy AI applications and services using Azure OpenAI, Azure AI Foundry, Databricks AI tools and Docker
• Design and execute evaluation frameworks to measure model quality, accuracy, reliability and reproducibility
• Implement CI/CD pipelines, testing frameworks, code quality controls and GitHub-based development processes
• Collaborate with stakeholders to translate business requirements into scalable AI solutions
• Contribute to architecture decisions, technical roadmaps and backlog definition while supporting delivery from prototype to production

Requirements

• Bachelor's or Master's degree in Statistics, Mathematics, Econometrics, Computer Science or a related field
• 8+ years of experience delivering data engineering, machine learning or AI solutions in enterprise environments
• Strong AI Data Engineering expertise with experience building production-grade AI and GenAI systems
• Hands-on experience with Azure AI Foundry, Azure cloud services and Databricks AI technologies
• Deep understanding of Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) solutions
• Strong experience with MLflow, GenAI evaluation frameworks and model benchmarking methodologies
• Advanced programming skills in Python and SQL with strong knowledge of data modelling and ETL/ELT processes
• Solid understanding of Delta Lake, Delta Live Tables, Unity Catalog, Spark Structured Streaming and large-scale data processing
• Experience with Azure Data Factory (ADF), Qlik Replicate and modern data architecture practices
• Excellent communication, stakeholder management, problem-solving skills and commitment to high-quality delivery

Nice to have

• Experience in commodity trading and supply operations, ideally within the energy sector
• Experience with Microsoft Copilot Studio, Databricks Mosaic AI, Agent Bricks or Genie
• Experience with LangGraph, AutoGen or other agent orchestration frameworks
• Familiarity with enterprise AI governance, security and compliance practices
• Experience leading technical teams, defining technical strategy and mentoring engineers

We offer/Benefits

• EPAM Employee Stock Purchase Plan (ESPP)
• Protection benefits including life assurance, income protection and critical illness cover
• Private medical insurance and dental care
• Employee Assistance Program
• Competitive group pension plan
• Cyclescheme, Techscheme and season ticket loans
• Various perks such as free Wednesday lunch in-office, on-site massages and regular social events
• Learning and development opportunities including in-house training and coaching, professional certifications, and courses
• If otherwise eligible, participation in the discretionary annual bonus program
• If otherwise eligible and hired into a qualifying level, participation in the discretionary Long-Term Incentive (LTI) Program

*All benefits and perks are subject to certain eligibility requirements