Member of Technical Staff - Applied AI Software Engineer, Health
Overview At Microsoft AI, our health team is on a mission to help millions of users better understand and proactively manage their health and wellbeing. We’re responsible for ensuring that Microsoft AI’s models and services are useful, trusted and safe across diverse customer health journeys. We’re assembling a world-class team of builders with backgrounds in healthcare, big tech, and frontier AI research labs. Our goal is to build transformative products for millions of users across copilot.com and the wider Microsoft ecosystem. We are looking for engineers with experience designing and building LLM-based products, and taking them through to production. You will play a pivotal role in establishing Copilot as the leader in safe, informative, trustworthy and useful health information. We are specifically looking for engineers with experience designing, building and running evaluations for LLMs. You will be expected to build eval pipelines, curate and synthesise datasets, design automated analyses and explain results to internal stakeholders. Responsibilities
- Be a key cross-functional team member, and close collaborator to product teams. Make significant contributions from translating research into benchmarks to co-authoring product roadmap.
- Design and build evaluation systems that test LLM capabilities in the healthcare domain, and interpret and communicate results.
- Run experiments to determine how different prompting techniques affect results on internal and industry benchmarks.
- Improve the tooling that is used internally to implement, run and analyze evaluations.
- Design internal benchmarking and regression testing capabilities that capture model accuracy, safety and utility.
- Bachelor's or Higher Degree in Computer Science, or related technical discipline AND significant Python programming experience / machine learning research.
- Experience building with and around LLMs including prompt / context and harness engineering.
- Deep expertise in evaluating and understanding the performance of agents.
- Experience collaborating in cross functional teams, working through ambiguity to deliver high quality results.
- Have 0 to 1 experience with a bias towards shipping and learning, while balancing a high-quality bar.
- Proven ability to collaborate and contribute to a positive, inclusive work environment, fostering knowledge sharing and growth within the team.
- Experience in healthcare technology, or experience in the health domain.
- Experience with data engineering - handling text dataset sourcing, curation, and processing tasks at scale.
- Demonstrated written and verbal communication skills with the ability to work closely with cross-functional teams, including product managers, designers, and other engineers.
- Passion for learning new technologies and staying up to date with industry trends, best practices, and emerging technologies and patterns in AI.
Recommended Jobs
Engagement Director: Financial Services
Company Description Technology is our how. And people are our why. For over two decades, we have been harnessing technology to drive meaningful change. By combining world-class engineering, indus…
Senior Product Manager, Reporting Experiences (Hiring Immediately)
The Trade Desk is a global technology company and the world’s leading independent platform for digital advertising, with over 3,000 employees across more than 30 offices. Our technology helps adverti…
Content Creator (Hiring Immediately)
Who We Are VML is a global creative company that combines brand experience, customer experience, and commerce to create connected brands that drive growth. Guided by a Human First philosophy, VM…
Emergency Veterinary Nurse Vacancies
JOB OVERVIEW Job Title: Emergency Veterinary Nurse – Teddington Location: Teddington Employment Type: Temporary Locum Rate: £20 – £30 per hour depending on experience. About the…
Staff Software Engineer, Continuous Integration
About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a qui…
Nanny in N16, Job ID J22C3B
A lovely family in the Hackney area is looking for a full time live out nanny to help them with their baby. The ideal candidate will be someone experienced and local who has some flexibility. All gen…
Head of Product, Multi-Product Experience
We’re on a mission to back the UK’s small businesses like no one else Small businesses are the backbone of the economy, and we’re here to help them win. We’ve built a platform that uses clever dat…
Head of Fundraising
Prospectus is delighted to be partnering with our client to recruit a Head of Fundraising. The organisation was founded by Elsie Fogerty in 1906 to offer a new form of training in speech and dra…
Head Chef (Hiring Immediately)
Company Description Join us at Greene King pubs, where we have always been the beacon for communities, wherever people want to come together. Our collection of community pubs are the heart and soul …
Wash Doctor Mobile Valeting Partner
Join our community of car care specialists and grow your business. Wash Doctors is an on-demand car care platform. We connect customers to you seamlessly through our partner app. Become a Wash Doc…