Skip to main content

Award details

Reinforcement learning to support decisions in health service delivery, Artificial Intelligence, Data Science, Health Services Research, Operational Research, Machine Learning – PhD (Funded) Ref: 4157

About the award


Academic Supervisors:

Associate Professor Thomas Monks,  University of Exeter
Dr Michael Allen, University of Exeter
Dr Alison Harper, University of Exeter

The University of Exeter’s College of Medicine and Health is inviting applications for a fully-funded PhD studentship to commence in September 2021 or as soon as possible thereafter.  For eligible students the studentship will cover Home tuition fees plus an annual tax-free stipend of at least £15,609 for 3 years full-time, or pro rata for part-time study.  The student would be based in the College of Medicine and Health at the St Luke’s campus in Exeter.

Project Description

We are seeking an exceptional PhD candidate to work across the fields of reinforcement learning and health services.  In reinforcement learning an artificially intelligent ‘agent’ acts as an optimiser and interacts with an uncertain environment in order to obtain ‘reward’. Unlike supervised learning where models are trained on large datasets, an agent starts with no data and must be trained in a synthetic environment that mimics a real system.  There are many ways to train an agent including, self play where an agent plays against historical versions of itself to improve. Many applications of reinforcement learning have been in the field of games.  For example, training reinforcement learning agents to beat the world’s top players in Go, Chess and the video game Starcraft 2. These highly constrained settings have provided ideal opportunities to study reinforcement learning, but there has been little research tackling the challenges an agent would face in real world systems which are far more complex and unconstrained. Reinforcement learning offers substantial potential for future health care in terms of efficiency and patient outcomes. This is an exciting research area filled with many method and application challenges including capturing the diversity of a real world health system in a training environment, and agent reward gaming.

This doctoral project will work in the methodological area of reinforcement learning and in particular how to train competent agents to manage health service delivery systems such as ambulance services, emergency departments and elective and emergency operations.  A successful candidate will initially focus on developing computer simulation models, such as discrete event and agent based simulations, from the health services and operational research literature, and adapting them to generate data to train standard reinforcement learning agents such as those provided by Open AI.  A specific focus of the work will be on use of neural network architectures, but there will be opportunity for classical meta-heuristic and evolutionary algorithms to serve as comparators in the optimisation of models.  The candidate will also explore novel approaches to training high quality agents, the impact of different reward structures and new architectures.

This is unique opportunity for a talented individual to work in one of the most important areas of modern AI. The successful candidate will join PenCHORD: a thriving data science and operational research team in Exeter’s Medical School.  The team has had successes in applying data science and mathematical modelling to many aspects of health care including diabetes, cancer, acute stroke, mental health, ambulance services, emergency departments and COVID19.  PenCHORD is funded by the National Institute for Health Research (NIHR) Applied Research Collaboration South West Peninsula (also known as PenARC). PenARC is one of 15 ARCs across England, part of an £135 million investment by the NIHR to improve the health and care of patients and the public.  Upon joining the team a candidate will be trained in modern approaches to open science, following the Turing Way, covering skills to provide runnable and transparent models and artefacts for other researchers and the NHS. Training in machine learning and computer simulation will be provided as required.

This award provides annual funding to cover Home tuition fees and a tax-free stipend.  For students who pay Home tuition fees the award will cover the tuition fees in full, plus at least £15,609 per year tax-free stipend.  Students who pay international tuition fees are eligible to apply, but should note that the award will only provide payment for part of the international tuition fee and no stipend. 

Informal enquiries about the project are very welcome by email to the Project Lead, Thomas Monks (

Entry requirements

Applicants for this studentship must have obtained, a First or Upper Second Class UK Honours degree, or the equivalent qualifications gained outside the UK, in an appropriate area of science or technology. In addition applicants must have obtained, or be about to obtain, an MSc in data science, health data science, operational research or other strongly relevant quantitative subject.

Applicants must have strong demonstrable programming skills in Python, Julia or equivalent applied in science.

It is desirable that a candidate has training and experience in one or more of:
• manipulating data;
• machine learning particularly using Neural Network frameworks such as PyTorch and Tensorflow.
• Computer simulation

If English is not your first language you will need to have achieved at least 7.0 in IELTS and no less than 6.0 in any section by the start of the project.  Alternative tests may be acceptable (see

How to apply

In the application process you will be asked to upload several documents. 
• CV
• Link to your Github/GitLab/BitBucket profile that highlights your coding and data science work.
• Letter of application (outlining your academic interests, prior research experience and reasons for wishing to undertake the project).
• Transcript(s) giving full details of subjects studied and grades/marks obtained (this should be an interim transcript if you are still studying)
• Two references from referees familiar with your academic work. If your referees prefer, they can email the reference direct to quoting the studentship reference number 4157.
• If you are not a national of a majority English-speaking country you will need to submit evidence of your proficiency in English.

The closing date for applications is midnight on 9th July 2021.  Interviews will be held virtually in the week commencing 19th July 2021.

If you have any general enquiries about the application process please email or phone 0300 555 60 60 (UK callers) +44 (0) 1392 723044 (EU/International callers)  Project-specific queries should be directed to the main supervisor Thomas Monks (


Application deadline:9th July 2021
Value:Home tuition fees plus an annual tax-free stipend of at least £15,609 for 3 years full-time, or pro rata for part-time study.
Duration of award:per year
Contact: PGR Admissions