The aim of the project was to examine in more detail the question of whether booked appointment times might reduce waiting times in Urgent Treatment Centres (UTC). This was done by considering a “generic” UTC – a kind of average over services of this type which were submitting high quality data to the Emergency Care Dataset.
Data on the paths patients took through these services, how frequently patients arrived and the number of staff present were taken from this data and used to build a Discrete Event Simulation model. The modelled average time taken for initial assessment of the patient, treatment, and any investigations or tests was chosen to give the best fit to actual waiting times. The distribution of patient arrival times was then modified to simulate the introduction of additional booked appointments, leading to a more even spread of patient arrivals throughout the day.
The model showed a decrease in overall waiting times when higher percentages of patients booked a timeslot in advance. This will be used to better inform policy on the number of booked appointments needed to make a significant impact to waiting times. Further analysis is expected to shed light on the impact of different ways of prioritising patients according to both clinical need and whether they have booked in advance.
This is a great test case for the use of Discrete Event Simulation to inform policy and strategy at a national level. It demonstrates the impact that such modelling can have and creates opportunity for similar work in the future, ensuring that decision making is supported by high quality analytical insight.
The aim of the project was to create an easy to use model for predicting demand and capacity for Covid vaccinations.
During the project the team came up against many barriers, the biggest of which was getting access to the data. The other issue they had with getting the data was the limits imposed on the amount of data you could download in one go, meaning that it was a lengthy process.
They modelled potential update scenarios and have started to work on this and have making great progress. They will use the previous vaccination data, along with the Winter 2022 vaccination data to strengthen the tool, so that following vaccination seasons it can be used more widely. They will look at other vaccinations it could be applied to, such as flu.
This will give users an easy to use tool to keep track of vaccinations and pick up problem areas quickly. It will save time and money on current methods and enable more people to use and understand the vaccination position
The aim of the project was to understand the current pathway to assessment of Autism and ADHD in children, identifying any bottlenecks. We wanted to know what change needed to take place in order to keep on top of ongoing demand levels and see patients within an acceptable time - what do we need to do to maintain a steady state and avoid a backlog from developing?
In addition to maintaining a steady state, we wanted to understand how to eliminate the backlog that has developed as a result of the ongoing demand not being met.
The main bottlenecks were identified and the issue is not at the beginning of the pathway (getting the completed questionnaires back from parents/schools) like it has been suggested at the start of the project, but further down the pathway due to clinical staffing capacity. Furthermore, the developed model shows that recruiting just one extra clinician would result in having a steady state with the backlog not growing. The model also shows possible staffing requirements to remove existing backlog in 1-5 years in addition to keeping a steady state.
The findings and the model demo have been presented to the stakeholders including service managers, clinical and operational leads, service heads and the Trust board, and discussions on plans for how the model is used to support any improvement work in the service is already taking place.
Nottingham University Hosptials has two major Acute Medicine Assessment areas (WB3 and AMRA), where patients can be reviewed before being sent to the most appropriate specialty base ward bed for their needs.
WB3 and AMRA run at very high occupancy levels, and patients frequently wait in the Emergency Department (ED) for a WB3/AMRA bed for more than 6 hours – this is a known cause of delay-related harm. Stakeholders feel that a major constraint is patients waiting in WB3/AMRA who are ready to be moved to a ward bed.
The team wanted to investigate what the impact would be if patients were able to leave WB3/AMRA more quickly. What would this do to the backlog of queues in ED, and how would it affect waiting times there?
A Discrete Event Simulation (DES) was made of the entire Acute Medicine assessment pathway. They then simulated several alternative scenarios in which patients leave WB3/AMRA more quickly after their period of assessment is completed – and were able to see the impact of this in the key metrics.
If all patients left WB3/AMRA within four hours of their request for a ward bed, patients would never have to wait in ED for a WB3/AMRA bed for more than 6 hours. Even if all patients left WB3/AMRA within 16 hours of their request, this would still reduce the number of patients waiting more than 6 hours by 90% - substantially reducing the risk of delay-related harm
We can now start to quantify the number of additional ward beds that would need to be freed up to support this improved flow, and bring these findings to discussions on how/where additional bed days could be released.
The aim of the project was to use Discrete Event Simulation (DES) to model the current wait list for High Intensity Cognitive Behavioural Therapy (HI-CBT). The team wanted to see if changing the allocation of therapist resources by running an evening face-to-face clinic, instead of an evening group, would improve overall wait times for therapy.
Patients were created as entities within the model, and average % preferences for IESO (online therapy), group therapy, virtual or face-to-face 1-1 appointments were assigned. Along with whether the patient was a priority, preferred evening or face-to-face appointments, and average number of appointments offered. They used current and historic data of people waiting or seen for therapy, attendance rates, and meeting with the data lead and service managers within Steps 2 Wellbeing, in order to gain an accurate picture of the waiting and treatment pathway for patients.
The model does confirm that the longest wait times appear to be for people waiting for face-to-face or evening 1-1 appointments. The model shows improvements in waiting times when an evening group is changed to a 1-1 face-to-face evening clinic.
The team is making changes to the model based on stakeholder feedback and will present to senior management within the service.
The project aim was to develop a Discrete Event Simulation (DES) model based on Colorectal Cancer pathway at Gloucester Royal Hospital to identify the key delays within the pathway and identifying feasible solutions.
The team met with cancer service leads to discuss the scope of the project and discussed the patient pathway in detail. Three years of anonymised patient records were used to create the model. Delays for the pathway were split into patient & provider delays to focus only on provider delays.
They used SimPy to create a DES model using the current colorectal patient pathway. They ran the simulation for 100 runs to get the average waiting time for each stage of the pathway to identify the stages that took the longest amount of time. The DES model showed us the key delay, which is diagnostic delays caused during investigative stages.
The findings from the analysis will be presented to cancer leads to support reducing the waiting times for colorectal patients. The DES model helped to evidence key bottlenecks of the patient pathway and will help to discuss the prospect of how increased resource can help improve patient outcomes.
The primary aim of the project was to improve Urgent Treatment Care (UTC) performance by developing a model which would help the Emergency Department (ED) team with the allocation of staff and rooms to match patient flow. The team wanted to understand what further staff and resources would be required for any variations in patient attendance. The ED team wanted to understand how changes in the patient pathway would reduce bottlenecks, for example, what would be the impact of front-loading diagnostics.
Initially, a discrete event simulation model was developed to show how patients currently flow through UTC. After an initial model had been developed, it was used to test the proposed alternative pathways for diagnostics. It was also used to model different numbers of staff and rooms. The model made it easy to understand the impact of changes to the existing provision and what would be most cost-effective.
The ED department was redesigned based on the model findings, which identified a need for additional rooms leading to reconfiguration of the space. It started a review of staffing rotas to see how they could be better aligned to findings from the analysis and future staffing rotas are designed to align with what the model identified was needed to meet the 95% target.
The project aimed to use forecasting and machine learning methods to predict the length of stay and arrivals to the Emergency Department (ED) at different times of day, days of the week and months of the year. Understanding this would allow the team to examine trends and predict future demand so they better plan workforce.
They have designed a web-based app where they can input the Sheffield Children’s hospital data and use it to explore trends in attendances and length of stay. This allows them to see patterns and suggest improvements, such as when it would be better for extra staffing. The web-page produces a “Forecast” demonstrating the attendances by hour/day/month/year as well as length of stay via interactive buttons. It is then able to give an estimate for a selected time to predict future demand.
They have completed the forecasting methods for this project and are now developing the machine learning model. The team hope to gain permission for this to be used on a wider scale so individual trusts can input their own data.
This was an attempt to model the impact of enforcing timely primary care interventions when 111 calls are undertaken.
More information: https://arc-swp.nihr.ac.uk/publications/modelling-nhs-111-demand-for-primary-care-services-a-discrete-event-simulation/
The aim of the project was to use available Primary and Secondary Care patient history and demographic information to better predict the chance of admission for respiratory conditions and subsequent length of stay, in order to better target interventions.
Due to time and data constraints, the modelling is currently at an early stage, but has demonstrated some interesting preliminary findings and put the structure in place for further development.
A Logistic Regression confirmed some expected factors but also revealed some surprises. An initial neural network with very little optimisation matches the Logistic Regression performance, with the potential to beat it when other features are included.
The model needs to take other features, particularly more patient history, into account to improve performance and be extended to predict length of stay as well as admission.
The aim of the project is to find out what role Patient Initiated Follow-up (PIFU) can play in the redeployment of capacity to address the backlog. This will be done by mapping outpatient pathways for rheumatology sub-pathway using discrete event simulation (DES).
The team did what-if modelling of resource use based on the proportion of patients on a PIFU pathway, the rate of PIFU patient-initiated requests and the use of advice & guidance. They used this to understand how released resource is redeployed, and how the upstream referral-to-treatment (RTT) waiting list behaves (size and waiting time to first outpatient).
The focus was on rheumatology as a case study since it had well documented pathways and good clinical evidence base on PIFU, with PIFU actively endorsed by NHSE and GIRFT. It is mainly an outpatient specialty with many chronic patients on long-term follow-up, meaning that the effect of PIFU is in theory amplified.
Other opportunities include discussing a spin-off model that includes effect of other digital musculoskeletal peri-treatment interventions or creating a proof-of-concept generalised DES that can be used for other PIFU specialties (PIFU is being advocated across most elective specialties).
On the PIFU rheumatology model itself, the team aim to continue refining the model based on internal stakeholder feedback. A secondary aim is to create an end-user toy tool that could help demonstrate some of the what-if scenarios to operational managers, in collaboration with NHSE Digital Care Model colleagues.
The project aim was to develop a predictive model to identify patients at high risk of admission and to provide explanatory feedback as to why the patients were at risk. There were three elements of the project:
Using structured data to generate a predictive model
Use unstructured data from patient notes to generate a predictive model
To combine the two models to create an enhanced model
Three models were developed using structured data – logistic regression, random forest and neural network. Logistic regression performed best.
For the unstructured data, attempts to use more advanced Natural Language Processing techniques (such as neural networks and transformers) were unsuccessful due to limited computing power. However, a more basic model using a Term Frequency-Inverse Document Frequency matrix with a random forest showed some improvement on accuracy.
Electronic health record data can accurately be used to predict whether a patient is at risk of non-elective admission. Administrative events are often better indicators than clinical measures, however EHR data is prone to several limitations and biases which can lead to counter-intuitive correlations. For analysis of unstructured data, greater computing power than I have access to is required to analyse the large quantities of patient notes, even when focusing on relatively short time frames.
It will make a difference to patient care by allowing patients at risk of non-elective admission to receive preventative care, leading to better health outcomes for the individual patients and freeing up inpatient resources for other patients.
The aim of the project was to see if the occurrence of violent incidents on hospital wards could be reduced, if incidents could be pre-empted and interventions could be taken to try to prevent them from happening.
The trust has a wealth of electronic information about service users, staff, incidents, and what happens on inpatient units. The plan was to combine this information into a dataset and use machine learning techniques to predict which of our wards were most likely to suffer from an incident the following day. A dashboard would highlight these wards to senior nursing staff, who could experiment with actions to reduce incidents.
Discussions with key staff helped to identify which features from our systems would be the most helpful in predicting incidents. A large dataset was constructed, and machine learning methods were used in an attempt to predict where incidents might happen. Unfortunately, the dataset did not prove to be informative enough to generate a working model – the predictions made were inaccurate and there was no way to apply them to the work environment.
The aim of the project was to employ geographical and statistical analysis of past and present travel times to understand the impact the flow of activity into London has on travel times for patients, whether referral pathways could be changed to minimise patient travel, and the extent to which additional sites would have a beneficial impact.
Initial analysis was undertaken on a sample of cardiac procedures within the hospital spell level data. The analysis identified a particular gap in the accessibility of cardiac surgery for patients in the Kent & Medway area, where there would be a significant benefit for the local population in the provision of a new cardiac surgery site or sites. The methods used gave a quantified and visual view of the impact of the options, and indicated the optimal configurations with 1 or 2 additional sites.
The team created a Streamlit application which allows a user to select from a list of possible new sites, and see the impact on travel times for Kent & Medway patients using maps, charts and a number of key travel time metrics.
The South East Cardiac Network have reviewed the findings, and this work will form part of the evidence base for service planning in the region. More generally, there has been interest in applying the methods and tools to other services.
There are three current significant health agendas to which this project broadly relates.
These are
- The inequalities agenda
- The digital and transformation agenda, and
- The Net Zero agenda.
The aim of this project was to explore feasibility of considering the impact of a new clinic location on inequalities and carbon emissions, and any co-benefits or trade-offs between the two. Ultimately, the aim was to enable decision making to incorporate these two significant areas.
The team have achieved:
The automation of a Health Equity Assessment to compare the profile of patients in a service to the population at large
The running of a number of variations to a logistic regression model to identify features predictive of someone not attending their appointment
Quantifying an estimate for the patient travel carbon emissions associated with the current service design.
They are intending to continue working together as a team to finish the project’s initial scope (e.g. inc. possible new clinic sites and model impact on unmet need / emissions) – potentially supported by code from another project, emphasising the benefit of open source.
They have been successful in securing a funding from the Greener NHS Healthier Futures Action Fund joint bid.
The aim of this project was to use various crime data variables and spatial analysis to turn them into useful insights for inclusion in assessed intelligence reporting.
The team chose to focus on violent crime offence data as they wanted to create our model using free and open source software.
They used QGIS and various Python libraries to explore the data. For the purpose of their model they obtained detailed geographic offence data uploaded by police forces in England and Wales.
The model was able to take a crime indicator (e.g. violent and sexual offences) for a region in the UK, prepare the data, apply and compare map classification schemes and run various analysis techniques to identify spatial autocorrelation, LISA , identify hotspots, coldspots and outliers. The statistical output provided a level of confidence in the findings that they can incorporate into intelligence confidence reporting levels.
The model will be shared with Commodity Threat Leads with the intention of using it internally to identify spatial trends in datasets that are restricted. Applying spatial analysis to these datasets will help identify regional variation and help quantify visual observations that will provide statistical certainty to any observed visual findings. The model will be proposed and applied to more restricted crime data in order to be used to draw out useful visualisations and insights to feed into intelligence reporting.