Randomised control trials (RCTs) are often described as the ‘gold standard’ for testing whether new treatments or interventions work. In this blog, Dr Ian Thomas who leads the ADR Wales Housing and Homelessness thematic area, looks at the use of linked administrative data as part of RCTs. Drawing on a pilot trial of an intervention for people leaving prison and at risk of homelessness, Ian explains what linked administrative data can offer and some of the challenges of this approach – particularly when working with seldom-heard groups.
What are randomised control trials (RCTs)?
Originating in the field of medicine, RCTs are increasingly being used to evaluate a range of complex social interventions, including those intended to prevent homelessness.
Typically, people in RCTs are randomly split into two groups:
- One group gets the new service or treatment.
- The other group continues with the support they would normally receive.
By comparing outcomes between the two groups, researchers can see if the new approach makes a difference compared to usual care.
What was the PHaCT trial?
Funded by the National Institute for Health and Care Research, PHaCT—short for Preventing Homelessness, improving health for people leaving prison, was designed to test the feasibility of conducting a full-scale RCT of a homelessness prevention intervention.
The intervention was run by the charity Crisis and offered one-to-one support to people leaving prison who were at risk of experiencing homelessness. Normally, people in this position would need to go to their local authority housing team and present as homeless.
PHaCT focused on two male prisons in England and Wales and involved Glasgow, Heriot-Watt, Wrexham and Cardiff Universities. One aspect of the trial was to explore whether linked administrative data such as healthcare records, could be used to track people’s outcomes over time.
Why using linked data could help?
Following up with people from seldom-heard groups, such as people leaving prison or experiencing homelessness, can be challenging. Instability in their lives means that multiple methods need to be adopted to maintain contact. Linking to administrative data offers a potential alternative approach.
Administrative data are generated through people’s everyday interactions with public services. When the appropriate approvals and legal safeguards are in place, researchers can link data years or even decades after an intervention ends, without needing to have up-to-date contact details.
Before considering whether linked data should be part of a full-scale RCT, PHaCT needed to establish whether the approach was feasible with people leaving prison who were at risk of homelessness.
What we did
Originally, the PHaCT trial had planned to link to healthcare data held by NHS England (formerly NHS Digital) and the SAIL Databank. The main question we sought to address was: How many pilot study participant records could be incorporated into the SAIL environment and feasibly followed up using linked data? However, time constraints meant that the NHS England linked data element was not feasible.
Participants were included in the SAIL test linkage if they:
- Were recruited from Welsh prisons.
- Consented to their data being linked.
- Had not withdrawn from the trial prior to attempting the linkage.
What we found
Although PHaCT aimed to recruit 80 people, only 34 joined the trial, partly due to delays in setting up contracts and challenges with data collection in English prison sites.
Of the people recruited, 17 met the criteria for the test linkage. Their details were sent securely to SAIL:
- 11 out of 17 people (65%) could possibly be followed-up using linked data.
- 6 out of 17 people (35%) could not be followed-up using linked data, due to missing or low quality information needed to achieve a link.
Given the small sample, no further breakdowns were conducted to protect people’s anonymity.
The PHaCT trial also followed up participants using traditional contact tracing methods, including nominated people and probation officers. After removing three withdrawals, only 3 out of 31 participants (10%) responded to follow‑up at three months.
Why does this matter?
To understand whether an intervention has lasting impact, researchers need to measure outcomes over extended periods of time. But, as PHaCT found, even short‑term follow‑up can be challenging with certain population groups. Linked administrative data offers a promising alternative to contact tracing approaches.
If participants consent, researchers can access administrative data securely held by organisations like the SAIL Databank. Linking data means that RCTs:
- Lose fewer people over time, retaining the ability to detect small changes in outcomes.
- Can measure outcomes over far longer periods than would be practical using usual contact tracing methods.
- Potentially reduce their costs, as comprehensive follow-up with seldom-heard groups can be resource intensive.
Challenges and limitations
Administrative data have important limitations, which broadly fall into two areas: who is captured and what is measured.
People belonging to seldom‑heard groups are often subject to structural and other barriers to accessing services, and may avoid them because of negative past experiences. This increases the likelihood that they are absent from public services data altogether.
Also, even where people do appear in administrative data, subjective outcomes like wellbeing or life satisfaction, are not routinely recorded. Depending on the intervention being evaluated and its intended outcomes, administrative data may therefore be of limited value as part of an RCT design.
On a more practical level, devolution adds further complexity to linked data in cross-UK studies, as was the case with PHaCT. Because certain policy areas are devolved matters – including housing and health – England, Wales, Scotland, and Northern Ireland often hold different datasets, with slightly different measures. Combining them can require specialist approaches, such as federated analysis in which data stays in each nation but is analysed in a coordinated way.
Final thoughts
The PHaCT trial showed that use of linked data in a RCT setting was possible, even for groups with potentially unstable living situations, such as people leaving prison who are at risk of experiencing homelessness.
Although it is encouraging that we could potentially follow up with 65% of participants, there is still room to improve this rate. In the PHaCT trial, we only collected people’s most recent settled address before entering prison. If we had asked for a brief address history, it might have increased the chances of finding people’s records based on these additional addresses.
While administrative data and its linkage have limitations, using existing data could help run more reliable and cost‑effective trials with extended periods of follow-up, and in doing so support the development of better homelessness services.
Outcomes from the PHaCT trial have now been published in:
- A paper summarising the pilot trial design and its findings, which includes the linked data element.
- A more focused qualitative paper on the implementation of the intervention and the pilot trial process.
The PHaCT team would like to acknowledge all the data providers who made anonymised data available for research via the SAIL Databank.
This post was originally published by ADR Wales, see the original post here