Major study into clinical trial oversight processes finds commonly used 'triggered monitoring' processes may not be effective
24 Aug 2018
The results of the TEMPER study, a major MRC CTU at UCL study about how best to run clinical trials, have been published in Clinical Trials journal. They show that a commonly used approach in oversight of clinical trials, so called "triggered monitoring", may not be as effective in current practice as expected, but they also provide evidence about how the approach could be optimised and improved.
"Monitoring" in clinical trials is a broad term referring to the activities trial sponsors (those responsible for running trials) undertake in order to make sure trial participants are protected and trial results are robust. Often, monitoring takes the form of frequent visits by sponsor representatives to the healthcare centres where trials are taking place. This has the advantage of giving monitors direct access to medical notes and other documentation that can show whether or not a trial is being run correctly. However, it is expensive and may not always be justified, particularly in lower-risk trials.
As part of a wider drive to get new developments in healthcare more quickly and more affordably to the wider public, trial monitoring has come under scrutiny as a particularly costly and inefficient process in drug development. Researchers are therefore looking for alternatives to the frequent site visit model that will still adequately protect patient safety and ensure trial data are reliable.
One alternative approach is triggered monitoring. This uses information reported to the sponsor from the trial centre to calculate an overall risk score for each, based on factors such as whether the centre is sending in trial data on time, whether the centre is sending all required information about participant safety, or whether the centre is correctly following the trial procedures. Each factor has an associated "trigger", and the more triggers fire at any time, the higher the centre's risk score. Centres with highest overall risk scores are prioritised for the next “monitoring visits”. This should be both more efficient, as some visits are avoided, and effective, as the high risk centres are still looked at in more detail.
This approach will only work well if the trigger rules themselves are reliable. TEMPER (TargetEd Monitoring: Prospective Evaluation and Refinement) aimed to test this. For three large clinical trials running in the UK, trigger rules were proposed to identify higher-risk centres for visits. Each centre triggered for a visit was paired with a centre that was similar in important respects - number of participants and time since the first participant joined the trial - but as different as possible in terms of risk (i.e. currently with a low overall risk score). Both centres were visited at a similar point in time, and the results were compared. If the approach worked, we would expect more “serious” findings from the visit to the higher risk centre in each pair.
The paired visits - 84 in total - took place between 2013 and 2016. Overall, there was no statistically significant difference between the amount of important findings at the higher and lower risk centres, indicating that the triggered monitoring approach, as currently used, is not effective enough.
However, there was some evidence that it may work better when a particular type of administrative problem is found before, rather than at, the centre visit. A large number of findings across all visits related to documentation of informed consent, the key step before participants join a trial where they formally agree to participate. There are now well-defined and commonly-used methods for checking these at the trials units at the time of trial entry rather than waiting until a centre is visited. The TEMPER researchers noted that when these findings were excluded (therefore replicating a hypothetical scenario where they had been checked before each visit), the higher risk centres did have significantly more important findings.
TEMPER also generated some information about the best use of triggered monitoring. Of the factors used to assess centre risks, the most useful appeared to be: the time taken for centres to resolve questions about data quality, known cases where a centre had not followed trial procedures, and a group of manually-added risk factors collectively known as "trial team concern". There was also some evidence that centre staffing was an important factor, for example the presence of a research nurse with a wide-ranging role at a centre was associated with a lower chance of serious findings at visits.
TEMPER was one of the largest ever pre-planned studies into how best to run trials, and has provided important evidence on a key clinical trial process. This sort of study is vital in helping trials be as efficient and effective as they can be, thereby protecting trial participants better, and getting reliable results and new treatments more quickly to the general public. As such, they form a core part of the work of the MRC CTU at UCL and we will continue to carry out research in this and other areas.
The study was funded by CRUK with support from the Medical Research Council.
Further information