To understand some of the issues which drive uncertainty in health technology appraisals, the Valid Insight team analysed the last ten years of Scottish Medicines Consortium (SMC) reimbursement decisions where products were not recommended for use within NHS Scotland. After removing non-submissions, 218 records were included in this analysis.

We reviewed both clinical and economic evaluation criteria, finding 278 unique mentions from the 218 records analysed. Our initial findings highlighted that economic critique was frequently the result of clinical uncertainties, therefore we focussed our analysis on understanding the issues driving this uncertainty. Our analysis showed that the clinical drivers were related to two issues: study design and outcomes from the evidence package.

Figure 1: Number of records with uncertainty associated with economic, outcomes-based and trial design related topics

Issues related to trial design were found in nearly half the reports (112 from 218), often with more than a single issue. Use of placebo control, a lack of direct comparison, non-inferiority and open label design were identified by the SMC as the key trial design issues driving uncertainty during appraisals. Others included limitations of trial length, the choice of comparator, as well as single arm studies.

Figure 2: Number of records with issues associated with trial design related topics

When exploring outcomes, comparable efficacy to other available options, missed endpoints, and limited clinical data, such as immature data or a lack of long-term data, were the most common critiques. Other clinical outcome critiques included contradictory data from the evidence package, limited data on quality of life, safety and the robustness of extrapolated data.

Figure 3: Number of records with issues associated with outcomes related topic

These clinical aspects were clear drivers of uncertainty in manufacturers’ economic case with unproven or unclear cost-effectiveness and no clear cost-benefit leading to a technology not being recommended for use.

It is key for manufacturers to identify critical gaps in the evidence package early, and if possible, amend to optimise the evidence package or plan for creating robust additional data generation which will support reducing the uncertainty for payers. Valid Insight has many years of experience in supporting manufacturers to do exactly this.

Reach out to discover@validinsight.com to understand in more detail specific cases and drivers.