We began our four-blog series by evaluating the role of real-world data (RWD) and real-world-evidence (RWE) in Real-world challenges and real-world data, and understanding the payer’s perspective, in Payers use real-world data cautiously. In our third instalment, we’re looking at how to find the right balance in using RWE.

With the rapidly growing availability of RWD, and the increasing demand for information on effectiveness, there’s a strong push for creating and using RWD. While regulators are seeing the value of RWE, and companies are investing in RWE programmes, payers are slower on the uptake.

Payers often use the argument that randomised controlled trials (RCTs) do not reassure predictability of product value in the real world. Companies hope to use RWE as a way to respond to these arguments – to confirm or deny the therapeutic value of the product. The value of RWE for payers therefore is as an aid to deeper understanding about what the product does in everyday use.

Reviews of the use of RWE can help to support or refute its value. These publications can be divided into three main classes based on their content:

• papers where the main message is in favour of using RWE (advocates)
• papers that review benefits and shortcomings of both RWE and RCTs (balanced)
• papers that call for caution or are outspokenly negative when discussing RWE (opponents)

The RWE proponents

The publications from RWE advocates describe the benefits of RWE studies without much criticism. Examples include reviews specific to a therapeutic area, such as gynaecology, ophthalmology or rheumatology, which describe RWE findings and contrast them with RCT predictions. They highlight the additional information that can be gained from studying RWD, such as:

• effectiveness in vulnerable patient populations
• healthcare resource use
• long-term safety
• safety in special groups such as pregnant women
• and long-term real-life outcomes

The reviews also give examples of practical consequences such as service improvements.

Publications funded or authored by large pharmaceutical companies offer insight into the dominant feelings about RWE in the industry. Pharma companies position RWE as a complement to RCT results, in order that payers and regulatory authorities can have better understanding of the performance of medications in practice.

Industry-sponsored papers often call for better sharing – perhaps code for ‘making available to the industry’ – of existing RWD. This is a sensitive issue as authorities can use RWD in negotiations on pricing and reimbursement and may not want to release the data and therefore lose this advantage.

There are also political and privacy questions:

• Political – should publicly funded databases be offered to private companies?
• Privacy – would patients want their RWD to be shared with big pharma?

Overall, industry sourced publications defend the position that RWE should be part of decision making, provided there are adequate measures taken to ensure quality.

The RWE opponents

Some publications are negative on the use of RWD. An American Society of Clinical Oncology (ASCO) March 2019 publication detailed how RWE could lead to false claims regarding effectiveness, and concluded “Journal editors and clinicians should be critical of studies that report effectiveness in the absence of efficacy and should question the plausibility of such findings”. A publication ‘Proceed with caution when using real world data and real world evidence’ by Korean professors who specialize in medical informatics, reviewed all weaknesses and technical issues and their consequences. And professors in law from the UNC Gillings School of Global Public Health believe that RWD carries legal risks, such as implications for off-label fraud enforcement.

Providing a balance

Most non-industry sponsored publications offer a balanced view. They recognise the weaknesses of RCTs, while cautioning the use of RWD.  Recommendations for RWE include:

• Combining different sources into an integrative approach might improve the validity of RWE
• Standardising data collection and processing
• Defining uniform and transnational quality standards

• Agreeing on and using criteria to define robustness of RWE

The GRACE (Good Research for Comparative Effectiveness) checklist was developed especially for observational studies that compare active therapies, and was tested on a series of 48 publications. Of these 48 articles, experts considered 21 ‘good’ and 27 ‘not good enough’. However, the correlation with the checklist outcomes wasn’t good enough and the authors concluded the checklist could only serve as a ‘first pass’, urging further development of adequate tools to check the quality of observational studies.

In conclusion, whereas several review publications hail the importance and future role of RWE, numerous publications are also highlighting its shortcomings. They call for caution when using RWE in healthcare decision making, and demand better quality and global quality standards. However, even if data gathering and analysis would be impeccable, there are still reasons to be cautious with the interpretation of RWE: bias (systematic error) and multiplicity issues.

To read about the balance of opinion on RWE in more depth, download our white paper Real-world evidence: Hurdles and the holy grail.

The Valid Insight team have deep expertise in leveraging real world data to help products achieve their full potential. Learn more by calling us: +44 (0) 20 3750 9833 or email us: discover@validinsight.com.