Uppsala Monitoring Centre/EHDEN pharmacovigilance evidence-a-thon

See also the accompanying article in Uppsala Reports here.

The week of 5 September had more than thirty colleagues from the Uppsala Monitoring Centre (UMC), EHDEN, DARWIN EU, and EHDEN Data Partners come together in Uppsala, Sweden to take part in the UMC/EHDEN pharmacovigilance evidence-a-thon.

The specific objective of this IMI EHDEN-sponsored event was to evaluate the feasibility and utility of performing characterisation analyses with EHDEN Data Partners to generate real world evidence in support of UMC’s ‘preliminary signal assessment’ process.  The scope of the evidence-a- thon involved all mature products with generic manufacturers, and sixteen outcomes that had previously been phenotyped within the OHDSI community as part of its COVID-19 research.  Prior to the event, the UMC team performed first-pass statistical screening to identify and rank drug-event combinations that would undergo ‘preliminary signal assessment’ during the week together.

Highlights

On Day 1, the UMC team, led by Niklas Norén, provided an overview of UMC’s current signal detection process. UMC collects spontaneous adverse event reports from the national pharmacovigilance centres which they aggregate into VigiBase, the WHO global database of individual case safety reports.  ‘First-pass statistical screening’ is based on their VigiRank algorithm, which combines stratified disproportionality analysis with other predictors such as geographic spread and the quality and content of individual reports.  Drug-event pairs thus identified are reviewed by pharmacovigilance assessors via an analytics platform for VigiBase called VigiLyze and internal tools of more experimental nature. External information sources like PubMed and DailyMed are also consulted.

The overall aim is to identify drug-event combinations, which warrant further investigation, based on strength of association, clinical relevance, and novelty.  These drug-event combinations then undergo ‘in-depth signal assessment’, which can include requesting original reports with more complete information from national centres, performing more thorough literature review, and performing various additional analyses within VigiBase.

Any drug-event combinations that still are deemed to merit further review after ‘in-depth signal assessment’ are communicated to the national pharmacovigilance centres, product manufacturers, and made available to members of the WHO programme as ‘Signals’ through VigiLyze. Most are also shared with the public via the WHO Pharmaceuticals Newsletter or peer-reviewed publications.  It is expected that other organisations (such as regulators or industry) may take the signals generated by UMC and perform their own ‘signal evaluation’ using other information sources UMC doesn’t have access to (such as clinical trial data and observational data) to reach a final determination about causality.

Later that day, eleven EHDEN Data Partners presented on their data which had been mapped to the OMOP Common Data Model. These Partners represented a range of data types, including national registries, general practice EHRs, and hospital databases.

Throughout the rest of the week, the UMC team reviewed ~100 drug-event combinations, most of which were dismissed upon initial review prior to considering potential use of observational data due to being already known on the product label or lacking consistency and clinical coherence, but nine drug-event combinations were brought forward with questions for consideration from RWE.  In eight of these cases, observational analyses were designed and executed across the EHDEN Data Partners, and results were reviewed collectively to determine if they provided insights to inform the UMC reviewer decision.

These pairs included a range of drugs (steroids, chemotherapy agents, Irritable Bowel Disease treatment, and an Alzheimer’s drug) and involved multiple outcomes (acute myocardial infarction, appendicitis, deep vein thrombosis, and ischaemic stroke).  RWE assisted the UMC in closing their review for four drug-event combinations and supported them in advancing four drug-outcome pairs  onto their ‘in-depth signal assessment’ process.  The ‘in-depth signal assessment’ pairs will now undergo the traditional UMC review process augmented by population-level estimation studies conducted across the OHDSI network to be collaboratively designed between EHDEN, UMC, and OHDSI researchers.  The one combination which the EHDEN databases were not able to support, involved an over-the-counter antihistamine, whose accuracy of capture was thought to be too unreliable to examine for anaphylaxis risk.

A range of observational analyses were applied to support the UMC team throughout the week.  Feature characterisation analyses were used to contextualise the exposure, indication, and outcome of interest, providing demographics and prevalence of prior conditions and drugs. These analyses proved useful in helping the UMC team recognise bias in the reported case series and to understand the risk of confounding attributable to the indication.

Incidence rate analyses were performed to estimate the background rate of outcomes in the general population and specific sub-populations, and used to quantify the rate of events observed during the drug’s ‘on treatment’ period and within an acute time-at-risk window.  Treatment pathway analyses was used to understand the prevalence of a drug’s use in first- and second-line therapy and to identify candidate comparator drugs for further exploration.  All cohorts and most analyses were designed centrally in a shared ATLAS user interface, and then JSON specifications were shared with Data Partners to import locally for execution.  Additional custom R/SQL scripts were also developed ‘on the fly’.

The execution time for most analyses was less than one hour, with several analyses requiring less than ten minutes to complete.  Results were collaboratively interpreted through review within ATLAS and sharing of aggregate statistics via MSTeams. The relative value of each Data Partner’s data varied across the analyses, depending on sample size of the question of interest, and specific questions primarily addressed, reflecting the need and value to have a large and diverse data network. Heterogeneity in results across populations, such as in the proportion of first-line Alzheimer treatments, underscoring the need for geographic representation, were also observed.

Summary

• The event made it clear that we currently have the ability to efficiently generate actionable real-world evidence to support preliminary signal assessment using our existing tools, data, and infrastructure. The week demonstrated that there isn’t a technical or scientific barrier, but rather simply a choice of resource allocation to integrate RWE into existing safety processes.
• As we worked through the drug-event combinations together, we began to see commonalities in the questions asked and were able to establish a more formulaic approach to addressing the questions.  Many of the needs seem potentially addressable through large-scale pre-computed summaries of aggregate statistics (such as drug-indication-outcome incidence and drug utilisation summaries), while others can likely be addressed through standardised analyses with limited customisable inputs. It was much easier to design useful analyses for the team once we shared the experience of worked examples, rather than designing in the abstract with only conceptual understanding of the problem space.
• Pharmacovigilance scientists and epidemiologists have historically worked with different data and come from different vantage points as it relates to safety and causality assessment.  Greater collaboration, including cross-disciplinary working sessions and in-depth education in both directions, is needed to establish shared understanding and to enable better alignment between decision makers' needs and the support that is provided
• External data holders may be receptive to strategic partnerships for pharmacovigilance, and not just the reactive one-off study execution that researchers are accustomed to engaging in. Collaborative projects through EHDEN may be one path forward if having greater international representation in our pharmacovigilance data portfolio is desired.
• The team agreed to collaboratively co-author a publication detailing the UMC/EHDEN evidence-a-thon experience and to share lessons learned to support the future use of observational data in pharmacovigilance activities.  The week’s experience also inspired ideas for how to develop new open-source analytic solutions that can support the pharmacovigilance use case even further.