Peetluk LS, Djibo DA, Layton JB, Deng J, Deshazo J, Richey MM, Ogilvie RP, Parambi RJ, Miller M, Song J, Tarazi W, Weatherby LB, Bell EJ, Anthony MS, McMahill-Walraven CN, Yang GW, Seeger JD, Amend KL. Considerations for negative control outcome analyses in vaccine effectiveness studies: post-hoc analyses from an adult vaccine effectiveness study conducted within the FDA BEST initiative. Presented at the 2024 ISPE Annual Meeting; August 27, 2024. Berlin, Germany.

BACKGROUND: Negative control outcomes are useful in observational studies to evaluate potential residual confounding. For vaccine effectiveness (VE) studies, the first 14 days after vaccination has been suggested as a negative control period, since vaccines are not effective immediately after administration. However, analyses from the FDA BEST Initiative observed that COVID-19 vaccines were moderately protective against COVID-19 during this time.

OBJECTIVES: 1) To evaluate reasons for the early apparent protective effect of BNT162b2 and mRNA-1273 monovalent COVID-19 vaccines on COVID-19 diagnosed in any medical setting, and 2) to examine alternative negative control outcomes.

METHODS: We investigated patterns of COVID-19 testing and diagnoses within 14 days of BNT162b2 or mRNA-1273 COVID-19 vaccination among cohorts of vaccinated and matched unvaccinated adults (18-64 years), using data from Optum and CVS Health claims data including COVID-19 vaccine records from Immunization Information Systems. We evaluated characteristics of individuals with concurrent COVID-19 diagnosis and vaccination on Day 0. With Optum data, we also estimated VE for alternative negative control outcomes.

RESULTS: The study included 341,097 (Optum) and 1,151,775 (CVS) matched pairs for BNT162b2 and 201,604 (Optum) and 651,545 (CVS) for mRNA-1273. From Day 0-2, 200 (Optum) and 11,953 (CVS) fewer BNT162b2 vaccinated individuals had a claim for COVID-19 lab testing than unvaccinated individuals. There were differences between groups with and without concurrent COVID-19 diagnosis/vaccination on Day 0. For example, 55% (Optum) and 85% (CVS) of vaccinated individuals with concurrent mRNA-1273 vaccination and COVID-19 diagnosis received the vaccine in an office vs. 11% (Optum) and 12% (CVS) without concurrent vaccination/diagnosis; results were similar for BNT162b2. Additionally, 38% (Optum) and 65% (CVS) of individuals with concurrent mRNA-1273 vaccination and COVID-19 diagnosis were from the Northeast vs. 13% (Optum) and 21% (CVS) without concurrent vaccination/diagnosis. For alternative negative control outcomes of urinary tract infections (UTI) and fractures, VE estimates indicated limited or no association; VE estimates for UTI were 4% (BNT162b2, 95% confidence interval [CI]: 1%, 8%) and 3% (mRNA-1273: 95% CI: -1%, 8%), and for fractures were -1% (BNT162b2, 95% CI: -6%, 5%) and 4% (mRNA-1273, 95% CI: -3%, 10%).

CONCLUSIONS: COVID-19 diagnostic testing patterns and co-recording of vaccine and COVID-19 diagnoses may limit the utility of using the 14-day window following vaccination as a negative control outcome in claims-based COVID-19 studies. Future COVID-19 VE studies should consider multiple negative control outcomes to assess potential for residual confounding.