SARS-CoV-2/COVID-19: How can we tell whether a vaccine works?

  • Efficacy in the elderly population for this vaccine was 94%
  • 10 severe cases of COVID-19 occurred, with 9 in the placebo group and 1 in the vaccinated
  • Short-term adverse events from the vaccine candidate were based on 8000 patients and deemed to be generally mild and self-limiting (see below for clarification on terminology) with fatigue and headache being the most common grade 3 adverse events at ~4% and 2% of vaccinees respectively.
  • Of the 95 cases, 5 were in the vaccinated group and 90 were in the unvaccinated group
  • 15 cases occurred in older adults
  • 20 of the cases occurred in individuals from diverse backgrounds: 12 Hispanic or LatinX, 4 Black/African-American, 3 Asian-Americans, 1 multiracial
  • 11 of the cases were severe. The exciting part: all of them occurred in the placebo group.
  • The vaccine candidate itself was generally well-tolerated with severe adverse events being rare and what one would expect from a vaccine i.e. injection site pain, headache, fever, redness, swelling. These were generally short-lived, which bodes well.
  • Severe adverse event and serious adverse event (SAE) are NOT the same thing. Serious adverse events are those which are life-threatening, life-limiting, or medically important in some way (e.g. requiring a prolonged hospital stay). Severe adverse events are not a true regulatory term per se, but generally it refers to a Grade 3 event as defined by the CBER industry guidance.
  • Generally, severe per the trial protocols of both Moderna and Pfizer, refers to the adverse event causing some level of interference with daily life or requires medical intervention. It is not a comment on the duration of that adverse effect which both companies report were generally short-lived. If for instance, you developed a fever that caused significant enough discomfort that you would take acetaminophen to lower it, the fever is considered a severe adverse event. Note that this also does not equate to an truly dangerous outcome (e.g. requiring urgent medical care, hospitalization, or causing prolonged disability), as this would be a grade 4 event.
  • Adverse event and adverse reaction are not synonyms. An adverse event is anything that happens in the course of a clinical trial regardless of whether or not it is caused by the product (adverse events are recorded for the placebo group as well). Adverse reactions are those adverse events which are likely to have a causal relationship to the receipt of the pharmaceutical. For example, if someone in a trial receives a vaccine (or placebo) and subsequently experiences a gun shot wound, this would be documented as an adverse event on the package insert. I cannot conceive of a scenario in which receipt of a vaccine could actually cause a gunshot wound so I will make the epistemic leap that these two things are not causally related. For those who think I am being facetious, this is a real adverse event documented on the package insert for Gardasil (though the individual in question was in the placebo group).
Deaths in the Gardasil vaccine trials. See page 8. Many methods exist to determine whether or not there is a causal relationship between an adverse event and the pharmaceutical in question. Broadly, there are those based on expert opinion, algorithms, and probabilistic methods. There is a good discussion here, but it is paywalled.

But what does it mean for a vaccine to be effective?

Vaccine effectiveness is defined with respect to a given phenomenon. For instance, vaccines can (rarely) protect against infection entirely (see text for details), but measuring this properly is cumbersome and this is not necessary for a successful campaign. With bacteria, vaccines can have some protection against colonization of the pathogen at the relevant anatomical site. Some vaccines are unable to completely shield from disease but can prevent the worst outcomes. In addition to this, vaccines confer protection beyond the vaccinee (see text for details). Halloran M, Longini I, Struchiner C. Design and Analysis of Vaccine Studies. Springer Science; 2010. Table 2.1
DESIGN OF VACCINE EFFICACY TRIALS TO BE USED DURING PUBLIC HEALTH EMERGENCIES — POINTS OF CONSIDERATIONS AND KEY PRINCIPLES. Who.int. Figure 2
N. E. Dean, P.-S. Gsell, R. Brookmeyer, V. De Gruttola, C. A. Donnelly, M. E. Halloran, M. Jasseh, M. Nason, X. Riveros, C. H. Watson, A. M. Henao-Restrepo, I. M. Longini, Design of vaccine efficacy trials during public health emergencies. Sci. Transl. Med. 11, eaat0360 (2019). Figure 1
O’neill R. On sample sizes to estimate the protective efficacy of a vaccine. Statistics in Medicine. 1988;7(12):1279–1288.

Will these vaccines protect against severe COVID-19?

What don’t we know yet?

References

  1. N. E. Dean, P.-S. Gsell, R. Brookmeyer, V. De Gruttola, C. A. Donnelly, M. E. Halloran, M. Jasseh, M. Nason, X. Riveros, C. H. Watson, A. M. Henao-Restrepo, I. M. Longini, Design of vaccine efficacy trials during public health emergencies. Sci. Transl. Med. 11, eaat0360 (2019).
  2. DESIGN OF VACCINE EFFICACY TRIALS TO BE USED DURING PUBLIC HEALTH EMERGENCIES — POINTS OF CONSIDERATIONS AND KEY PRINCIPLES. Who.int.
  3. Halloran M, Longini I, Struchiner C. Design and Analysis of Vaccine Studies. Springer Science+Business Media; 2010.
  4. O’neill R. On sample sizes to estimate the protective efficacy of a vaccine. Statistics in Medicine. 1988;7(12):1279–1288.
  5. Pfizer and BioNTech Conclude Phase 3 Study of COVID-19 Vaccine Candidate, Meeting All Primary Efficacy Endpoints | Pfizer. Pfizer.com. 2020.
  6. Analyzing Phase 3 COVID-19 Vaccine Trials | School of Public Health | University of Illinois at Chicago. Publichealth.uic.edu. 2020.
  7. Dan J, Mateus J, Kato Y, Hastie K, Faliti C, Ramirez S, Frazier A, Yu E, Grifoni A, Rawlings S et al. Immunological memory to SARS-CoV-2 assessed for greater than six months after infection. biorxiv. 2020.
  8. Chu H, Halloran M. Bayesian estimation of vaccine efficacy. Clinical Trials. 2004;1:306–314.
  9. Gilbert P, Janes H, Huang Y. Power/Sample Size Calculations for Assessing Correlates of Risk in Clinical Efficacy Trials. Stat Med. 2016;35(21):3745–3759.
  10. Moderna’s COVID-19 Vaccine Candidate Meets its Primary Efficacy Endpoint in the First Interim Analysis of the Phase 3 COVE Study | Moderna, Inc. Moderna, Inc. 2020.
  11. A Phase 3, Randomized, Stratified, Observer-Blind, Placebo-Controlled Study to Evaluate the Efficacy, Safety, and Immunogenicity of mRNA-1273 SARS-CoV-2 Vaccine in Adults Aged 18 Years and Older. Modernatx.com. 2020.
  12. A PHASE 1/2/3, PLACEBO-CONTROLLED, RANDOMIZED, OBSERVER-BLIND, DOSE-FINDING STUDY TO EVALUATE THE SAFETY, TOLERABILITY, IMMUNOGENICITY, AND EFFICACY OF SARS-COV-2 RNA VACCINE CANDIDATES AGAINST COVID-19 IN HEALTHY INDIVIDUALS. Pfe-pfizercom-d8-prod.s3.amazonaws.com. 2020.
  13. Kucharski A, Eggo R, Watson C, Camacho A, Funk S, Edmunds W. Effectiveness of Ring Vaccination as Control Strategy for Ebola Virus Disease. Emerging Infectious Diseases. 2016;22(1):105–108.
  14. B. D. Kirkpatrick, S. S. Whitehead, K. K. Pierce, C. M. Tibery, P. L. Grier, N. A. Hynes, C. J. Larsson, B. P. Sabundayo, K. R. Talaat, A. Janiak, M. P. Carmolli, C. J. Luke, S. A. Diehl, A. P. Durbin, The live attenuated dengue vaccine TV003 elicits complete protection against dengue in a human challenge model. Sci. Transl. Med. 8, 330ra36 (2016).
  15. Human Challenge Trials for Vaccine Development: regulatory considerations. Who.int. 2016.
  16. Will a small, long-shot U.S. company end up producing the best coronavirus vaccine?. 2020 [accessed 2020 Nov 19]. https://www.sciencemag.org/news/2020/11/will-small-long-shot-us-company-end-producing-best-coronavirus-vaccine
  17. Pinkbook | Varicella | Epidemiology of Vaccine Preventable Diseases | CDC. Cdc.gov. 2020.
  18. Guidelines on clinical evaluation of vaccines: regulatory expectations. Who.int. 2016.
  19. Nauta J. Statistics in Clinical Vaccine Trials. Springer-Verlag Berlin Heidelberg; 2011.
  20. Clemens J. Evaluating new vaccines for developing countries. Efficacy or effectiveness?. JAMA: The Journal of the American Medical Association. 1996;275(5):390–397.
  21. Statistical Considerations for Clinical Trials During the COVID-19 Public Health Emergency Guidance for Industry. Fda.gov. 2020.
  22. Development and Licensure of Vaccines to Prevent COVID-19 Guidance for Industry. Fda.gov. 2020.
  23. Halloran M, Longini I, Struchiner C. Design and Interpretation of Vaccine Field Studies. Epidemiologic Reviews. 1999;21(1):73–88.
  24. Agbabiaka, T.B., Savović, J. & Ernst, E. Methods for Causality Assessment of Adverse Drug Reactions. Drug-Safety 31, 21–37 (2008). https://doi.org/10.2165/00002018-200831010-00003
  25. Iannelli MD V. How Often Should You Do Vaccine Titer Testing? — VAXOPEDIA. VAXOPEDIA. 2019.
  26. Tokars J, Patel M, Foppa I, Reed C, Fry A, Ferdinands J. Waning of Measured Influenza Vaccine Effectiveness Over Time: The Potential Contribution of Leaky Vaccine Effect. Clinical Infectious Diseases. 2020.
  27. Human Challenge Trials for Vaccine Development: regulatory considerations. Who.int. 2016.
  28. Keech C, Albert G, Cho I, Robertson A, Reed P, Neal S, Plested J, Zhu M, Cloney-Clark S, Zhou H et al. Phase 1–2 Trial of a SARS-CoV-2 Recombinant Spike Protein Nanoparticle Vaccine. New England Journal of Medicine. 2020.
  29. Toxicity Grading Scale for Healthy Adult and Adolescent Volunteers Enrolled in Preventive Vaccine Clinical Trials. Fda.gov. 2020.
  30. Benefits of Influenza Vaccination: Selected Publications | CDC. Cdc.gov. 2016.
  31. Addetia A, Crawford K, Dingens A, Zhu H, Roychoudhury P, Huang M, Jerome K, Bloom J, Greninger A. Neutralizing Antibodies Correlate with Protection from SARS-CoV-2 in Humans during a Fishery Vessel Outbreak with a High Attack Rate. Journal of Clinical Microbiology. 2020;58(11).
  32. Gupta, A., Madhavan, M.V., Sehgal, K. et al. Extrapulmonary manifestations of COVID-19. Nat Med 26, 1017–1032 (2020). https://doi.org/10.1038/s41591-020-0968-3
  33. Fajnzylber, J., Regan, J., Coxen, K. et al. SARS-CoV-2 viral load is associated with increased disease severity and mortality. Nat Commun 11, 5493 (2020). https://doi.org/10.1038/s41467-020-19057-5
  34. Lindner D, Fitzek A, Bräuninger H, Aleshcheva G, Edler C, Meissner K, Scherschel K, Kirchhof P, Escher F, Schultheiss H et al. Association of Cardiac Infection With SARS-CoV-2 in Confirmed COVID-19 Autopsy Cases. JAMA Cardiology. 2020;5(11):1281.
  35. Puelles V, Lütgehetmann M, Lindenmeyer M, Sperhake J, Wong M, Allweiss L, Chilla S, Heinemann A, Wanner N, Liu S et al. Multiorgan and Renal Tropism of SARS-CoV-2. New England Journal of Medicine. 2020;383(6):590–592.
  36. Neuroinvasion of SARS-CoV-2 in human and mouse brain. REGENHEALTHSOLUTIONS (RHS). 2020.

--

--

--

I write about vaccines here. You can find me on Twitter @enirenberg and at deplatformdisease.com (where I publish the same content without a paywall)

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Hiring in Post Covid 19 Era | Proquest Background Verification

I am used to seeing patients in kidney failure

Finally discovered the corona vaccine..!!

Lessons From Japan, Rebellious Haircuts, and the Reopening of the Vatican

Experts Share with Experts: Clinical Trials Post Covid-19

Correlates of Protection, a Double-Lung Transplant, and Indonesia’s Expected Baby Boom

It Doesn’t Matter Where COVID-19 Originated

Royal Caribbean To Restart Cruises in June, Require Vaccines For Adults

Royal Caribbean To Restart Cruises in June, Require Vaccines For Adults

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Edward Nirenberg

Edward Nirenberg

I write about vaccines here. You can find me on Twitter @enirenberg and at deplatformdisease.com (where I publish the same content without a paywall)

More from Medium

The Cycle of the ‘Hood

On Buying Local Greens

What is Medicine? An Honest Perspective.

SAVE YOUR CATS FROM THESE ALLERGIC PLANTS BEFORE ITS TOO LATE