Vaccines are one of the greatest inventions of modern medicine. It has eradicated smallpox and massively reduced the spread of many childhood infectious diseases, such as measles, mumps, rubella and diphtheria. Yet, vaccines are in some sense victims of their own success. When dangerous vaccine-preventable disease are no longer around, people start to get complacent or have their minds poisoned by anti-vaccine misinformation and skip vaccinations.
Then vaccine-preventable disease come back. Some people can skip vaccinations or have an immune system that cannot react to vaccines without causing outbreaks of vaccine-preventable diseases because they can hide in the herd and be surrounded by people who are not infected or infectious. However, vaccine refusers tend to cluster geographically and when someone imports a case of measles from Europe, that can cause a larger outbreak of measles in the United States. It is also possible for the total fraction of people with immunity to drop below the threshold for herd immunity and allow the spread of vaccine-preventable diseases.
Fear of alleged side effects of vaccines are not uncommon, but it is also being fanned by anti-vaccine activists on the Internet who claims it frequently causes terrible things. But how can you tell truth from bunk when it comes to real vaccine side effects? How can scientists determine if something is a real side effect or if the proposed condition is just as common among those that do not vaccinate? This article provides a brief overview of what constitutes evidence for causation between vaccines and a side effect and what cannot be considered evidence no matter how much people want to believe.
Relying on anecdotal evidence
Some people insist that they or someone they know experienced something after getting a vaccine. This may sometimes be a real cause-and-effect relationship, but often it is just something that happens to occur after getting the vaccine and people think it means that the vaccine caused it. This is often reinforced by pseudoscientific myths, such as the false claim that an inactivated flu vaccine can give you the flu. In reality, there are dozens of diseases that have flu-like symptoms that might feel like the flu to the average person, but is really caused by some other pathogen, such as respiratory syncytial virus, adenovirus, coronaviruses or even the common cold. Anecdotal evidence like this is insufficient to scientifically establish a cause-and-effect relationship between a given vaccine and a particular side effect.
Abusing Vaccine Adverse Events Reporting System (VAERS)
The Vaccine Adverse Events Reporting System (VAERS) is a system where anyone can submit an alleged adverse event after receiving a vaccine. The mere existence of a VAERS report or even many reports is not evidence for a causal relationship. Besides the fact that anyone can report it, it does not contain a control group with information how common something is among those that are not vaccinated and does not constitute a scientific comparison to such a control group. When public health experts look at the medical documents, it often turns out that the most serious adverse events are completely unrelated to vaccines.
A common falsehood pushed by anti-vaccine activists that 108 people has died from measles vaccination since 2003. But these are just the number of VAERS reports that cites death as an alleged adverse event. When experts go over the death certificates, they find other well-understood causes of death, such as co-sleeping or drowning. This anti-vaccine myth has been rated as false by both Snopes and PolitiFact.
There is ongoing monitoring of the VAERS system and suspected cases are being handed over to independent researchers that analyse medical documents or perform large scientific studies to test hypotheses generated by VAERS submissions. VAERS reports on their own is not evidence of a cause-and-effect relationship. Arguing that it is is a method known as VAERS dumpster diving. In fact, the VAERS website itself contains precisely this kind of warning:
Please note that VAERS staff follow-up on all serious and other selected adverse event reports to obtain additional medical, laboratory, and/or autopsy records to help understand the concern raised. However, in general coding terms in VAERS do not change based on the information received during the follow-up process. VAERS data should be used with caution as numbers and conditions do not reflect data collected during follow-up. Note that the inclusion of events in VAERS data does not imply causality.
Relying on single, unconfirmed studies
Another classic method employed by anti-vaccine activists is to cherry-pick single studies that appear to provide evidence for some dangerous side effect, when the bulk of the scientific literature, systematic reviews and detailed investigations carried out by public health experts. Many of these single studies also have a flawed methodology that are often ignored by anti-vaccine activists.
The reason that single studies with flawed methodology cannot be seen as reliable evidence for a causal relationship between a vaccine and an alleged side effect is that they may not represent the mainstream scientific position that would become clear from reading the bulk of the scientific literature. The way science works is that getting published is the start of the scientific conversation, not the end of it. You can find single studies for just about anything, from the false claim that schizophrenia is somehow a form of demonic possession to chocolate consumption boosting the probability of winning a Nobel Prize.
So how can you get knowledge of what side effects a given vaccine actually has that is based on credible scientific evidence? To understand this, we turn to systematic reviews and reports produced by independent groups of medical experts.
Systematic reviews and independent reports by medical expert groups
A systematic review is a paper that has systematically gone over relevant published research on a particular research question and synthesized the results. This kind of study, if done in a methodologically solid way, can provide much stronger evidence than any given study. This is because the noise of single studies with certain kind of biases will drown out by the combined signal of multiple, independently carried out studies. They offer more confidence that a given adverse event truly represents a real vaccine side effect.
However, systematic reviews are not perfect. They are vulnerable to their own problems and biases. For instance, there is a tendency for researchers to have an easier time to publish results that suggest a positive relationship than studies that find no association. This is known as publication bias and can be estimated using a funnel plot where the “missing” studies represents a gap in the graph. Another issue is cherry-picking whereby the authors manipulate their inclusion and exclusion criteria to get precisely the studies they want into their systematic review in order to produce a specific results.
Some systematic reviews are published in scientific journals, whereas others are carried out by independent medical experts at, for instance, National Academy of Medicine. They have produced many large reports that investigate many different kinds of claims of alleged vaccine side effects where they survey hundreds of studies.
Establishing causation is hard. Science is hard. Do not rely on anecdotal evidence or single studies that may be methodologically flawed. Instead, rely on methodologically solid systematic reviews and reports made by independent medical expert groups.