Causality & The Vaccine Adverse Event Reporting System

I spend more time than I care to admit having online debates with people. Recently, I was having a discussion with a mother about the Vaccine Adverse Event Reporting System (VAERS). She was of the opinion that vaccines cause a wide range of things such as autism and death. She argued that all of the reports of adverse events in VAERS were a cause for concern and she seemed to be under the impression that VAERS is a reliable way to establish a causal connection between the vaccine and the reported adverse event. Her logic was that since the CDC (and the FDA) manages VAERS, they are vetting the submitted reports based on quality and hence the reports in its database are reliable. In particular, she was concerned about the Gardasil HPV vaccine and wanted to delay her daughter’s vaccination.

To determine whether VAERS is reliable for determining causality, let’s look at what it is. Why does VAERS exist? In 1986, the United States passed the National Childhood Vaccine Injury Act, which requires health professionals to report any adverse event that may be suspected of being caused by an administered vaccine. In response to this bill, the CDC and the FDA established VAERS in 1990. This is a good thing and certainly favorable compared to having no system in place to report potential adverse events.

VAERS is a passive reporting system. This means that anyone can go online and submit a report, be it doctors, nurses, patients or parents. In fact, when I say anyone, I mean literally anyone. This page of data from the VAERS database shows deaths occurring after administration of the Gardasil HPV vaccine. A quick perusal of the data shows that some of the reports are not being made by people who have witnessed the adverse event, but rather by people who have read about it on the internet:

A consumer reported that he/she read an [sic] internet concerning a female who on an unspecified date was vaccinated with a dose of GARDASIL (lot # , dose and route not reported). On an unspecified date, the patient died of “clot blood” eight hours after vaccination. This is one of several reports from the same source. No further information is available.

There are other sad stories within the data, including a girl dying on a lacrosse field from cardiac failure and a son who committed suicide by hanging himself.

These reports are quite grave and it’s understandable that if someone believes these are legitimate reports investigated by the CDC for quality and causality, that person would certainly be inclined to be concerned about the Gardasil vaccine. Therefore, it’s important to determine whether the CDC is actually determining causality from these reports.

The VAERS database is fully accessible to the public. It turns out the answer to our question is not difficult to find, as there is a full page disclaimer about the reliability of the VAERS database that the user must acknowledge before being given access to the data. I encourage you to read the entire page, but in case you don’t, here are some excerpts:

When evaluating data from VAERS, it is important to note that for any reported event, no cause-and-effect relationship has been established. […] VAERS collects data on any adverse event following vaccination, be it coincidental or truly caused by a vaccine. The report of an adverse event to VAERS is not documentation that a vaccine caused the event. […] VAERS reports can be submitted voluntarily by anyone, including healthcare providers, patients, or family members. Reports vary in quality and completeness. They often lack details and sometimes can have information that contains errors. […] A report to VAERS generally does not prove that the identified vaccine(s) caused the adverse event described.  It only confirms that the reported event occurred sometime after vaccine was given. No proof that the event was caused by the vaccine is required in order for VAERS to accept the report. VAERS accepts all reports without judging whether the event was caused by the vaccine.

vaersdisclaimer

Anti-vaccine advocates often cite VAERS data as evidence that vaccines cause autism. The amount of cognitive dissonance that is required in order for them to click this button must be astronomical.

I suspect the person with whom I was debating never tried to access the data from the VAERS website. If she did, she must have not read or ignored this disclaimer.

Whenever VAERS evidence is cited to “prove” some sort of adverse event, that person is falling prey to the logical fallacy known as post hoc ergo propter hoc. (I covered this in a previous post.) In short, an adverse event occurring after vaccination does not prove that the vaccine caused the event. This fallacy seems lost on many anti-vaccine advocates, as they cite VAERS as a reliable way to determine that vaccines are dangerous.

From Sherri Tenpenny’s “Tenpenny on Vaccines” Facebook page:

>20,000 VAERS adverse event reports have been filed about Gardasil. There were 129 new reports just since 10/1, including 2 deaths. Vaccines are what need to be feared, NOT the “fear” of a temporary illness. You will never question your decision to NOT vaccinate again if you spend some time on this website: www.VAERS.hhs.gov

I could list thousands of other instances of anti-vaxxers invoking the VAERS database as reliable evidence, but it would take an inordinate amount of time and I think you get the idea. In short, the data in VAERS is not reliable for determining causality. The fact that some individuals continue to view it as reliable is a result of confirmation bias combined with post hoc ergo propter hoc.

It should be noted that another argument used by anti-vaccine advocates to make the VAERS data appear more alarming is to point out that VAERS suffers from under-reporting. This is true with any passive reporting system. For example, many adverse events such as swelling at the injection site are likely to not be reported. However, this argument is a non-sequitur, i.e. an argument in which its conclusion does not follow from its premises. The fact that some adverse events are not reported is completely irrelevant when it comes to determining causality. If all of the events were reported, we would simply be left with a larger sample of events from which we cannot reliably determine causality.

So why bother having VAERS when it is not a reliable way to determine causality? The answer is that VAERS is a kind of early warning system. If there are many reports of an adverse event, then researchers can conduct higher quality studies in which confounding factors are controlled for in order to determine whether the event is actually being caused by the vaccine. This makes sense – remember that to avoid falling prey to the post hoc ergo propter hoc fallacy, higher quality evidence must be considered.

In this regard, VAERS has been successful in the past. For example, in 1999, there were many reports in VAERS of intussusception occurring after administration of the rotavirus vaccine RotaShield. Further studies were conducted which confirmed the increased risk and this data led to the subsequent removal of the vaccine from the US market. In another instance, VAERS determined that there was a high incidence of Guillain-Barré syndrome occurring as a result of the meningococcal conjugate vaccine, Menactra, and further controlled studies are underway to investigate the issue.

Finally, the inconsistent logic of anti-vaccine advocates must be highlighted. These individuals often dismiss any information about vaccine safety provided by the CDC because they believe the CDC to have ties to pharmaceutical companies. (The Shill gambit.) Yet they don’t hesitate to cite reports from VAERS as reliable evidence, despite the fact that VAERS is sponsored by the CDC. Basically, they ignore the CDC information that goes against their beliefs and deem CDC information that supports their beliefs to be irrefutable. Once again, confirmation bias reigns supreme.

Don't be fooled by memes such as this. VAERS data cannot reliably tell us this. (Also, change the zero on the left to a one, since an immune-compromised woman recently died from pneumonia caused by the measles (autopsy confirmed.))

Don’t be fooled by memes such as this. VAERS data cannot reliably tell us this. (Also, change the zero on the left to a one, since an immune-compromised woman recently died from pneumonia caused by the measles – this was confirmed by the autopsy.)

So whenever you hear a passionate argument from an anti-vaccine advocate who cites VAERS as evidence that a vaccine caused an adverse event, remember the limitations of the VAERS data and recognize that we need higher quality evidence in order to come to this conclusion.

Experience with Dr. Sherri Tenpenny on Facebook

Recently, I was perusing some anti-vaccination pages on Facebook. I came across the page of Dr. Sherri Tenpenny, quite popular in the anti-vaccine movement. She holds a Bachelor of Arts and a specialization in osteopathic medicine.

Tenpenny posted a link to an article by Sayer Ji of greenmedinfo. The headline proclaims “Study Calls Into Question Primary Justification for Vaccines.” Tenpenny shared the article, saying “Did you know that the basic premise of vaccine effectiveness has been called into question?” The article states:

newly published research has revealed that in some cases no antibodies are required for immunity against some viruses.

[…]

This view, however, has been called into question by the new study:  “Although this concept may apply to other viral infections, our findings with VSV turn this view upside down, indicating that during a primary infection with this cytopathic virus, innate immunity can be sterilizing without adaptive immune contributions.”

Does this strike a mortal blow to the antibody theory which underlies vaccinology, and constitutes the primary justification for the CDC’s focus on using vaccines to “boost” immunity?

The remainder of the article goes on to make a case that scientists have gotten it wrong, that vaccines are largely ineffective at providing immunity to diseases and that the entire basis for how vaccines are supposed to work is utterly wrong.

I must confess, I don’t have a background in biology and therefore I was not sure whether the arguments made by Sayer Ji had any validity. Being a good skeptic, I decided to dig up the original study and read it myself to see whether the representations made by Ji’s article were accurate. Again, my lack of expertise in this area made it difficult to understand the contents of the study itself.

Having no other recourse, I decided to contact the corresponding author of the study. This felt like a long shot; based on my own past experience, authors are sometimes difficult to get in touch with. In some cases, the author has moved on to another institution and ceased using their email address in the process.

The corresponding author for this study is Ulrich H. von Andrian, M.D. and Professor of Immunopathology at Harvard Medical School. The message I sent was the following:

Hello,

I am a blogger and I stumbled upon a recent anti-vaccine article on greenmedinfo here:

http://www.greenmedinfo.com/blog/study-disproves-cdcs-primary-justification-vaccination

The article is saying that your study “B Cell Maintenance of Subcapsular Sinus Macrophages Protects against a Fatal Viral Infection Independent of Adaptive Immunity” calls into question the entire premise upon which vaccines are founded. I am highly skeptical of course, but I confess that I am a physicist and the content of the paper is outside of my knowledge.

I was wondering if you could offer a general comment regarding the article on greenmedinfo.

Cheers

I was pleasantly surprised when I received a response within two hours of this correspondence:

Hi Adam — The online article you referenced below misrepresents our paper.  Our work in no way calls into question the utility of vaccines, which in my personal view are among the most impactful and cost-effective accomplishments of modern medicine.  Our study had examined the immune response to a viral infection of mice that were immunologically naive, i.e. they had never ‘seen’ the virus before.  The animals were able to survive the infection and clear the virus without requiring T or B cells. This per se is not a huge surprise; there are countless invertebrate species that don’t have T or B cells and survive infections every day.  Our observation merely suggests that at least in some settings mammals can do the same.
A key point is that although the mice in our study were able to deal with the infection, they did get infected (i.e. they got ‘sick’ so to speak).  The purpose of vaccines is to prevent infections from occurring in the first place.  Had we vaccinated our mice prior to using the virus, the infection would never have occurrred even if we exposed the animals to a lethal dose of the pathogen.
I hope this is helpful.

Best, Uli

Basically, Sayer Ji’s article misrepresented the original paper and was being used to scare well-meaning parents that happened to stumble upon the article.

Given this information, I decided that it was my duty to share this response with the followers of Tenpenny’s Facebook page. Surely, if an article is said by the authors to misrepresent their study, this must be reliable.

What followed was a very revealing indication regarding the unreliability of Sherri Tenpenny’s page. I posted the authors’ response to the article.

tenpenny

Within 24 hours, my reply was deleted and I was banned from commenting on Tenpenny’s Facebook page. This begs the question: how reliable is this page as a source of information when all opposing viewpoints, no matter how valid, are deleted?

More comical was that some people still defended the article as if they somehow knew better than the study’s authors. Others accused me of making up the story and showed their inconsistent logic in the process. How do they know that Sayer Ji’s article was truthful? This is an example of confirmation bias, in which an individual favors a source of information that agrees with their own belief and they discredit other sources that go against their belief. In this case, Sayer Ji’s article is being favored as an accurate depiction of the research study and my contacting the authors is being discredited. If a third party wanted to know the truth, both Ji’s and my accounts should be taken with a grain of salt, since it is possible to get the truth directly from the study’s authors.

This exercise serves to highlight the unreliability of the information peddled by Sherri Tenpenny. This is a prime example of why skepticism is so important. Without it, claims are taken at face value and beliefs are formed based on inaccurate information.

Anecdotes of “Vaccine-Injured Children” – What Can They Really Tell Us?

If you’re familiar with the information provided by anti-vaccine websites, then you have probably encountered some anecdotal (word of mouth) accounts of children getting injured by vaccines. The story usually goes something like this: a child receives one or more vaccines and then within a short time frame after vaccination, the child experiences an adverse event. Examples of some of these stories include sudden infant death syndromeautism, and others.

My heart goes out to the parents of these children. To some, this may come across as a disingenuous sentiment from an internet blogger. However, the truth is that a member of my immediate family has autism and I have witnessed firsthand, for over two decades, some of the challenges that parents may face following such a diagnosis. I have also experienced a variety of challenges as well.

These anecdotal stories can be incredibly alarming to the well-meaning parent that is trying to decide whether to vaccinate their own child, or to the parent trying to determine whether a vaccine was the cause of their child’s ailments. The end result is that many parents could come away with the impression that vaccines are unsafe and potentially life-threatening.

Reliability

It is certainly understandable that parents might associate an adverse event with a vaccine if these two occur around the same time. What is crucially important, however, is how reliable this temporal (timing) correlation of events is in terms of informing us as to what is true. Can this evidence reliably tell us whether there is a causal relationship between the two events?

Let’s consider the example of the measles, mumps, rubella (MMR) vaccine and autism. Although there is a lot of other evidence on this topic, for now, I am only interested in looking at the temporal coincidence of events and determining what this specific evidence can tell us.

Every year, roughly 4 million children are born in the US. It is estimated that the current rate of autism spectrum disorder (ASD) is about 1 in 68. So every year there are over 58,000 children diagnosed with ASD, which amounts to an average of approximately 160 children diagnosed with ASD every single day.

On average, children are diagnosed with ASD around the age of 4, but signs of ASD can be noticed by parents as early as before the 1st birthday. The vaccination schedule in the US recommends a first dose of MMR at 1 year of age and a second dose at 4-6 years.

Armed with this information, imagine a 6-year calendar (or 6 normal calendars linked together) that represents the first 6 year span of children’s lives. Next, populate that calendar with dots representing the time at which children are diagnosed with ASD. As mentioned earlier, the average rate is 160 per day, so each day on the calendar needs on average 160 dots. Now, children aren’t very likely to be diagnosed in the first few months, so move those dots to a later time in the calendar. For simplicity, let’s assume that all the kids’ parents follow the recommended schedule and give the first dose of MMR on their first birthday and the second dose on their 4th birthday.

vaccalendar

Snapshot of a 2 week span in the calendar. One dot represents one ASD diagnosis on the given day. Is it possible to determine which dots are coincidental and which are not?

With this calendar in mind (marked with dots representing ASD diagnoses and two vaccine dates marked), is it possible that there are dots that happen to fall on the vaccination days, or within one week of vaccine administration? Given the large number of dots, basic statistical probability would say that it is extremely likely that some dots will fall after vaccine administration. In other words, it would be extremely unlikely if there were NO dots following the vaccination days. Given this, it is very likely that we will have some coincidences in timing between some ASD diagnoses and vaccination. This doesn’t mean that those dots are absolutely coincidences, but the simple truth is that based solely on this evidence, there is simply no way to determine which dots are coincidental, if any, and which dots might be due to the vaccines, if any.

This may seem like a somewhat contrived example, but the truth is that one can set it up however they like and the main conclusion is unchanged. We could add or remove vaccines, increase or decrease the number of diagnoses, change the length of the calendar, change the vaccine type, change the adverse event from autism to death or something else. We could even change the distribution of dots so that more dots are clustered around one time compared to others. No matter what, there is just no reliable way, based solely on this evidence, to be certain as to whether the adverse event is causally associated with the vaccine. We cannot separate the coincidental events from the causally associated ones. In order to do this, we need more evidence. We also need more evidence in order to answer questions such as: How many children that received the MMR were diagnosed? How many children that did NOT receive the MMR were diagnosed? Is there a significant difference between these two groups?

Unfortunately, instead of being skeptical and asking themselves these kinds of questions, many parents will understandably fall victim to a logical fallacy when faced with these kinds of vaccine injury stories. They may also fail to recognize that any emotional attachment to their belief, while being completely normal, has no influence on the truth.

Post Hoc Ergo Propter Hoc

Concluding a causal relationship based on succession of events is known in philosophy as a logical fallacy called post hoc ergo propter hocor post hoc fallacy for short. It is a logical fallacy that states “Since event Y followed event X, event Y must have been caused by event X.” For more on this, Paul Henne, a philosophy graduate student at Duke University, has a great video with further explanation.

Inconsistent Logic

Consider two cases:

1) You read a blog post about a child that received a vaccine and subsequently developed autism
2) You read a blog post about a child that received a vaccine and did not subsequently develop autism

If case 1 is sufficient evidence to assume causation, then case 2 must be sufficient as well. Clearly, this logic is inconsistent because it leads us to contradictory conclusions when both cases are considered.

Conclusion

Sadly, many well-meaning parents in the anti-vaccine movement fall prey to this logical fallacy. So from now on, if you hear about someone referring to their “vaccine-injured” child, remember that they are likely coming to this conclusion based on a succession of events and that in order to determine what is really true, we need to be skeptical and consider higher quality evidence.