Jenny Splitter and Non-Celiac Gluten Sensitivity

Splitter and NCGS

Celiac disease is an autoimmune condition where the immune system reacts against a modified version of a gluten protein that results in a cross-reaction with the small intestine because of similarities in protein sequence. Individuals with this condition can get chronic inflammation, cancer and malabsorption of important nutrients if they do not eliminate gluten from their diet, so this is a very real condition. However, it has spawned a dietary fad and many people who do not have celiac disease have self-diagnosed themselves with “non-celiac gluten sensitivity” (NCGS) and avoid gluten like the plague in a misguided quest for healthy eating. There is no scientific evidence that NCGS exists, but that has not put a dent in the popularity of gluten-free products.

Grounded Parents is a blog about parenting written by parents who are secular skeptics and it is part of the Skepchick Network. Recently, they published a post written by Jenny Splitter defending decisions to eliminate gluten for those who claim to have NCGS. Now, Splitter did not argue that there is scientific evidence for this condition. Instead, she appealed to placebo medicine, ignored the negative consequences of overfitting noise, deployed a classic anti-skeptical trope based on the perfect solution fallacy and even rationalized negative evidence.

Splitter starts of my making two scientifically valid remarks: (1) there is no evidence for the existence of a special sensitivity to gluten apart from celiac disease, (2) gluten-free does not necessarily mean healthy. However, she then deploys the following remarkable argument:

If you stop eating gluten, and physical symptoms improve, have you proven some significant scientific correlation? Of course not. But let’s be realistic. If you’re suffering from something for which doctors have given you no cure, and you find something that seems to work, why wouldn’t you just stick with it? At some point it becomes obnoxious to demand to see the scientific evidence behind every lifestyle choice an individual makes. None of us live in some sort of perfect evidence-based world. Sometimes we all wing it.

There are numerous problems with this paragraph.

Why not just stick with it?

Chronic conditions involve relapsing and remitting events, which means that the condition fluctuates from day-to-day and week to week. Thus, there is considerable background noise that interferes with any causal inference from diet to alterations in symptoms. It is often the case that people think that something thinks causes an improvement or deterioration when it just happens to correlate with the natural ups and downs of the condition (regression to the mean). So chances are, when you have “found something that seems to work” you are just overfitting the random noise of your condition (especially when confirmation bias comes into play and you remember the hits and forgets the misses). In other words, the error rate of this method is probably substantial. This means that it is unlikely to be productive for long and instead lead to excessive anxiety about diet, elimination of unharmful food items (or perhaps even important, healthy items as well), an overconfidence in personal ability to distinguish harmful dietary items from harmless, an overreliance on placebo effects and the illusion of control. A “mistake” (i.e. diet alterations that do not improve symptoms above and beyond placebo or caving and eating one of those eliminated food items) can be interpreted as personal failure to manage a supposedly diet-manageable condition instead of random noise and so could lead to additional self-blame. Thus, sticking with something just because it “seems to work” is not always rational (since the error rate is substantial) and not without negative consequences.

There is one objection that will surely be deployed and that is the “adults should be able to do whatever they want”. However, this is not about what adults should or should not be able to do. Rather, it is about what is reasonable and what is not.

The perfect solution fallacy as an anti-skeptical trope

Another major issue with the paragraph quoted above is that it makes use of a classic anti-skeptical trope: the misinterpretation of scientific skepticism as an obnoxious and intrusive demand for evidence in all aspects of life. In reality, scientific skepticism on a personal level is about not fooling yourself. The trope erects a perfect solution (“demanding evidence behind every lifestyle choice”) and then dismisses the more realistic approach of critical self-examination because it fails to fulfill this imaginary ideal that no one actually holds. Thus, it is a combination of a straw man fallacy and a perfect solution fallacy. The fact that it is not really practically possible to “demand evidence for every lifestyle choice” does not mean that she should refuse to have a skeptical approach in the context of the gluten-free fad.

Generalizability

To drive the point home, let us look at this paragraph again. However, this time we will replace “stop eating gluten” with your favorite alternative “treatment”, such as acupuncture, homeopathy, herbs or reflexology:

If you [start reflexology/acupuncture/homeopathy], and physical symptoms improve, have you proven some significant scientific correlation? Of course not. But let’s be realistic. If you’re suffering from something for which doctors have given you no cure, and you find something that seems to work, why wouldn’t you just stick with it? At some point it becomes obnoxious to demand to see the scientific evidence behind every lifestyle choice an individual makes. None of us live in some sort of perfect evidence-based world. Sometimes we all wing it.

No reasonable person would ever accept this kind of argument for reflexology. Thus, we should not accept this argument for the gluten-free fad.

Rationalizing negative results

It gets even worse in the next paragraph, where Splitter makes use of classic alternative medicine rationalizations of negative study results:

Plus, let’s remember what the gluten sensitivity study actually showed. While it looks like there’s no support for most claims of gluten sensitivity, that doesn’t mean that the symptoms weren’t real in many cases or that gluten was found to be harmless for every condition other than celiac disease. In other words, as always, more research is needed.

A reflexology proponent would rationalize negative results by arguing that although “it looks like there’s no support for most claims of [reflexology], that doesn’t mean that the symptom [relief] weren’t real in many cases or that [reflexology] was found to be [ineffective] for every condition […]. In other words, as always, more research is needed.” Yes, there are many additional complex complications with the gluten situation that is not true for e. g. reflexology. However, no reasonable person would ever accept this kind of special pleading for reflexology. Thus, we should not accept it for gluten-free fad. Yes, we should not automatically dismiss someone without celiac disease who eat a gluten free diet, but we should not so casually stoop to classic alternative medicine / anti-skeptical tropes.

Emil Karlsson

Debunker of pseudoscience.

10 thoughts on “Jenny Splitter and Non-Celiac Gluten Sensitivity

  • February 4, 2015 at 19:23
    Permalink

    Based on the articles linked by Jenny Splitter, a lot of the symptoms commonly attributed to NCGS might be caused by FODMAPs rather than gluten. Low-gluten diets may be effective in reducing symptoms simply because such diets are likely to be low in FODMAPs as well. In particular, bread products are high in both gluten and FODMAPs.

    So on first glance, it may be incorrect to say that people are “overfitting the random noise of [their] condition”. Rather, people are experiencing a real correlation, and simply misidentifying the cause. Given that, I think it could be rational for people to continue low-gluten diets. Although obviously it would be better if people could identify the real reason why the diet works, and thus be able to identify better diets. I am not sure whether products marketed as gluten-free are usually low in FODMAPs, or if it’s just the avoidance of bread which does all the work.

    • February 5, 2015 at 00:19
      Permalink

      So on first glance, it may be incorrect to say that people are “overfitting the random noise of [their] condition”. Rather, people are experiencing a real correlation, and simply misidentifying the cause.

      The correlation between gluten free diets and reduction in symptoms is also a “real correlation”. The fact that they are wrong about the cause does not make the correlation any “less real”.

      The precise nature of the correlation between “eating gluten free diet” and “feeling better” is irrelevant for the accuracy of the claim that people who claim to have NCGS are overfitting noise. When people who claim to have NCGS overfit noise, they are attributing variation (that, in reality, is unexplained variance from the vantage point of mainstream models of gluten sensitivity) to gluten consumption. This is true regardless of what the real source of this variation turns out to be.

      In essence, you are stating a tautology as if it was a dichotomy. It isn’t.

      Whether or not the perceived correlation is due to regression to the mean or FODMAPs as the underlying cause, the alleged causal conclusion tied to the effects of gluten from the correlation between diet and symptoms is not justified.

      Given that, I think it could be rational for people to continue low-gluten diets.

      I think you are confusing “rational” with “happenstance vindication”. Behavior cannot be considered rational or irrational on their own, only in relation to certain goals and underlying reality. An aggressive medical intervention can be rational if the evidence for the underlying condition that the medical intervention treats exists, but it is not rational to go ahead with that medical intervention based on a hunch in the absence of evidence even if (in retrospect and by happenstance), it ended up being the right call.

      Eating low gluten diets based on the goal of reducing gluten consumption under the belief that you have NCGS is irrational. Eating low gluten food based on the goal of reducing FODMAPs under the belief that FODMAPs contribute to the symptoms in a causal manner might very well be rational, but at that point the entire edifice of NCGS has already collapsed.

    • February 5, 2015 at 02:30
      Permalink

      You’re right. All correlations are tautologically “real” correlations, even if they are correlations with noise.

      What I meant is that the correlation is predictable. If someone continued with the same diet or similar, they would continue to experience a reduction in symptoms. There would be no regression to the mean. This is distinct from the hypothesis of “natural ups and downs of the condition”.

      Whether low-gluten diets used to be “rational”, or simply “happenstance vindication” is moot now. The research is in. It appears to have vindicated that typical low-gluten diets are effective, just not because of NCGS. So by my count, Jenny Splitter, who was defending the diet but not defending the NCGS hypothesis, has had her position vindicated. (Whether she used correct reasoning to get to that position is a different matter.)

    • February 5, 2015 at 12:30
      Permalink

      What I meant is that the correlation is predictable.

      All correlations are predictable, regardless of whether there is a causal connection or not. That is the very thing that correlation means e. g. an increase in X follows an increase in Y means that you can predict where X is going by changing Y.

      It appears to have vindicated that typical low-gluten diets are effective, just not because of NCGS.

      To my knowledge, only a single RCT has been done on low FODMAP diet / dietary fructose restriction and it consisted of a non-random sample of 26 people who either had clinically diagnosed IBS or fructose malabsorption (i.e. not unknown conditions that was the subject of the exchange with Splitter).

      Astonishingly, their inclusion criteria consisted of, among others, “instruction at least 3 months before recruitment in the low FODMAP diet […], and marked and sustained global improvement in gastrointestinal symptoms on the diet.” This means that the study had a very biased selection and only included those people for which low FODMAP diet seemed to work, thus loading the dice in favor of the research hypothesis at the outset.

      It is thus too early to make sweeping generalizations about dietary fructose restriction, let alone the tenuous connection between NCGS and low gluten diets supposedly mediated by fructose.

      But let us be charitable by assuming, for the sake of argument, that none of this matters. Then low gluten diet would still be irrational since low FODMAP diets would be better, both in terms of efficacy and match between condition and treatment. So even on the best possible case for your position, low gluten diets would still be irrational in this context.

      So by my count, Jenny Splitter, who was defending the diet but not defending the NCGS hypothesis, has had her position vindicated. (Whether she used correct reasoning to get to that position is a different matter.)

      Jenny Splitter did not “defend the diet”. Splitter defended a particular approach to the management of medical conditions, namely “[i]f you’re suffering from something for which doctors have given you no cure, and you find something that seems to work, why wouldn’t you just stick with it?” I made a detailed refutation to this kind of reasoning, and as far as I can tell, Splitter conceded her position (and you have made no effort to defend it), falling back on the “don’t be a dick when criticizing pseudoscience” approach instead.

      Suggestive happenstance vindication for a position arrived at by flawed and unscientific reasoning does not a reasonable position make. A reasonable position is not merely a position that is true (and I have shown that there are reasons to doubt it), but a true position held for good reasons.

    • February 5, 2015 at 17:51
      Permalink

      Thank you for your assessment of literature on FODMAPs. I did not know it was only one small study.

      One point of disagreement remains, which is predictability of correlations. I do not agree that all correlations are predictable. If I roll a dice five times, and find that the result is correlated with the last digit on my watch, then that is a correlation. However, if I continue to roll the dice, you would not predict the correlation to continue.

    • February 6, 2015 at 12:29
      Permalink

      Thank you for your assessment of literature on FODMAPs. I did not know it was only one small study.

      There are more studies, but those are observational and not RCTs. I did not mean to give the misleading and false impression that the only study ever done was a small RCT. That being said, observational studies are not able to draw causal inferences, so RCTs are probably the relevant study type to consider for this question.

      One point of disagreement remains, which is predictability of correlations. I do not agree that all correlations are predictable. If I roll a dice five times, and find that the result is correlated with the last digit on my watch, then that is a correlation. However, if I continue to roll the dice, you would not predict the correlation to continue.

      Let us do the math on this.

      Assume that there is a correlation between A and B and that the correlation coefficient r > 0. Then to get the coefficient of determination (R2) for this correlation, we simply square the correlation coefficient: r2. R2 can be shown to be the ratio of the explained variance (i.e. the variance of the first variables that can be accounted for by variance of the second variable) to the total variance. As long as R2 is non-zero, the correlation has predictive merit (as there is at least some of the variance is explained by the model). The only way for R2 to be zero (and the first variable therefore lack predictive merit) is if r equals to zero, but then there was no correlation to begin with! So no matter how you look at it, a correlation always have a predictive merit. Of course, it does not have to be a strong predictive merit (after all, correlations can be weak), but predictive merit nonetheless.

    • February 6, 2015 at 19:25
      Permalink

      Emil, earlier I made a distinction between a “real” correlation and “overfitting random noise”. You rejected the distinction. Now you are making a distinction between cases where there is a correlation, and cases where “there was no correlation to begin with” (although there appeared to be a correlation due to insufficient data). That is exactly the distinction I have been attempting to make, and you have rejected multiple possible ways for me to communicate it. I won’t belabor the point further.

    • February 6, 2015 at 22:34
      Permalink

      Emil, earlier I made a distinction between a “real” correlation and “overfitting random noise”. You rejected the distinction. Now you are making a distinction between cases where there is a correlation, and cases where “there was no correlation to begin with” (although there appeared to be a correlation due to insufficient data). That is exactly the distinction I have been attempting to make […].

      The distinction between “correlation” and “no correlation” is not the distinction between your term “real correlation” and my term “overfitting random noise”. Those are different things because a “real” correlation still exists when people are overfitting random noise (whereas you portray these as somehow opposing possibilities). If a “real” correlation (to use your terminology although there is no such thing as “real correlation” above and beyond “correlation”) between gluten-free diets and subjective improvement did not exist, there would be no way of wrongly inferring causation from correlation in the first place.

  • February 9, 2015 at 09:29
    Permalink

    This may be a lame contribution because I have no idea where I read it, but I did read somewhere that gluten avoidance could cause a person’s system to react badly to gluten; ie a self-induced or gluten sensitivity; the statistics I think coming from subjects with sparse availability of gluten in their diets for other reasons, rather. Seems like a logical possibility to me, and not a great idea, if true.

Comments are closed.

%d bloggers like this:

Hate email lists? Follow on Facebook and Twitter instead.

Subscribe!