I made a short summary of the rebuttal laid out in The Triumph of Modern Quantitative Science in the comment section on the blog post written by the cultural anthropologist Will over at Skepchick. He decided to write a comment attempting to respond to my criticisms. Unfortunately, his reply can be boiled down to claiming that I have a big ego, asserting that he really has no interest actually debating the arguments, half-a-dozen quotes out of context and denials that he ever made the arguments I am objecting to in the first place. His additional contributions in the comment section was meager. When I explained to another commentator that he or she was misunderstanding my position, that I was in no way, shape or form objecting to having anti-harassment policies and suggesting that he or she should take a step back, take a breath and actually read what I wrote, Will asserted without justification that I was derailing the conversation using emotions.
Clearly, being a skeptic does not imply being rational. I have no interest in interacting with Will any further on this issue, but I will write a final reply for completeness. Note that I will be less charitable in this reply that I usually am.
1. Musings on the size of my ego
Let me start off by saying this: Wow, what an ego on you! That you could think my entire post was all about you! Sorry to disappoint, but it’s not. You’re merely one small piece of what I’m talking about in this post. I’m getting at general trends, not specifically targeting you. If I was doing that, I’d have mentioned you by name and used your exact wording and not said things like “themes I notice in the skeptical community” and “arguments I’ve seen around the skeptical blogosphere.” I also clarified in the comments below the article that these arguments are really amalgamations of arguments I’ve seen/heard over time. I also clarified in the comments below the article that these arguments are really amalgamations of arguments I’ve seen/heard over time.
So, no, this post is not all about you.
I find it peculiar that Will asserts that I am merely one small piece of what he is talking about. Yet he links multiple posts that I was involved in and both posts about racism was specifically written to address my arguments (or straw men of them, as it where). In general, Will’s reply seem to be a clear post hoc rationalization for being called on his multiple misrepresentations. Those arguments are not something he is merely “seem around the skeptical blogosphere”, but specific arguments that I brought up (although he presents them in a twisted form). Furthermore, I never stated that his post was all about me. Needless to say, the size of my ego has nothing to do with the potential soundness (or potential lack thereof) in any of my arguments.
2. Will appears to has little interest in debate
Also, I’m not going to take as much time to respond to any future comments on this thread as I’ve made my point and I will just let respectful disagreements stand and let other commenters respond if they wish. But I felt I should at least respond to you since I did mention you by name and point to a couple of blog posts about you.
In other words, Will does not seem to care that much about actually defending or discussing his assertions. His reply to me seems to primarily be out of courtesy, not a genuine interest to reach the most reasonable conclusions in a friendly conversation. This sets the tone for the rest of the comment he made.
3. Rationalizing the cult accusation.
It’s a rhetorical device. I am not saying there is a literal cult that worships quantitative research. I was using hyperbole and metaphor.
I called him on his vacuous tactic and now he pretends he was never serious. This is going to be a recurring trend for Will throughout his reply.
4. How is it even possible for Will to continue to misunderstand?
Uh, no. I’m pointing out that those are different questions and when people are talking about their experiences of sexism/racism, demands for quantitative evidence are unnecessary at best (and hurtful at worst). And that’s exactly what you did.
I explained my arguments and the justifications for those arguments in detail. Racists where primarily asserting that the problem of racism was minor and therefore negligible. Greta responded by calling for testimonials. I replied that this was an irrational strategy and that quantitative data was the most rational way to refute the racists an said that this data should be given more attention.
Will apparently twists this into me callously demanding quantitative evidence when people are talking about their experiences. Clearly, Will does not even make the slightest attempt to understand what I am actually arguing.
5. First quotation out of context
For example, your exact response to Greta Christina’s call for people to share their personal stories about experiencing racism was: “more scientific evidence, less anecdotes.” So it’s not me that’s trying to re-frame the discussion. It’s you.
This is taken out of context. The question posed by Greta was: “How, exactly, would you like marginalized people to proceed? Is there any possible way we can make oppression and marginalization and bigotry visible, which will meet with your approval?”. Greta specifically asked it as a rhetorical question for those individuals who objected to stories about experienced with the “you are just playing the victim card”. I do not object to marginalized individuals telling their stories. I understand that the context of the post I first responded to may made me appear as if I was and I should probably have used additional qualifiers. Instead, I was fed up with the recurring attempts at refuting racists and misogynists who incorrectly claim that the size of the problem is negligible by asking for testimonials and anecdotes. I felt that this was the wrong way to go, and that false claims about the prevalence of sexism or racism being low is refuted by studies that show that the prevalence is high.
I was trying to shift the focus away from less rational tactics against racists towards what I argue are more rational tactics. In other words, the thing that rationally refute racists is more scientific evidence, and less anecdotes. But the fact remains, Will has incorrectly characterized the nature of the discussion. It was not about the qualitative question of how people experience racism versus the prevalence of racism, but about existence versus prevalence and which of these two were most relevant for refuting racists who believe that the problem is negligible.
6. Second and third quotation out of context
Your own words: “I am insisting that the discussion should be framed around the size of the problem, not merely it’s existence.”
More of your words: “My general point has been that it is much more productive to talk about the size of the problem than if it merely exists or not and that anecdotes tell you nothing about the size.”
Again, I was referring to the fact that presenting scientific evidence that the prevalence of sexism and racism is substantial will rationally refute the assertion, made my misogynists and racists, that the problem is negligible. In other words, I am objecting to the overemphasis on anecdotes, compared with scientific evidence.
7. Another bait-and-switch
So it’s not that I’m “incorrectly” framing the discussion—it’s that I am not framing it the way you want it framed. And you don’t get to decide how marginalized people frame their discussions of social justice.
Will attempts yet another bait-and-switch approach, this time by equivocating what the word “discussion” refers to. There were two levels of discussion: (1) what the best way for skeptics to refute racists are and (2) the particular discussion between me and Greta about my counterarguments. I absolutely agree that I attempted to frame (1) as focusing on size rather than mere existence, but it is Will who has incorrectly framed (2). In other words, he has not grasped what the dispute between me and Greta was about. There is nothing wrong with me trying to frame the discussion about (1) in a different way by providing counterarguments. That cannot be said about Wills false characterization about (2).
8. The first bait-and-switch: deployment number 2
I called Will on his first bait-and-switch where he tried to equate qualitative studies with anecdotes. Apparently he did not really grasp the argument, since he deployed his trick a second time:
Your exchanges with Greta and Ian were just tiny little pieces of a larger mosaic that makes up this obsession with objectivity and quantitative data. I mentioned you by name once in a comment, and it was in response to the question of whether anecdotes are useful data. It is clear to me that you feel that they are not. When you title your blog post “The Plural of Anecdote is not Scientific Evidence,” that’s a pretty clear indication of your feelings about anecdotes.
My argument was that anecdotes by themselves do not represent scientific evidence. However, in qualitative studies, researchers critically investigate, analyze, interpret and publish their data. It is that makes qualitative studies real science, not anecdotes. So the plural of anecdotes (i.e. a bunch of anecdotes) do not in themselves qualify as scientific evidence automatically. However, they can become scientific evidence when investigated, analyzed, interpreted and published by scientists.
9. Fussing over semantics
And speaking of misrepresentation, the only use of the word “impossible” on this page is in your comment. I never used the word impossible. Sure, I could have put a qualifier in my original post: “We cannot advance any real/effective solutions…” Is that not your position?
Actually, paraphrasing an argument does not make it a misrepresentation. Will asserted that the argument was that “We cannot advance a solution unless we know the size and scope of a problem.” and my description of his formulation is not generally problematic.
My position is, and has been, that solutions for a problem needs to be (i.e. should be) informed by the size of the problem. Otherwise, there is a risk of there being a disconnect between the proposed solutions and the actual problem. The example I gave in the discussions at FtB was that it might be the case that the problem is underestimated. Preparing for a flood with an umbrella is a bad idea. Now, that does not mean that you cannot advance solutions if you want. Some may even be approximate and appropriate based on the currently available information. This, however, misses the central point I was trying to make: solutions should be informed by the size of the problem.
10. Understanding the larger context
Her post had absolutely nothing to do with racists who believe racism is negligible. It had to do with the double bind put upon marginalized people.
Actually, the blog post was inspired by the twitter war Greta was having with some racists. Greta herself admitted that some of these put forward the argument that the prevalence is low, so therefore the problem of racism is negligible. Her exact words were:
I don’t know how you’re defining “serious person”… but the whole reason I was collecting stories of instances of racism in the atheist community is that I was in an argument on Twitter with more than one person claiming that examples of racism in the atheist community were outliers or anomalies, and that racism in the atheist community was a trivial problem that could be ignored.
This shows that Will has not spent any considerable time actually trying to understand the situation.
11. Anecdotes and qualitative studies
I stand in awe over the ability Will has of not understanding what is being written, even when he is directly quoting it.
I wrote that “You made it appear as if it was an attack on qualitative studies, which it was not. Qualitative studies with a reasonable methodology transforms anecdotes to reliable data when the scientists are critically analyzing, interpreting and publishing their results.” He replied that:
So then your premise that “the plural of anecdote is not scientific evidence” is wrong then? Anecdotes can serve as scientific data? Or are you maintaining that somehow anecdotes magically become something else when they’re collected in qualitative research?
No, my argument is that a bunch of raw anecdotes by themselves cannot be considered scientific evidence. But if scientists carefully investigate, analyze, interpret and publish their data, the anecdotes are transformed to actual results of scientific research and this research (not the raw anecdotes) becomes scientific evidence. This is not magic, it is science. This is why a carefully carried out case study published in a peer-reviewed scientific journal is more impressive than a story posted by a random person in the comment section of a blog post.
Will then goes on to deny that he defended anecdotes in the first place.
12. More uninformed attacks of quantitative studies
I wrote that: “In quantitative studies, on the other hand, there is operationalization of variables (and therefore an unambiguous way to measure them) and replicates.” Will responds that:
Oh, I see. No subjective bias in quantitative data collection. Operationalization of variables happens in a vacuum free of any other contexts. All variables are readily apparent and visible. Got it. *eyeroll*
No, that is not what I am saying. What I am saying is that if variables have been operationalized, there is exists a consistent and unambiguous way to measure them. This is a reasonable implication because few serious scientist would attempt to operationalize something that could not be measured in a reasonably valid and repeatable way. This does not exclude the existence of biases in the process of operationalization or data collection. It just means that we agree on what we are measuring and how we are measuring it. This is not as prevalent in qualitative studies. What unit do you measure feelings, beliefs and thoughts in? All variables are not apparent and visible, but the one’s we measure are (otherwise we couldn’t measure them in the first place).
Will then launches into an attack against replication, a corner stone in all science by another straw man. I did not claim that exact replication was required, but that replicates in quantitative science increases objectivity and that this is not as prevalent in qualitative studies. He quotes LeCompte & Goetz as stating that:
“Problems of uniqueness and idiosyncrasy can lead to the claim that no ethnographic study can be replicated. However, generation, refinement, and validation of constructs and postulates may not require replication of situations. Moreover, because human behavior is never static, no study can be replicated exactly, regardless of the methods and designs employed” (p. 35).
As we saw, exact replication is not being requested (something that is at any rate impossible in any area of science as there will be random differences). LeCompte & Goetz are wrong in two more respects: even social science studies needs to be replicated. In essence, that is how we know the results are actually robust. Finally, human behavior may not be static, but a lot of psychology research has been replicated without any particular problem, even those that are not strictly lab experiments, so there should be no principle reason why social scientists should not attempt to replicate ethnographic studies. Replication is a fundamental criteria for good scientific practice and no amount of smokescreens and straw men of critics will change that.
In his reply, Will continues to repeatedly misunderstand the argument I was making. He makes claims about my ego, does not appear to be interested in having an honest discussion, rationalize the lack of evidence for his cult accusation by replying that he was never meant it seriously to begin with, quotes me a number of times out of context, launches another bait-and-switch tactic via equivocation, deploys the previous bait-and-switch despite being called on it the first time, has not read enough of the comments in the FtB blog posts to understand the overarching context, misunderstands the relationship between anecdotes and qualitative evidence, and makes a number of ignorant attacks of quantitative studies. Finally, he is dismissive of the need for replication, a cornerstone of modern science.
There is no point interacting with Will on this topic anymore. He does not make an effort at trying to understand the position and arguments of his critics, but merely tries to indiscriminately mow down everything that is being said with a (metaphorical) steamroller. For Will, being right seems to matter more than knowing what is reasonable.